What a self-hosted AI agent actually is, and the two meanings

The distinction stopped being a hobbyist preference somewhere between AutoGPT's publication on 30 March 2023 and the EU AI Act entering into force on 1 August 2024, with general-purpose model obligations applying from 2 August 2025. "Where does the prompt go" is now a question with a legal answer attached, and the two architectures below answer it differently.

Where the term came from

The "agent" half is old. Multi-agent systems were an academic field through the 1990s — the first International Conference on Autonomous Agents ran in 1997 — long before anyone had a transformer to put inside one. The "self-hosted" half came out of web hosting, where it meant running WordPress on your own box instead of wordpress.com.

The two words fused in March 2023, when AutoGPT was published. AutoGPT was self-hosted by construction: you cloned a repository, pasted your own OpenAI key into a .env file, and the loop ran on your laptop. Nobody called it self-hosted at the time, because there was no hosted alternative to contrast it with. Then the agent platforms shipped, the contrast existed, and the retronym stuck.

By November 2024 the tool-access layer had a standard too — Anthropic published the Model Context Protocol on 25 November 2024. That matters here. MCP is what lets a self-hosted loop talk to your internal systems without a vendor sitting in the middle of the call.

What it is not

Four neighbouring terms get used as synonyms. They are not synonyms.

A local LLM is not an agent. Running ollama run llama3.1 gives you inference on your own hardware. That is a model answering a prompt. An agent is a loop: observe, decide, call a tool, observe the result, decide again, with state that survives between turns. Ollama is one component an agent might use. By itself it has no loop, no tools, no memory, and no stopping condition.

On-premises is not self-hosted. On-prem means a vendor's container running in your VPC, on the vendor's release schedule, phoning home for licence checks and telemetry. You host the bytes. You do not control the software.

Open-source is not self-hosted. Open-source describes the licence of the code; self-hosted describes where the process runs. You can run open-source agent code on someone else's cloud, and you can self-host something that is not open-source at all. n8n is the clean example: widely self-hosted, shipped under the Sustainable Use License, which is not an OSI-approved open-source licence.

BYOK is not self-hosted. Bring-your-own-key means you pay for inference directly. The orchestration, the retry logic, the prompt and the trace all still live with the vendor.

The two meanings, labelled

People use "self-hosted AI agent" for two materially different architectures. Settle which one you mean before the procurement conversation, not during it.

Sense A is what most teams mean and most blog posts describe. Sense B is what most buyers think they are buying. If someone says "self-hosted agent" and you do not know which one they mean, the conversation is about nothing yet.

A build that qualifies

A Sense B agent that does nightly competitor-page summarisation is four containers in one docker compose file.

The loop fires on a schedule, fetches pages, embeds and stores them in Qdrant, asks the local model to diff against last week's summary, and writes both the output and every intermediate step into Postgres. Because the trace is in Postgres, "what did the agent do on the 14th" is a SQL query, not a support ticket.

Sizing is the part teams underestimate. Through Ollama, llama3.1:8b is roughly a 4.7 GB download at 4-bit quantisation and runs on a 16 GB machine; llama3.1:70b is around 40 GB of weights and wants either two 24 GB consumer cards or one 48 GB-plus datacentre card. A second default worth changing: Ollama serves requests with a 2,048-token context window unless num_ctx is raised, so a retrieval prompt stuffed with six competitor pages silently loses most of them while the run still reports success.

One documented footgun in exactly this stack: Ollama binds to 127.0.0.1:11434 and has no built-in authentication. Teams needing access from another container set OLLAMA_HOST=0.0.0.0, and if the host firewall is loose they have just published an unauthenticated endpoint that executes arbitrary prompts and pulls arbitrary models. Bind it to the Docker network, not the internet.

For the coding-agent version of this stack, see self-hosted AI coding agents: what to actually run.

How to tell whether something qualifies

Five tests. Four passes is Sense A; five is Sense B.

1. Cut the vendor network. Does the loop still start? If inference is a hosted API, it will not. That is fine — it just means Sense A, and you should stop telling auditors otherwise.
2. Can you read the full trace in a database you own? Every tool call, every retry, every token count. If the trace lives in a vendor dashboard, you have observability on loan.
3. Can you pin a version and refuse an upgrade? An image tag pinned to a digest is self-hosting. A container that auto-updates on the vendor's schedule is not.
4. Do you hold the secrets? If the agent's database credentials are pasted into a vendor's integrations UI, the vendor holds them.
5. Is the step cap enforced outside the model? Unbounded loops are the most common production failure mode for autonomous agents — the agent decides to keep going and keeps paying for the privilege. AutoGPT shipped a --continuous mode in 2023 whose own documentation warned against running it unsupervised, and the warning has aged well. The fix is a hard iteration ceiling in the orchestrator, not an instruction in the prompt. A model asked to limit itself is not a control.

Self-hosting buys you the audit trail and the kill switch. It does not buy you judgement, and it does not make the agent safe to hand a credit card. Delegating spend authority to an autonomous agent remains an open problem across the field, self-hosted or otherwise — control over where the process runs is a different problem from control over what it decides.