From Copilots to Delegates: Designing Software for Long-Running AI Agents

Inline coding assistants and conversational chatbots operate under a synchronous session model. The user types a query, the system holds the connection open, issues a prompt to a language model, and returns a single stream of text. The state of this interaction is ephemeral, living and dying with the active HTTP request or WebSocket connection.

Autonomous delegates break this session model. A delegate is tasked with a high-level goal—such as resolving a repository issue or reconciling monthly vendor invoices—that may require dozens of model reasoning loops, external API calls, and sandboxed test executions. These tasks can run for minutes, hours, or even days.

To build reliable delegates, we cannot rely on persistent runtime memory. Server upgrades, network timeouts, and container recycling will inevitably terminate long-running processes. We must shift from memory-bound session loops to durable background execution systems.

In a traditional software architecture, an agent is often written as a simple loop: while task not complete, select tool, run tool, update context. If the server restarts mid-loop, the state of the agent is lost, and the next run starts from scratch, wasting tokens and risking duplicate mutations.

Instead of storing the current state of the agent as a mutable database row, we should store it as an immutable sequence of historic events. This is the event sourcing pattern. When a background worker resumes a task, it reconstructs the agent's state by replaying these events from the database.

Every phase transition—creating a plan, selecting a tool, receiving tool outputs, encountering schema validation errors, and receiving human confirmation—is appended to this journal. This architecture ensures total auditability and guarantees that side effects are executed exactly once.

Replaying every single model generation and tool execution from scratch on restart is computationally expensive and slow. To optimize recovery, the system must write periodic state checkpoints. A checkpoint represents a flattened snapshot of the agent's working memory, active environment tokens, and planned execution graph.

When designing these checkpoints, strict serialization is mandatory. The entire state envelope must be serialized into strongly-typed schemas (such as Pydantic models in Python or TypeScript interfaces) and saved to a durable transaction database like PostgreSQL or Redis.

This boundary forces the agent to serialize its internal memory state explicitly at the end of every tool step. By persisting this boundary on disk, we separate the stateless LLM reasoning worker from the stateful execution engine.

A durable agent task must define explicit interfaces for state serialization, execution phases, and replay triggers. The example below details a TypeScript schema for an event-sourced agent loop that can be suspended and resumed across processes.

Long-running agents execute actions inside external systems. Mutating tools, such as running a shell command or making a third-party API post request, are major failure vectors. A network drop, an infinite loop in generated code, or an API rate limit will block a stateless agent loop.

We isolate these failures using three patterns: Ephemeral sandboxing, Timeout circuit breakers, and Exponential backoff retry with jitter.

Mutating shell commands must execute inside ephemeral container sandboxes (such as Docker or firewalled Kubernetes pods) to prevent system-level damage. Every tool execution must have a strict timeout budget (e.g., maximum 30 seconds). If a tool fails or times out, the circuit breaker trips, the failure is written to the event log, and the agent is forced to re-plan or request assistance.

High-stakes actions—like issuing payments, deleting data, or merging code to production—must not be executed autonomously. We require Human-in-the-Loop (HITL) authorization.

An approval gate is not an afterthought implemented at the UI layer. It is a core state transition in the agent's event loop. When a task reaches a protected tool step, the execution loop suspends itself, serializes the current checkpoint, writes an `APPROVAL_REQUESTED` event, and alerts the human supervisor.

Once the human approves or rejects the payload, the supervisor writes a signed `APPROVAL_GRANTED` event. The task queue picks up the suspended task, replays the audit trail, verifies the cryptographic signature of the approval event, and resumes execution.

Verify these architectural boundaries before deploying autonomous delegates to production environments.