How AI Agents Work: The Agent Loop, Tools, Planning and Memory

From answering to acting

A plain LLM turns text into text. An AI agent uses an LLM to pursue a goal — "fix this failing test", "find three suppliers and compare prices", "book a meeting with everyone free on Friday" — by taking actions in an environment over many steps. Coding agents such as Claude Code, Codex, Gemini CLI, Goose and OpenCode are today's most widely used examples.

The agent loop

At its core every agent is the same loop:

flowchart TB
  G["Goal + instructions + available tools"] --> T["LLM: think about next step"]
  T -->|"tool call"| A["Execute tool (code, search, file, API, browser)"]
  A --> O["Observation: tool result or error"]
  O --> C["Update context / memory"]
  C --> T
  T -->|"done or needs a human"| F["Final answer / hand-off"]

This is the ReAct pattern (Yao et al., 2022): interleave reasoning ("the test fails because the date is parsed in UTC; let me open the parser") with acting (a tool call) and observing the result. Earlier work such as Toolformer (Schick et al., 2023) showed models can learn when to call tools; today, models are trained to emit structured tool calls natively (Tool Calling and MCP).

In code, a minimal agent is surprisingly short:

messages = [system_prompt, user_goal]
while True:
    reply = llm(messages, tools=TOOLS)          # model may request tool calls
    messages.append(reply)
    if not reply.tool_calls:                     # no more actions → final answer
        break
    for call in reply.tool_calls:
        result = run_tool(call.name, call.args)  # sandboxed, with permission checks
        messages.append(tool_result(call.id, result))

Everything else — planning, memory, safety, evaluation — is about making this loop reliable.

The building blocks

Planning. Agents perform better when they write and update an explicit plan, and when they reflect on failures — Reflexion (Shinn et al., 2023) stores verbal lessons from failed attempts and retries.

Memory. The context window is short-term memory and fills up quickly with tool output. Agents summarise, write notes to files, or retrieve from long-term stores; Generative Agents (Park et al., 2023) combined a memory stream with retrieval by recency, importance and relevance. Deciding what goes into the window at each step is often called context engineering.

The environment matters as much as the model. SWE-agent (Yang et al., 2024) showed that giving a model a purpose-built agent–computer interface — concise file viewer, search and edit commands with clear feedback — substantially improved its results on SWE-bench compared with a raw shell.

Workflows vs agents

Anthropic's widely cited guide (Building effective agents) distinguishes:

• Workflows – LLM calls orchestrated along predefined code paths: prompt chaining, routing, parallelisation, orchestrator–workers, evaluator–optimiser.
• Agents – the LLM dynamically directs its own process and tool use.

Its advice: start with the simplest thing that works, and add autonomy only when the task genuinely needs flexible, multi-step decisions — agents cost more tokens, are slower and can compound errors.

Why agents fail

Benchmarks on real tasks (see Evaluating Agents) and practitioner reports point to recurring failure causes, many of them in the software layer rather than the model:

• Tool selection: the right tool is missing, or too many similar tools confuse the model.
• Tool descriptions and errors: vague parameter docs, unhelpful error messages, outputs too large for the context.
• Environment: missing dependencies, authentication walls, CAPTCHAs, changing GUIs.
• Long horizons: small error rates per step compound over dozens of steps; context is lost or polluted.
• Security: content the agent reads can contain instructions that hijack it (Prompt Injection).

This is why the software an agent calls — and how well it is described, permissioned and tested — matters so much, and why this site documents software from an agent's point of view.

Next

Learn the mechanics of tool use in Tool Calling and MCP, then continue with Planning and Memory, A2A and Multi-agent Systems and Sandboxes, Browsers and Computer Use (in progress).