Agentic AI Architecture and Design Patterns: Why Closed Loops Matter More Than Model Intelligence

TL;DR: A chatbot runs an open loop: it predicts tokens and stops without knowing whether the answer worked. An agentic system runs a closed loop: it senses the environment, plans, acts through tools, observes the result, and verifies what happened before the next step. The gap between a demo and a production agent is not model size. It is architecture: verification gates, recovery loops, and feedback at every important step.

In 2023, researchers at Carnegie Mellon built WebArena, a realistic environment of working websites for e-commerce, forums, software development, and content management. They gave agents long-horizon tasks that look like real web work: find a product, post to a forum, update a site. The best GPT-4-based agent completed 14.41% of those tasks end to end. Humans completed 78.24% (Zhou et al., ICLR 2024).

The model was not ignorant. Ask it to explain the task and it could write a clear essay. The gap was architecture.

Explaining a checkout flow is one thing. Executing fifteen dependent actions against a changing website, where one wrong click changes the state for the next fourteen steps, is a different problem entirely. That is the problem agentic architecture solves.

Tell an open loop from a closed loop

A chatbot is an open-loop system. It takes a prompt, generates a response, and stops. It never checks whether the code compiled, whether the refund posted, or whether the customer's problem was actually resolved.

flowchart LR
    A[User prompt] --> B[Generate tokens]
    B --> C[Output text]
    C --> D[Stop]

An agentic system is a closed loop. It senses the environment, plans an action, executes it through a tool, observes the result, and verifies what happened before deciding what to do next.

flowchart LR
    S[Sense] --> P[Plan]
    P --> A[Act]
    A --> O[Observe]
    O --> V[Verify]
    V -->|Match| S
    V -->|Mismatch| R[Recover or escalate]
    R --> S

The difference is not intelligence. It is feedback. The first system trusts the model's first answer. The second system trusts only what the environment reports back.

A chatbot answers. An agent acts, observes what happened, and decides what to do next. That loop has five stages.

Run the five stages of the agent loop

Each stage has its own engineering job and its own failure mode. Skip one and the loop becomes decoration.

1. Sense. Read the current state: conversation history, customer records, API responses, files, search results, policies. The agent starts from what the environment says now, not what it believed a turn ago.

2. Plan. Pick the next action toward the goal. This can be a single next step, a three-step plan, or a full decomposition with dependencies and fallbacks.

3. Act. Execute through a tool: an API call, a SQL query, a code runner, a message send.

4. Observe. Capture what the tool actually returned. Not what the model expected. Not what the prompt implied. The raw result.

5. Verify and adapt. Check whether the result satisfies the condition for continuing. On success, update state and move on. On failure, retry, correct, replan, escalate, or stop safely.

The model's expectation is not the environment's state. The observation is what connects the two. A status check might return:

{
  "status": "delivered",
  "delivered_at": "2026-08-28"
}

Or it might end the plan:

{
  "error": "ORDER_ALREADY_REFUNDED",
  "retryable": false
}

Both are useful only because the next step is forced to read them.

Trace the loop on one refund ticket

A customer writes:

"My order never arrived. I want a refund."

A weak chatbot answers with an apology and a promise. A closed-loop agent works the ticket:

sequenceDiagram
    participant C as Customer
    participant A as Agent
    participant O as orders API
    participant P as policy API
    participant H as Human reviewer
    C->>A: My order never arrived
    A->>O: get_status(ORD-10482)
    O->>A: delivered 2026-08-28
    A->>O: get_delivery_proof(ORD-10482)
    O->>A: signed_by customer
    A->>P: check_refund_eligibility(ORD-10482)
    P->>A: eligible, limit 5000 cents
    A->>H: refund exceeds automated limit
    H->>A: approve
    A->>C: refund processed

Each action changed what the agent knew about the next action. The status check contradicted the customer's claim. The proof check resolved the contradiction. The eligibility check exposed the policy boundary.

The escalation handed the money decision to a person. Remove any of those observations and the agent either over-refunds or refuses a valid claim. That is why the loop is the system, not the model inside it.

Decide what stays a deterministic workflow

The loop earns its cost only on work that needs judgment. The first design decision is scope: which parts of the job should never involve an agent at all. Fixed paths still carry most of the volume in most operations, and they should stay fixed.

Use a deterministic workflow when every step is known in advance:

flowchart LR
    A[New order] --> B[Validate payment]
    B --> C[Create shipment]
    C --> D[Send confirmation]

Use an agent when the path cannot be specified in advance:

flowchart LR
    A[Customer reports unusual problem] --> B[Determine intent]
    B --> C[Find records]
    C --> D[Investigate]
    D --> E{Choose action}
    E -->|Handle| F[Resolve]
    E -->|Escalate| G[Human review]

The distinction is not hype. A workflow is cheaper, faster, and easier to audit. An agent is necessary only when the environment can change the plan in the middle of the run.

The strongest production systems usually combine both: workflows for the parts that should never vary, agents for the parts that require judgment. For the judgment parts, reliability becomes an architecture problem, not a model-selection problem.

Measure the whole task not the single step

A model that succeeds on 95% of individual steps sounds excellent. Across a twenty-step task, it fails most of the time. The arithmetic is simple and unforgiving: 0.95^20 is about 0.358, or 35.8%.

The effect is even worse as the task gets longer.

That is why single-step accuracy is a demo metric. Whole-task completion is the production metric. An early mistake changes the state in which every subsequent decision is made.

A wrong product ID leads to wrong product details. Wrong details lead to a wrong recommendation. A wrong recommendation leads to a wrong transaction. The longer the chain, the more chances a small error has to become a major one.

A feedback loop interrupts that chain. Without it, errors multiply like this:

flowchart TD
    A[Wrong action] --> B[Wrong state]
    B --> C[Wrong assumption]
    C --> D[Wrong action]
    D --> E[Failure]

With verification, the same run looks like this:

flowchart TD
    A[Action] --> B[Observe]
    B --> C{Verify}
    C -->|Expected| D[Continue]
    C -->|Unexpected| E[Recover or replan]
    E --> A

The architecture does not make the model infallible. It contains the consequences of being wrong.

Single-step accuracy is a demo metric. Whole-task completion is the production metric.

Find the three ways agents fail

Not all agent failures are the same. Treating them as one undifferentiated "the model messed up" is why so many agent deployments stay fragile. Production systems need to distinguish three error classes and put a different gate in front of each one.

Consider a refund again.

• A planning error is refunding before checking whether the order was already refunded.

• An execution error is calling the correct refund API but sending "amount_cents": "$50.00" when the API expects 5000.

• A grounding error is the payment API returning an error while the model tells the user, "Your refund has been processed."

The API knows the truth. The model's text does not. Production systems read the actual state, and they enforce that habit with gates.

Place a validator between the model and the world

The easiest way to improve an agent is often not to replace the model. It is to place a validator between the model and the world.

flowchart TD
    LLM[LLM proposes action] --> G1[Validation gate]
    G1 -->|Invalid| LLM
    G1 -->|Valid| T[Tool executes]
    T --> O[Observed result]
    O --> G2[Verification gate]
    G2 -->|Expected| N[Next action]
    G2 -->|Unexpected| R[Recover or replan]
    R --> LLM

The first gate asks: "Is this action valid?" The second asks: "Did this action produce the state we expected?" Two gates give two distinct protections.

Check before the action runs:

• Schema

• Required fields

• Data types

• Permissions

• Business rules

• Risk level

• Idempotency

• Allowed tools

Check after the action runs:

• HTTP status

• Database state

• Transaction status

• Test results

• Updated records

• Expected side effects

• Policy compliance

The more deterministic the gate, the better. A JSON schema should decide whether an integer is an integer. A database query should decide whether a row exists. A test suite should decide whether the code passed.

Use the model where judgment is needed. Use deterministic systems where deterministic systems are possible.

The same principle decides how much inference compute to spend. Generate more candidates only where a verifier can rank them. Without a verifier, ten candidates are ten guesses. With a verifier, they are ten chances to find the right one.

The five patterns below are where those gates live in practice.

Ground every action in a validated tool

Tools are what allow an agent to affect systems outside its context: databases, payment systems, search engines, CRMs, code runners, messaging platforms, e-commerce APIs, internal services. Tool use, however, introduces a new problem. The model can call the right tool incorrectly.

The four ways tool calls fail

The last failure is particularly dangerous. An agent saying "Done" does not prove that anything happened. The system of record is the authority.

Four rules for safe tool calling

1. Validate every argument. Use strict schemas before execution. Bad input should return a structured error the agent can understand and correct.

2. Make mutations idempotent. If a network timeout causes the agent to retry a refund, the retry should not create a second refund. Use idempotency keys for operations where duplicate execution has consequences.

3. Return machine-readable errors. Instead of "Something went wrong," return a typed object with a code, message, and retry flag.

4. Assign risk levels. Reading an order is fundamentally different from deleting a customer account.

A production tool contract

A strict schema turns the model's proposal into a contract:

{
  "name": "payments.refund",
  "description": "Refund an eligible order",
  "input_schema": {
    "type": "object",
    "required": ["order_id", "amount_cents", "currency", "idempotency_key"],
    "properties": {
      "order_id": {"type": "string", "pattern": "^ORD-[0-9]{8}$"},
      "amount_cents": {"type": "integer", "minimum": 100, "maximum": 50000},
      "currency": {"type": "string", "enum": ["USD", "EUR", "GBP", "INR"]},
      "idempotency_key": {"type": "string", "pattern": "^[a-f0-9]{32}$"}
    },
    "additionalProperties": false
  },
  "safety_tier": "mutation_high_risk",
  "error_contract": {
    "code": "string",
    "message": "string",
    "retryable": "boolean"
  }
}

The model decides what it wants to do. The contract decides whether the proposed action is structurally acceptable. The detail that does the most work is amount_cents as an integer, not a string.

The classic failure is a model passing "$50.00" into a field expecting cents, and a lenient parser reading it as 5000. The schema is where that bug dies.

The model proposes the arguments. The schema disposes.

Once tools tell the truth, the agent can learn from what they say. That is the next pattern.

Give reflection a grounding signal

Reflection is the agent's ability to look at a failed attempt, extract a lesson, and not repeat the mistake. Useful reflection requires evidence, not self-praise.

The Reflexion framework split this into four parts: an actor that executes, an evaluator that scores against a concrete signal, a reflector that compresses the failure into a lesson, and a memory log that carries the lesson into the next attempt. On HumanEval, Reflexion lifted GPT-4 from 80.1% to 91.0% pass@1 without changing a single model weight (Shinn et al., NeurIPS 2023). The lessons lived in text, not gradients.

But reflection has a catch. A model asked to review itself with no external signal often cannot tell a correct answer from a wrong one. The lesson may be noise. The reflection system may actually degrade performance.

A bad reflection contains almost no operational information:

"I should be more careful next time."

A useful reflection contains something the next attempt can use:

"Do not call payments.refund before orders.get_status. The payment service returns ORDER_ALREADY_REFUNDED when the transaction has already been reversed."

The useful version names the relevant tool, the correct sequence, the failure signal, and the behavior to change. That is the kind of rule you can paste into the system prompt.

Research on intrinsic self-correction found that models can struggle to improve their own reasoning without external feedback, and can sometimes make correct answers worse during attempted self-correction (Huang et al., ICLR 2024). The remedy is not more reflection. It is grounded reflection. Good grounding signals include:

• Unit tests

• API responses

• Database state

• Search results

• Calculators

• Compilers

• Policy engines

• Human review

CRITIC showed the same direction: self-correction works when it is routed through external tools like calculators, search engines, and code execution, not when it is routed through the model's own confidence (Gou et al., ICLR 2024).

Never make self-confidence the only source of verification.

The evaluator should be the least intelligent component in the system. Save the model's intelligence for the actor and the reflector. Once reflection is grounded, the next question is how the agent decides what sequence of actions to attempt in the first place.

Match the plan to the shape of the task

There is no universally best planning strategy. The right strategy depends on how predictable the environment is.

ReAct interleaves reasoning and acting, so the next decision depends on the latest observation. On ALFWorld, it beat imitation and reinforcement-learning baselines by a large absolute margin because grounding each decision in fresh observations suppressed the hallucinations that derail pure chain-of-thought reasoning (Yao et al., ICLR 2023).

Plan-and-Solve writes the full roadmap first, then executes it. That works for linear tasks with strict dependencies, like exporting customer records, validating the schema, transforming fields, importing them, and running integrity checks. The risk is that plans become stale. If step two reveals that the data format is completely different from what the agent expected, steps three through five may no longer make sense.

Tree search explores multiple paths and uses a scorer to decide which looks most promising. On Game of 24, Tree of Thoughts reached a 74% solve rate where standard prompting reached 4% (Yao et al., NeurIPS 2023). But search is expensive. You should not use it for a task that is just a single API lookup.

Replan on three triggers only

A production agent should not replan every time anything goes wrong. Endless deliberation is its own failure mode. Three events justify a new plan:

1. An observation invalidates a plan assumption. The plan assumes the order is unfulfilled. The API says it is delivered. The plan is now fiction.

2. A tool returns a permanent error. A timeout does not require a new plan. A permission error probably does.

3. The goal changes. The customer adds a new requirement, or an upstream system updates a constraint the plan was built on.

A plan is a hypothesis about the future, not a contract with reality.

Planning complexity should match task complexity. The same is true of agent count.

Split complex work across specialized agents

A single agent has one context window, one attention budget, and one view of the task. For complex workflows, splitting responsibilities can help. A typical architecture has a supervisor that decomposes the work, specialists that execute it, and reviewers that verify it.

flowchart TD
    S[Supervisor] --> R[Researcher]
    S --> E[Executor]
    S --> V[Reviewer]
    R --> E
    E --> V
    V --> S
    S --> Out[Result]

Frameworks such as AutoGen provide conversation primitives for orchestrating multiple agents, tools, and human inputs (Wu et al., arXiv 2023). MetaGPT passes structured documents between product, architecture, and engineering roles (Hong et al., ICLR 2024). ChatDev pairs programmers with reviewers in chat chains (Qian et al., ACL 2024).

But more agents do not automatically mean more intelligence. Research on multi-agent debate found that multiple model instances proposing and critiquing answers can improve reasoning and factuality on some tasks (Du et al., ICML 2024). The benefit depends on genuine diversity.

Five copies of the same model with the same prompt do not debate. They agree, confidently, on whatever the first one said. Useful diversity comes from:

• Different models

• Different prompts

• Different tools

• Different information sources

• Different roles

• Deterministic validators

• Adversarial reviewers

Five copies of the same assumption do not make five independent checks.

Agent-to-agent communication should not become an endless conversation. A better pattern is a strict contract:

{
  "subtask_id": "sub_4091",
  "assigned_to": "policy_specialist",
  "instruction": "Check whether order ORD-10482 is inside the return window",
  "permitted_tools": ["policy.check_window"],
  "max_turns": 2,
  "output_schema": {
    "eligible": "boolean",
    "policy_clause": "string",
    "days_remaining": "integer"
  }
}

Three fields do the heavy lifting. permitted_tools limits scope creep. max_turns prevents chatter. output_schema makes the result machine-checkable.

A specialist that cannot wander is easier to control, debug, and price.

Autonomy scales well right up to the moment an action cannot be undone. That is where the final pattern draws the line.

Gate irreversible actions behind policy or a human

The first four patterns make an agent capable. The fifth makes it accountable. The rule is simple: the model proposes, and policy or a person disposes.

Not every action deserves the same scrutiny. Gating everything turns human review into a bottleneck that everyone routes around. A better model is three risk tiers.

Two defaults make this safe. Approval requests that time out resolve to deny, never to execute. And when an agent escalates, it should send a decision, not a transcript.

{
  "escalation_id": "esc_8042",
  "user_intent": "Cancel subscription and refund annual charge",
  "completed_steps": [
    "Identity verified via OAuth session",
    "Renewal confirmed three days ago",
    "Payment method validated"
  ],
  "blocker": "Refund exceeds automated approval limit",
  "requested_decision": "Approve or deny the refund to the original Visa card"
}

The reviewer should immediately know what the user wants, what the agent already did, what blocked automation, and what decision is required. That is the difference between human-in-the-loop and simply handing the problem back to a human.

Assemble the six layers of a production agent

A reliable agentic system can be drawn as six layers. The LLM is one of them. It is not the source of truth.

flowchart TD
    subgraph Goal
        G[What should be accomplished]
    end
    subgraph Reasoning
        R[Agent / reasoner plans and chooses actions]
    end
    subgraph Policy
        P[Policy / guardrails permit or deny]
    end
    subgraph Tools
        T[APIs · databases · search · code · apps]
    end
    subgraph Observation
        O[Observation layer captures what happened]
    end
    subgraph Recovery
        Rec[Verification / recovery decides next step]
    end
    G --> R
    R --> P
    P --> T
    T --> O
    O --> Rec
    Rec -->|Continue| G
    Rec -->|Retry| R
    Rec -->|Escalate| H[Human reviewer]

Teams that do not want to hand-build every layer can start from a platform where the loop already exists. YourGPT runs the agent as a visual flow with validation between steps and an Escalate to Human action for the gate layer. That is the only mention in this post; the rest of the architecture is platform-agnostic.

Trace one ticket through all five patterns

The five patterns are not separate features. They compose inside one run. Here is the same refund ticket, with the pattern doing the work at each step:

flowchart TD
    A[TICKET: I never got my order] --> B[SENSE: extract ORD-10482]
    B --> C[PLAN: check status → policy → refund]
    C --> D[ACT: orders.get_status]
    D --> E{Schema validation P1}
    E -->|PASS| F[OBSERVE: delivered]
    F --> G[DELEGATE: policy specialist P4]
    G --> H[OBSERVE: eligible]
    H --> I[ACT: payments.refund]
    I --> J{Risk gate P5}
    J -->|BLOCKED| K[REFLECT: do not retry, escalate P2]
    K --> L[ESCALATE: structured brief P5]
    L --> M[Human approves]
    M --> N[RESUME: refund executes]
    N --> O[VERIFY: read payment state P3]
    O --> P[RESPOND: notify customer]

Notice what the model was not allowed to do. It was not allowed to assume a tool succeeded. It was not allowed to execute a high-risk mutation just because it generated valid JSON. It was not allowed to retry a blocked action.

It was not allowed to treat its own explanation as proof. The model provided intelligence. The architecture provided control.

Let the loop learn without retraining

Most production agents do not update model weights after every interaction. They improve through the operating procedure, not the parameters. A useful loop looks like this:

flowchart LR
    A[Run] --> B[Observe outcome]
    B --> C[Identify failure]
    C --> D[Classify failure]
    D --> E[Store lesson]
    E --> F[Update policy / memory / workflow]
    F --> G[Evaluate again]

The improvements come from:

• Episodic memory

• Structured feedback

• Tool outcomes

• Updated policies

• New examples

• Error logs

• Evaluation results

• Prompt or policy updates

• Retrieval from previous successful trajectories

The system becomes better because its operating procedure improves, even if the underlying model stays the same. That is often the more practical form of agent improvement.

Benchmark consistency not demos

Traditional chatbot evaluation asks whether the answer sounded right. Agent evaluation asks whether the world ended up in the right state. Newer benchmarks focus on interaction, tools, state, and task completion.

WebArena showed the gap early: 14.41% for the best GPT-4-based agent against 78.24% for humans on the same long-horizon web tasks. tau-bench introduced pass^k, which measures whether an agent can reliably complete the same task repeatedly. Even a strong function-calling agent can drop below 25% on pass^8 in retail, because consistency is harder than a single lucky run (Yao et al., ICLR 2025).

SWE-bench shows how much scaffolding matters. Frontier systems now resolve a majority of the human-verified SWE-bench Verified split, but the leaderboard is as much about the agent framework, tools, and verification loops as it is about the base model (Jimenez et al., ICLR 2024).

Track the metrics that predict production

A single benchmark score can hide operational unreliability. An agent that succeeds 80% of the time passes four consecutive runs only about 41% of the time (0.8^4 ≈ 0.41). A single impressive run is a demo. A boring streak of passes is a product.

Track these:

• Task completion rate

• Failure rate

• Retry rate

• Recovery rate

• Escalation rate

• Tool error rate

• Invalid tool-call rate

• Average steps per task

• Cost per successful task

• Latency per successful task

• Repeated-run consistency

• Safety violations

The most important metric is cost per successfully completed task. Not cost per token. Not cost per API call. Not cost per conversation.

If a cheap agent fails 40% of tasks and needs human rework, while a more expensive agent completes 95%, the cheaper one is often the more expensive one at the business level.

Frequently asked questions

Ship an agent that survives being wrong

Before shipping an agent that can take meaningful actions, verify five areas:

1. Measure whole-task completion, not single-turn accuracy. The product of per-step success is the number users actually experience.

2. Validate every tool call. Use schemas, permission checks, business rules, and structured errors.

3. Verify important side effects. Do not assume a successful-looking tool call means the world changed. Read the relevant state back.

4. Gate high-risk actions. Reads can be autonomous. Reversible writes need controls. Irreversible actions need policy or human approval.

5. Test repeated runs. A lucky success is not reliability. Run tasks repeatedly and track pass^k-style consistency.

The models will keep improving. They will reason better, use tools better, and handle larger contexts. But none of that eliminates the underlying systems problem.

A model that is 95% reliable on an individual step can still be unreliable across a long workflow. A model can generate the correct API arguments and still misinterpret the response. A model can produce a correct plan and still encounter an environment that invalidates it.

That is why the defining question for production agents is not how intelligent the model is. It is what the system does when the model is wrong.

Reliability is not something you get by choosing a smarter model. Reliability is something you architect.