plaination Xplaining Tomorrow Today
The Six Layers of AI Agents: What Actually Holds Together in Production
← Back to all articles
AI Jun 17, 2026 · 8 tags

The Six Layers of AI Agents: What Actually Holds Together in Production

MCP won. Memory is a first-class primitive. The six-layer agent stack that replaces the 2024 diagrams — and why most teams overcomplicate it from day one.

#AI agents#MCP#memory#agent architecture#O'Reilly#Paolo Perrone#production#tool protocols

Your LLM Stack Is Not an Agent Stack

A chatbot that calls one API is not an agent. It never was. An agent is a think-act-observe loop: the model reasons about a task, takes an action, observes the result, and loops until the job is done. Everything else is plumbing.

A chatbot needs inference and maybe retrieval-augmented generation. An agent needs state management, tool protocols, persistent memory, and guardrails. That is a completely different infrastructure problem.

The Old Diagrams Are 14 Months Out of Date

Letta published an AI agents stack diagram in November 2024 that became the default reference everywhere. That diagram is 14 months old. MCP didn’t exist yet. Memory was a vector database subset. Nobody was shipping provider-native SDKs.

The stack has six layers in 2026, based on Paolo Perrone’s updated analysis on O’Reilly Radar:

Layer 1: Models and Inference

Run the model via API, managed open-weight provider, or self-host. This layer is commoditizing — model differences matter less each quarter. The real decision is cost versus latency. Prototype on closed-source, deploy on open-weight. The prototype-to-production gap here is the smallest of any layer. Interlocking brass keys sliding into matching recessed groov

Layer 2: Protocols and Tools

This layer didn’t exist in 2024. Every framework had its own JSON schema for tool definitions. Now MCP is the standard with adoption by OpenAI, Google, and Microsoft, plus a Linux Foundation donation. The protocol debate is over.

MCP won. The remaining question: how to lock down MCP servers before exploitation. Endor Labs analyzed 2,614 MCP servers and found 82% prone to path traversal and 67% to code injection. The attack surface is new even if the exploits are classical.

Layer 3: Memory and Knowledge

Where most teams get stuck. In 2024, memory meant “pick a vector database.” In 2026, memory is a first-class architectural primitive with three distinct tiers. Bigger context windows didn’t kill memory — they changed the trade-off between in-context and on-demand retrieval.

Mem0’s State of AI Agent Memory 2026 benchmarks 10 memory approaches. The standard benchmarks: LoCoMo (1,540 questions, four categories), LongMemEval (500 questions, six categories), and BEAM (tests memory at 1M and 10M token scales — volumes no context window expansion can solve). A polished river stone resting flat, casting a sharp grid-li

Layer 4: Agent Frameworks

This is where you orchestrate the think-act-observe loop. The landscape splits into framework-agnostic tools and provider-native SDKs. Lock-in risk is real — every framework makes different assumptions about failure handling.

Layer 5: Evaluation

This is where the prototype-to-production gap is widest. A demo agent handling a handful of prompts beautifully is nothing like running one across thousands of users. Evaluation in 2026 measures more than accuracy — token consumption, latency, guardrail adherence, and recovery all matter.

Layer 6: Guardrails and Safety

The least mature layer, the one that matters most when things break. Guardrails constrain what an agent can do in real time. Most teams underestimate this until a production incident proves them wrong.

The Bottom Line

Start thin. Add complexity when something breaks. Most teams overcomplicate memory from day one — start with conversation history in Postgres. Add vector search when history exceeds context limits. Add agentic memory only when your agent needs to learn across sessions.

The six-layer stack is a map, not a prescription. Invest in the layer where you feel the prototype-to-production gap most. A tangled knot of dried vines slowly unraveling into six per

Quiz

Question 1: What percentage of MCP servers had path traversal vulnerabilities (Endor Labs)?

Answer: 82%, out of 2,614 analyzed.

Question 2: Name the three AI agent memory benchmarks in 2026.

Answer: LoCoMo, LongMemEval, and BEAM. Six frosted glass shelves aligned vertically, each holding a

Question 3: What is the recommended model selection pattern?

Answer: Prototype on closed-source, deploy on open-weight.

Sources

Watch the full lesson