Agent Observability: Monitoring AI Agents in Production
AI agents fail silently, returning confident, wrong answers with no error thrown. Here's how teams are building observability to actually catch it.
The GetCoreTech Team Sep 13, 2026 · 9 min read
What Is MCP (Model Context Protocol)? How Anthropic's Open Standard Became the "USB-C for AI Agents" — And Why It Just Rebuilt Itself From Scratch
MCP — the Model Context Protocol — is the open standard that lets AI models call external tools, read files, and query databases through one consistent interface instead of a custom integration for every model-tool pairing. Anthropic released it on November 25, 2024; by mid-2026 its official SDKs were pulling in close to half a billion downloads a month, with both the TypeScript and Python SDKs having crossed one billion total downloads, and OpenAI, Google, Microsoft, and AWS have all built native MCP support into their own platforms. In July 2026, the protocol's maintainers shipped its biggest architectural change yet — ripping out the session-based handshake that defined MCP since launch — a sign of how fast "explain the basics" and "explain what's currently happening" have become the same article.
What MCP Actually Solves
Before MCP, an AI application that wanted to check a calendar, query a database, or edit a file needed a custom integration written specifically for that model and that tool. Every new tool meant re-doing that wiring for every model you wanted to support — an M×N combinatorial problem. MCP standardizes the connection itself: a tool (like a GitHub repo, a Postgres database, or a design app) exposes an "MCP server" that describes what it can do, and any "MCP client" — Claude, ChatGPT, a custom agent — can talk to it the same way. Build one server, and every compliant client can use it; build one client, and it can use every compliant server.
How It Works, Concretely
An MCP server exposes three main things: tools (actions the model can call, like create_ticket or run_query), resources (data the model can read, like a file or a database row), and prompts (reusable templates). The client — the AI application — discovers what's available and lets the model decide when to call something, based on the user's request.
Under the hood, MCP is built on JSON-RPC 2.0, and until mid-2026 it ran over a stateful transport: a client opened a session with a server, and that session persisted across a conversation. That design choice is exactly what changed in the protocol's most recent overhaul.
The July 2026 Rewrite: MCP Goes Stateless
On July 28, 2026, MCP's maintainers released spec version 2026-07-28, and its headline change is a stateless protocol core — transforming MCP from a bidirectional stateful protocol into a request/response stateless one. Concretely, the maintainers officially retired the initialize/initialized handshake and the Mcp-Session-Id header, meaning any request can now land on any server instance behind a plain load balancer without shared session storage — a change aimed squarely at running MCP servers at production web scale rather than as long-lived local processes.
The same release added a few other structural pieces worth knowing if you're building on MCP today. Multi Round-Trip Requests (MRTR) replaces server-initiated requests like elicitation and sampling that previously required a held-open stream — a server can now ask a stateless client for missing input and get an answer back on a retry, rather than keeping a connection open. Header-based routing puts method and tool names in dedicated HTTP headers so gateways can route and authorize on headers directly instead of parsing every JSON body. Cacheable list results let clients cache a server's tool catalog instead of re-fetching it on every reconnect. Authorization hardening includes RFC 9207 issuer validation and a formal shift away from Dynamic Client Registration toward client ID metadata documents.
A twelve-month deprecation window is now official policy, and Roots, Sampling, and Logging are deprecated in favor of the newer extension model, though they'll keep working for at least a year. Cloudflare, Google Cloud, Microsoft Foundry, and AWS Bedrock AgentCore all confirmed same-day support, which tells you this wasn't a proposal — it was a coordinated ecosystem rollout.
MCP Isn't Anthropic's Anymore
The other structural shift happened earlier, on December 9, 2025: Anthropic donated MCP to a new Agentic AI Foundation (AAIF), a directed fund under the Linux Foundation, alongside Block's goose agent framework and OpenAI's AGENTS.md. The foundation's platinum membership tier includes AWS, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft, and OpenAI — meaning direct AI competitors now co-govern the protocol's roadmap rather than any single vendor controlling it unilaterally. That matters practically: it's the reason enterprise procurement teams who previously hesitated over single-vendor lock-in now have a neutral-governance story to point to, similar to how Kubernetes moved from a Google project to genuine multi-vendor stewardship.
The Adoption Numbers, Checked Against the Source
Adoption claims around MCP get inflated easily, so it's worth sticking to what's actually traceable. Anthropic's own SDK download figures and the official MCP registry are primary-source; broader "X% of enterprises use MCP" style claims mostly are not. What holds up: the official MCP registry counted 9,652 latest server records and 28,959 total server/version records as of a May 24, 2026 snapshot. The AAIF grew to 146 total member organizations by February 24, 2026, adding 97 new members in its first roughly eleven weeks — among the fastest-growing efforts in Linux Foundation history. Client-side support arrived well before governance did: OpenAI's Agents SDK shipped MCP support in March 2025, Google DeepMind built MCP into the Gemini API by mid-2025, and MCP support in VS Code Copilot reached general availability in July 2025.
Where the data gets softer: widely cited figures like "78% of enterprise AI teams run MCP in production" or exact Fortune 500 adoption percentages trace back to single-vendor marketing surveys rather than independently verifiable sources, and should be treated with real skepticism rather than repeated as fact.
The Part Adopters Underweight: Security
MCP's fastest-growing failure mode isn't a scaling problem — it's a trust problem, and it's specific to how the protocol works. A tool's description — the text that tells the model what the tool does — is reviewed once, when a client first connects to a server. A tool's response, every time it's called afterward, goes straight into the model's context with no equivalent review. That gap is what security researchers call tool poisoning: a malicious or compromised MCP server embeds hidden instructions inside what looks like ordinary help text or tool output, and the model follows them because it can't distinguish an instruction from data at that point in the pipeline.
This isn't theoretical. A May 2026 disclosure from OX Security exposed up to 200,000 vulnerable MCP instances across IDEs, internal tools, and cloud services, rooted in how official MCP SDKs handle local STDIO transport — and Anthropic confirmed the behavior was by design and declined to change it, treating it as a deployment-configuration issue rather than a protocol bug. Separately, the Cloud Security Alliance documented in July 2026 that widely used developer tools — Cursor, Claude Code, Gemini CLI, GitHub Copilot, and Amazon Q — auto-execute project-defined MCP servers with developer-level OS privileges and no process isolation, and Microsoft's own security team independently reached the same conclusion, publishing guidance that treats a tool description change with the same scrutiny as a dependency update.
The scanning data reinforces the pattern rather than being an outlier: an Equixly offensive-security assessment found 43% of tested MCP servers vulnerable to command injection, and a BlueRock Security scan of over 7,000 servers found 36.7% vulnerable to server-side request forgery. OWASP now ranks tool poisoning third on its dedicated MCP Top 10 list. None of this means MCP is uniquely broken — it means the protocol's core convenience (a model deciding at runtime which tool response to trust) is also its core attack surface, and the July 2026 spec's authorization hardening addresses the connection layer, not this response-trust gap.
MCP vs. Building Custom Integrations: When It's Actually Worth Adopting
You're maintaining multiple AI-model integrations to the same tools. MCP's entire value proposition is collapsing M×N custom integrations into M+N standardized ones — if you're only ever using one model with one tool, the standardization overhead may not pay off yet.
You want to swap AI vendors without rebuilding tool access. Because MCP is client-agnostic, tools built against it work with Claude, ChatGPT, or any compliant client without rewrites.
You're running MCP servers you don't fully control. Treat any third-party or community MCP server the way you'd treat an unreviewed dependency — vet the source, scope its permissions tightly, and don't grant it more filesystem or network access than the specific task requires.
FAQ
Q: Is MCP the same as an API?
A: Not exactly. A traditional API is a fixed set of endpoints a developer codes against in advance. MCP is a protocol for AI models to discover and decide which tools to call at runtime, based on a natural-language request — it standardizes how that discovery and invocation happens, rather than defining the tools themselves.
Q: Do I need to be an Anthropic customer to use MCP?
A: No. MCP is an open standard now governed by the Agentic AI Foundation under the Linux Foundation, not an Anthropic product. OpenAI, Google, Microsoft, and AWS all support it in their own platforms independently of any Anthropic account or API key.
Q: What's the difference between an MCP client and an MCP server?
A: A client is the AI application initiating requests — Claude Desktop, an IDE's AI assistant, a custom agent. A server exposes tools, data, or prompts that a client can call. The same organization can build both, but they're architecturally separate roles.
Q: Does the July 2026 update break existing MCP integrations?
A: Deprecated features like Roots, Sampling, and Logging will keep working for at least twelve months under the new formal deprecation policy, so nothing breaks immediately — but teams that built directly on session identifiers face real migration work, since sessions no longer exist at the protocol level.
Q: How worried should I be about MCP security before adopting it?
A: Worried enough to scope permissions deliberately, not worried enough to avoid it — the vulnerabilities documented above are mostly about how servers are deployed and trusted, not flaws that make the protocol unusable. Least-privilege tool access, sourcing servers from vetted publishers, and treating tool descriptions as untrusted input are the baseline controls security researchers now recommend.
FAQ
LLM observability typically stops at prompt logs, response logs, token usage, cost, and latency for a single call. Agent observability goes further, capturing the full multi-step decision path — planning, tool selection, intermediate reasoning, and replanning — because an agent's failure often traces back several steps before the final wrong output, not to the last call alone.
It's usable in production and already widely emitted by major frameworks, but it is not stable in the formal sense — every gen_ai.* attribute and span still carries OpenTelemetry's "Development" status as of mid-2026, meaning names and structures can still change between releases. Most practitioners recommend adopting it while pinning a specific version rather than always tracking the latest.
Because the failure isn't in the transport layer that alerts are built to catch. The agent returns a normal, timely HTTP 200 response — it's the content of that response that's wrong, whether from calling the wrong tool, hallucinating on incomplete context, or fabricating output around an upstream error it silently absorbed. Standard uptime and error-rate monitoring has no visibility into response correctness.
It depends on scale and regulatory exposure. Framework-native tracing (like OpenAI's Agents SDK built-in tracing) or an open-source OpenTelemetry-based setup can cover basic visibility for smaller deployments. Teams in regulated industries, or running agents autonomously for hours without human review, more often adopt a dedicated platform for the evaluation loop and governance/compliance reporting layered on top of raw tracing.
Treating the agent's own logs as ground truth. Multiple documented incidents show agents that fail to accurately log their own tool-call errors, then generate plausible output that papers over the failure. Capturing tool-call outcomes through telemetry that's independent of the agent's self-reporting is what closes that blind spot.
The GetCoreTech Team
We write about the SaaS, AI, and infrastructure decisions builders actually have to make.
Comments
Log in or sign up to join the discussion.
Loading comments…