I gave an agent one brief and pointed it at my own platform: connect as a real MCP client, exercise every tool, and find every place where what the tool says and what the tool does disagree. Review it as adversarially as possible. It came back with a list. An evaluating agent can confabulate as readily as the ones it judges, so I took none of it on the agent’s word and confirmed each item against real state first. What survived was a set of tools that lied: a parameter the docs described and the code ignored, the headline feature absent from the exact responses the docs said carried it, an update that reported failure while the write committed. For a product a human drives, those are annoyances. For one an agent drives, they are serious bugs, and finding them is what separates a working platform from a demo that recorded well. This is the case study, and the testing discipline that catches the class.

This is the final post in MCP by Design. The series built an MCP server in Go, composed it, steered the agent driving it, and closed the knowledge loop. This post turns the lens on the platform itself. It is grounded in the open-source txn2/mcp-data-platform, also available hosted as Plexara. The adversarial pass described here ran against the public ACME demo deployment at version 1.81.2.

§Why a Tool That Lies Is Worse Than One That Fails

Start with why this matters more for MCP than for an ordinary API. When a REST endpoint returns a wrong status, a human reads it and investigates. The human has other channels: the UI, the logs, a colleague. An agent has exactly one channel. The tool result is the only thing it learns from, and it plans its next call on the assumption that the result was true. The two failure modes post was about the agent ignoring what you told it. This is the inverse: the agent believing what you told it, when what you told it was wrong.

So a tool that fails cleanly is not the problem. An error the agent can read is recoverable, which is the whole reason the error contract from the previous post exists. The problem is a tool that succeeds in its own words while doing something other than what it said. The agent has no way to detect the gap. It writes the lie into its plan, and the user inherits the result. The tools/call post framed the result as the contract; a tool that misreports its own outcome is breaching that contract in the one way the consumer cannot catch.

§Three Ways a Tool Lied

The adversarial pass found the same disease in three forms.

The result contradicted the side effect. manage_artifact update with new content returned the error "no fields to update". The write had committed. The version counter had advanced to 2 and the stored byte size matched the new content exactly. The tool reported failure and succeeded anyway. This is the worst of the three, because the side effect and the reported outcome point in opposite directions. An agent that reads “no fields to update” does one of two wrong things: it retries, duplicating versions, or it tells the user nothing changed. Both are wrong, and the truth, that the update landed, is the one thing the agent cannot see.

The description promised what the code ignored. The tool schema for a describe call advertised include_sample=true with the text “see actual data values, which helps understand column meaning.” Setting it returned nothing extra and raised no error. An agent asking for samples to disambiguate a column got silence, and silence is ambiguous: no interesting values, or a feature that did nothing? Worse, the platform’s reason to exist, the semantic cross-enrichment from earlier in the series, was absent from the two responses the docs described most vividly. trino_describe_table and trino_query came back as bare columns. The enrichment data existed and was correct, but it sat in a sibling resource the agent only got if it knew to make a second read. The promise was loud in the docs and invisible at the moment it should have fired.

The platform advertised dead capability. An entire gateway toolkit was down. Every call to it failed with session not found, because the gateway held a stale upstream session and never re-initialized. Yet the connection listing showed it present with no hint of trouble, and the platform’s own capability map counted it among six live toolkits. Nearby, two managed resources listed cleanly and then failed on read, orphaned uploads from releases long past that had survived dozens of versions because nothing ever read them back. There was no liveness surface. The platform assumed its own stored state and its dependencies were fine and only found out otherwise when a caller tripped over the failure. For something whose pitch is “the authoritative context,” confidently serving whatever still answers while silently dropping whatever does not is a bad failure shape.

None of these are missing features. Every one is a tool whose words and behavior disagreed, which is exactly the failure an agent cannot route around.

§Why Mocks Could Not Catch Any of It

There is one root cause under all three, and it is worth internalizing because it explains the platform’s own prior headline bug too. That bug, a prompt write that failed every time in production, shipped because a SQL mock rubber-stamped a query that real Postgres rejects. The mock asserted the handler called the store with the right arguments. It could not assert the store accepted them, because the store was not real.

Every finding above has the same shape. The artifact-update bug is a disagreement between the write path and the response path; a unit test on the handler with a mocked store sees the handler return and never checks what landed. The orphaned resources are a read-after-write failure across releases; no unit test reads back what an earlier version wrote. The dead gateway is a live-dependency lifecycle failure; no mock reproduces an upstream session that has gone stale. They share a property: each appears only when you run the real assembled system, and then read back what you wrote or touched. A unit test that passes because it hands the code the correct input proves the code compiles against an interface. It does not prove the system told the truth about what it did. The bugs live in the gap between “the call returned success” and “the thing the call claimed actually happened,” and a mock cannot see into that gap because the mock is the thing standing in for reality.

§The Gates That Catch a Lie

The fix is not more unit tests. It is gates that exercise the real thing and read back. The platform runs three, and they are the part of the SDLC a vibe-coded server skips by definition.

The first is a real-Postgres round-trip. Tests tagged integration and named for the pattern run every write tool’s path against an actual pgvector database, not a mock:

## test-realdb: Real-Postgres round-trip gate for tool write paths
test-realdb:
	$(GOTEST) -count=1 -tags=integration -run 'RealDB' ./...

These live next to the code they guard, a *_realdb_integration_test.go beside the store in pkg/memory, pkg/resource, pkg/oauth, pkg/portal, and more. They catch the SQL that a planner only rejects against a live engine, the class that the prompt bug belonged to.

The second is a live smoke test that connects to a running server as a real MCP client, with an admin key, and drives every user-facing write tool end to end. Its assertions are the important part, because they are read-backs, not status checks:

t.Fatalf("manage_prompt read-back: content mismatch: got %q want %q", got["content"], content)
t.Fatalf("save_artifact read-back: size_bytes not persisted: %v", got["size_bytes"])
t.Fatalf("capture_insight read-back: recall_insight did not return id %s: %v", id, got)

Each one writes, then reads the thing back through a separate tool, then asserts the read matches what it wrote. save_artifact is not done when the call returns; it is done when a subsequent read returns the right size. capture_insight is not done until recall_insight can find the id. This is the shape that catches a tool whose result contradicts its side effect, because it ignores the result entirely and checks reality. The gate’s own comment states the rule: this is the layer that catches deployment drift the in-process tests cannot see.

The third gives the tests teeth. Coverage alone is a lie of its own, since a line can run without any assertion noticing if it were wrong, so the release gate runs mutation testing and holds it to a threshold:

mutate:
	gremlins unleash --workers 1 --timeout-coefficient 3 --threshold-efficacy 60 ./pkg/...

Mutation testing flips operators and conditions in the real code and fails if the suite still passes. A 60% efficacy floor means the tests have to actually notice the damage. Combined with an 80% coverage gate on changed lines, it makes the cheap green checkmark expensive enough to mean something.

The throughline across all three: the default shape of a test is “write, then read back, then assert the read matches the intent,” never “write, then assert the call returned success.” The bugs live in the difference between those two test shapes.

§Found One Week, Gone the Next

What makes this a case study is what happened after the list. The adversarial pass ran on a Monday. Over the following week, the findings closed in commits: the cross-enrichment folded into the structured output of trino_query and the describe call so it fires where the docs always said it did; the artifact update stopped reporting failure on a committed write; the gateway learned to reconnect a dropped upstream session and surface per-connection health; presigned URLs signed against a configurable public endpoint instead of an internal hostname; and the smoke and real-DB gates grew the read-back assertions that would have caught the artifact bug in the first place. The same discipline that found the lies by driving the real system is the discipline that now fails the build if they come back.

That is the loop I would want any MCP platform held to. You do not find this class of bug by reading the code, because the code looks correct; the handler returns, the mock agrees, the test is green. You find it by running the assembled thing as the agent will run it and reading back what it claims to have done. And once found, you do not just patch the instance; you add the read-back that makes the gap a failing test forever.

§The Lesson Generalizes

Strip away the platform and the lesson is for every MCP server anyone ships. The tool description is a contract and the tool result is ground truth, and the reader on the other end is a model with no second channel and no skepticism, that will plan three steps deep on whatever you returned. A vibe-coded server passes its demo because the demo is one happy path a human watched. The SDLC is everything that makes the result trustworthy when nobody is watching and the consumer is another machine: the Go gates from the first post that constrain the code as it is written, and the read-back gates here that constrain what it claims once it runs.

That is the series. Read the protocol off the wire, then build for a reader that cannot tell when you are wrong. That takes two separate things the protocol does not give you. Design, which is what the server does: compose for the join, steer the agent, capture what gets learned. And an SDLC, which proves it does it: the Go leash that constrains the code as it is written, and tests that read back instead of trusting the call. The protocol is the easy part. Design and the SDLC are the engineering, and skipping either ships a demo.

The platform behind this series is txn2/mcp-data-platform, available hosted as Plexara.