Your OpenAPI Spec Is Not an Agent Interface: Compiling Governed MCP Tool Surfaces

Code: github.com/bassrehab/api-mcp-compiler · Docs: api-mcp.subhadipmitra.com · Install: pip install api-mcp-compiler

Every gateway vendor now ships a button that turns an OpenAPI spec into an MCP server. Press it and your 87-operation service becomes 87 tools, each one a faithful transliteration of an endpoint that was designed for a frontend team in 2019. The conversion is real, the demo works, and the hard problem has not been touched.

The hard problem is deciding which tools should exist at all, what they should be called, what they should accept and return, and which of them must never run without a human saying so. That is interface design, and the interface’s consumer is a language model choosing its next action from names and descriptions. I spent the past few months building a compiler around that problem: api-mcp-compiler, which takes OpenAPI 3.x, Swagger 2.0, or WSDL 1.1 and produces an MCP tool surface with provenance on every field, a governance manifest as a separate reviewable artifact, and destructive operations compiled in disabled until a person approves them.

I will also show you the part that did not go the way I expected. I pre-registered four evaluations to prove that a semantically planned surface beats naive one-tool-per-operation, and all four came back underpowered. I think the numbers are more useful than the claim would have been, and I would rather report them precisely than bury them. But the reason the project exists is the governance half, and that half was never on trial.

The transliteration trap

Take a real specification and convert it operation by operation. Here is what the agent inherits:

• Colliding names. One API I ingested had ten operations whose summaries all read “Get Details.” An agent selects tools by name and description. Ten tools that describe themselves identically is not a tool surface, it is a lottery.
• Transport plumbing promoted to interface. Pagination cursors, response envelopes, and format toggles become tool arguments the model is invited to guess at, even where the server has perfectly good defaults.
• Dead surface. Deprecated operations arrive with the same standing as live ones, spending the model’s attention on endpoints the provider intends to withdraw.
• Destructive parity. DELETE /accounts/{id} shows up enabled, right next to the harmless reads, because the spec never said which operations should make a reviewer nervous. Specs describe capability, not intent.

None of this is the spec’s fault. An OpenAPI document is a contract between machines that already know what they want. An agent tool surface is an interface for a consumer that decides what it wants by reading the interface. Those are different artifacts, and I have come to think the second should be compiled from the first the way object code is compiled from source: through explicit, inspectable stages, not string substitution. I have written before about where organizations sit on context maturity and how the protocol stack is settling; this project is what I think the bottom layer of that stack should look like when it grows up.

A compiler, not a converter

The pipeline has five stages, each emitting an artifact with its own versioned JSON Schema contract, so a change in one stage cannot be silently misread by the next.

flowchart LR
    A[OpenAPI / Swagger / WSDL] --> B[Ingestion<br/>Semantic IR]
    B --> C[Planning<br/>proposals + overlay]
    C --> D[Policy<br/>governance manifest]
    D --> E[Generation<br/>MCP server]
    E --> F[Evaluation<br/>state oracles]

Ingestion is deliberately paranoid. No network calls happen during it, remote $ref resolution is denied by default, and file loads are restricted to explicitly allowed directories. Every field in the resulting IR carries provenance: where in the source it came from, how it was derived, and with what confidence. Nothing is silently dropped. A construct the compiler cannot resolve becomes an explicit Ambiguity record, a first-class output rather than a warning scrolling past in a log.

The whole flow is a CLI:

python -m api_mcp_compiler.cli inspect  examples/openapi/inventory_service.yaml  # IR, side effects, provenance
python -m api_mcp_compiler.cli review   examples/openapi/inventory_service.yaml  # planner proposals, as markdown
python -m api_mcp_compiler.cli policy   examples/openapi/inventory_service.yaml  # governance manifest
python -m api_mcp_compiler.cli generate examples/openapi/inventory_service.yaml  # surface, destructive tools disabled
python -m api_mcp_compiler.cli report   examples/openapi/inventory_service.yaml  # self-contained HTML approval doc
python -m api_mcp_compiler.cli approve  examples/openapi/inventory_service.yaml \
    --overlay build/inventory.overlay.json --risk destructive                    # decisions land in an overlay
python -m api_mcp_compiler.cli serve    examples/openapi/inventory_service.yaml \
    --overlay build/inventory.overlay.json --out build/inventory_server.py      # runnable MCP server

The planner proposes, you dispose

The planning stage looks at the IR and proposes changes to the baseline one-tool-per-operation mapping. Seven kinds of proposal exist:

1. Rename: name tools around tasks instead of handler methods, so listWarehouseItems becomes list_items_held_warehouse, derived from the operation’s own summary.
2. Group: group by the coarsest structure the specification actually states (the first path segment), not a taxonomy the planner invents.
3. Project: drop optional transport arguments that have server defaults, pagination cursors being the classic case.
4. Describe: rewrite descriptions for a model reader, stripping web formatting and foregrounding actual side effects.
5. Reclassify: turn addressable reads into MCP resources so lookups stop consuming tool slots.
6. Omit: drop deprecated operations, so agent attention is not spent on surface the provider intends to withdraw.
7. Compose: propose workflow tools for lookup-then-act sequences.

Every proposal is recorded with a kind, a target, a rationale written for a person, and a confidence derived from signals in the specification rather than the planner’s self-regard.

The important part is what the planner does not get to do: decide. A plan is a set of proposals. Decisions live in an overlay file that the approve command generates, bound by digest to the exact revision of the specification. When the spec changes, the approvals do not silently carry over. Nobody hand-edits JSON, and the tool surface never shrinks without a recorded reason.

Tools that earn the right to execute

Generation runs everything through an emission gate. The docs state the rule plainly: a tool is emitted executable only if it earned it. Reads pass automatically once validated, because a reviewer clicking through twenty-three read tools one at a time is doing data entry, not governance. Writes and destructive operations need a recorded approval, granted by risk class, by group, or by individual name.

Everything that fails the gate is still emitted, but disabled, carrying the reason it was refused. This sounds like a small choice and is not: a surface that silently omitted an operation would be indistinguishable from one where the operation never existed, and nobody reviews an absence. The refusal is compiled into the artifact, not enforced by a runtime configuration flag that someone can quietly flip in production.

Two more properties do a lot of the safety work:

• Inference is safety-monotone. Analysis may reclassify an operation as more restricted than the spec suggests, never less. A write that cannot demonstrate its authorization story fails closed.
• Confirmation does not transfer, and does not keep. An approval token for a destructive call is bound to a digest of the specific arguments and is single use with an expiry: a lapsed token is refused and a fresh confirmation is required (TTL enforcement shipped in 0.2.0). A confirmation for one operation cannot be replayed to authorize a different one.

Alongside the code, the policy stage emits a per-tool governance manifest: narrowest sufficient scopes rather than a union of alternatives, approval class, rate budgets (calls per minute, concurrency, daily budget), and rollback guidance for destructive operations. Policy is a separate artifact precisely so that the people who review governance never have to read generated code to find it.

The SOAP path is not a footnote

If you have a legacy estate, this is the part you actually care about. WSDL 1.1 is a first-class input, not a translation layer over the REST pipeline: namespaces, XSD types, SOAP headers, faults, and bindings all survive ingestion. XSD types translate to JSON Schema with their facets intact, including pattern, enumeration, and length, and minOccurs and maxOccurs cardinality. Document/literal and rpc/literal bindings are both ingested. Section 5 encoding is refused rather than approximated, because it serializes values as a reference graph the compiler does not write.

SOAP also changes what inference is allowed to do. WSDL carries no signal equivalent to an HTTP method, so a SOAP operation is never classified by inference: every one arrives unclassified, and the emission gate blocks it until a reviewer records the side effect. This is deliberate. An operation named DeleteCustomer might delete, and might return a receipt.

The path is verified against 40 WSDL documents from the public SAWSDL test collection, fetched and never redistributed, and against two live public SOAP services that answered real calls through generated servers.

The part I could not prove

The obvious question: does any of the planning actually help the agent? I assumed the answer was yes and set out to demonstrate it. Because “assumed the answer was yes” is exactly the condition under which people fool themselves, the harness enforces pre-registration. Before a model-backed comparison runs, the hypothesis, corpus, arms, model, success criteria, budget, primary statistical test, threshold, and falsification conditions are written down and digested. Runs record the digest of their registration, the harness refuses to combine runs whose digests differ, and registrations are append-only. A result cannot be attached to a hypothesis written after the fact, including by me.

Scoring is deterministic. A task succeeds if the service ends in the state the goal asked for, judged by an oracle against the final store, not by a model grading a transcript. Oracles name source operations rather than tool names, so one corpus scores both arms without favoring either. The primary test was McNemar’s exact test, two-sided, alpha 0.05, which on a 24-task corpus needs at least six discordant pairs, all favoring the same arm.

Four registrations ran to completion:

Let me be precise about what the table says. It is not evidence that planning works: every point difference is one discordant pair against a threshold of six. It is also not evidence that planning is useless: an underpowered test that fails to reject tells you about the test, not the world. At the discordance rate the runs actually observed, a 55-task corpus expects about 2.3 discordant pairs, and roughly 150 tasks would be needed to expect six. These corpora were never going to answer the question, and the specs where I expect design to matter most, the 400-operation enterprise WSDLs with a decade of accretion, are exactly the ones that are never public.

If I had run one comparison, gotten 14 against 13, and skipped the registration step, this section would be a triumphant chart. Pre-registration is the reason it is a table with “inconclusive” written four times, and I have come around to thinking that is the more useful artifact.

The most interesting measurement in the project is hiding in the TMDB row. Both arms scored 28 of 34 with zero discordant pairs, but they did not get there the same way. The agent reached for a composite tool on 29 of 34 tasks, 18 of the 29 composites the plan offered were used at least once, and total calls fell from 312 to 289, a reduction of 7.4 percent. Context bytes rose about 20 percent, because a composite returns the last step’s payload while the baseline agent often stopped at the smaller one it needed. That is a finding: composition changed what the agent did without changing what it achieved, and the trade it made was calls for context. (Secondary, and it stays secondary: the semantic arm used fewer calls on 17 tasks and more on 9, a two-sided sign test at p = 0.169. Suggestive, not evidence.)

What the harness caught instead

The evaluations refused to resolve the headline hypothesis, and paid rent anyway. Building oracles that check final state instead of model behavior turned the eval harness into an instrument, and the instrument found defects the demos never would have:

• Redaction deleted a live service’s answer because the field name TitleCaseWordsWithTokenResult contains the word “token”. The tool returned [redacted] with HTTP 200 and no hint that anything had been withheld. Sensitivity is now decided on the noun a field name ends on, not on any overlap with a secret vocabulary.
• A generated SOAP body used the message part’s own name instead of the element the part references. Every local fixture passed. Every real service returned a fault.
• 22 of 40 generated tools carried schemas that no JSON Schema validator would accept, because the source specification writes "maximum": "50" as a string: valid JSON, an invalid schema, and invisible until something checked it mechanically.

This is the general lesson I keep relearning, most recently while building eval loops for verifiers: deterministic evaluation infrastructure is worth building even when the experiment is inconclusive, because the harness audits everything it touches.

And it is why the inconclusive table does not undermine the project. Provenance, digest-bound approvals, fail-closed writes, and non-transferable confirmation tokens were never claims about task success rates. They are claims about what an agent may do without a human, and they hold whether semantic planning turns out to be a win, a wash, or a rounding error.

What this is not

The compiler documents its own limits, and I will repeat them here. A generated artifact cannot provide server-side authorization, protection against confused-deputy designs, or end-user identity propagation. Those are properties of the deployed service, and a tool surface that claims to solve them from the client side is lying to you. The evaluation corpus is two public APIs and small task sets, which is a floor, not a ceiling.

Try it

The repo is Apache 2.0, typed strictly, and ships with an executable notebook, golden artifacts for reproducibility, and the full docs, including the evaluation methodology with every registration public. It is on PyPI, 0.3.0 at the time of this update:

pip install api-mcp-compiler

Two things would genuinely move this forward. First, larger task corpora: the arithmetic above says roughly 150 tasks, and that is breadth I cannot generate alone. Second, ugly specifications. The hypothesis that tool surfaces should be designed rather than transliterated is at its most plausible on large, old, colliding surfaces, and those never appear in public benchmarks. If you have a WSDL from 2009 that you are willing to share even in redacted form, I would like to compile it.