I fetched 957 public OpenAPI specs. Most can't tell you what changed.

An API is a boundary you put up so the other side doesn’t have to understand your system. That is the whole deal. You don’t need to know how our pricing service works, you need to know that this endpoint takes these fields and gives you back that shape, and that we’ll tell you before any of it moves.

The contract is where that promise lives. So I went and read 957 of them, from 828 companies, to see how much of the promise is actually in there.

If you haven’t had to deal with one: an OpenAPI spec is a file, YAML or JSON, describing a REST API endpoint by endpoint. What paths exist, what each accepts, what it returns. It’s machine-readable so tools can generate client libraries and docs from it without a human in the loop.

I measured three things. None of them is whether the file parses.

• Does the description still track the code? Often not, and nothing says so.
• Does the change history tell a consumer what moved? Usually not, and this was the surprise.
• Does the contract carry the abstraction at all? Frequently there is nothing there to carry.

In that order, because that is the order I stopped being able to explain the results away.

1. The description stops tracking the code, and nothing says so

Of the 957, 699 still return a document. 278 of those no longer serve a spec at the URL the public directory lists, and for 260 of them the dead address is on the company’s own domain.

flowchart LR
    A["957 specs<br/>828 companies"] --> B["733<br/>published by<br/>the company"]
    B --> C["699<br/>returned a<br/>document"]
    C --> D["421<br/>live at their<br/>own URL"]
    C --> E["278<br/>only via an<br/>archived copy"]
    style E stroke-width:2px

What happened at the recorded URL	Specs
404 or 410, the path is gone	107
Host doesn’t resolve, or refuses connections	96
Now wants credentials (401/403)	30
Returns HTML, a docs page where the spec used to be	29
Malformed, or JSON that isn’t a spec	9
5xx, possibly just a bad afternoon	7

203 are properly gone. The 30 behind auth probably still exist and I just can’t see them, so calling those dead is unfair. The 7 server errors might have come back by the time you read this.

The 29 returning HTML are the ones I nearly missed, because my parser binned them as failures and moved on. I only looked again because a batch of “failures” had all returned HTTP 200.

Those URLs work fine. Somebody rebuilt a developer portal, the route that used to serve openapi.json now renders a documentation page, and every code generator pointed at it started receiving markup instead of a schema. No 404. Nothing in anyone’s dashboard. The URL is in perfect health and the contract is gone, which is a worse failure than a 404 because a 404 at least has the decency to be loud.

Nobody made a mistake there. There is just no channel from the team that moved the route to the people depending on it.

2. The change history isn’t written for anyone downstream

This is the one that changed how I think about the problem.

For 249 specs I could read the commit history of the spec file itself, so I looked at what the most recent change to each one said. Not whether it was a good commit message. Whether a consumer reading it could tell what to go and check.

69% couldn’t. Some of what they say:

“spec release” ”[*] update doc” “V3 (#14)” “Update spec January 25th.” “Update API Reference.” “Update API repository (356 APIs)”

My favourite is “Update spec January 25th.”, which raises the question of which January, and answers it with a full stop.

And from the 31% that work:

“Delete identifier property for products-uuid endpoint GET and PATCH” “add consent-documents [GET] end point” “feat: add GET /docs endpoint serving Swagger UI”

The second group names a path, a field, a method. Read it and you know whether it touches you. The first group confirms that something happened.

I assumed this would split along the human/robot line. It doesn’t:

Who writes the spec	Specs	Change says what changed
A pipeline	75	33%
A named person, by hand	47	32%

Identical. That surprised me and I think it’s the most useful thing in this survey. It isn’t an automation problem, which would be fixable with better tooling on the provider’s side. Nobody is writing the change for the consumer, whether there’s a person in the loop or not. The history exists because git exists, not because anyone intended it as a record.

So the sequence is: something changes in the code, the spec gets regenerated, the file lands, and the one artifact that could have said “this field is gone now” says “spec release”.

flowchart LR
    A["code changes"] --> B["spec regenerated"]
    B --> C["file committed<br/>'spec release'"]
    C --> D["published at<br/>a URL"]
    D -.->|"nothing travels<br/>this way"| E["the consumer,<br/>human or agent"]
    E -->|"guesses from<br/>the diff"| D
    style E stroke-width:2px

3. The contract doesn’t carry the abstraction

I then read 103 specs properly, resolving every internal reference and walking every operation. 273 findings. The pattern is not what I expected going in.

What’s missing	Specs	Operations	Median share of the spec
No example payload on a success response	64	4,254	99.5%
No example of a request body	44	1,800	90%
Only success responses documented	28	1,601	100%
No security scheme defined at all	27	2,206	100%
Neither a summary nor a description	11	966	56%

Read the median column, not the counts. It’s the part that changed my mind:

• 100% is the median share of operations documenting no failure case, in the specs that have that problem.
• 99.5% is the median share showing no example of what comes back.

These are not specs with a few gaps someone hasn’t got round to. They are specs where that whole category of information was never filled in, once, anywhere.

There’s syntax breakage too. Duplicated operationId values, reference pointers resolving to nothing, a document declaring one OpenAPI version while using the other version’s layout. I found 14 cases the specification forbids outright.

But I want to be honest about how little that matters, because it’s the part I originally led with. Of the 103 specs, 80 have only the semantic gaps above and exactly 1 has only syntax problems. A validator catches the syntax on the first run. Nothing catches an endpoint that documents no error case, because nothing is wrong with the file. It parses. It’s just that a consumer reading it cannot tell what happens when the call fails.

That’s the abstraction not making it into the contract. The boundary is there, and the thing that was supposed to make the boundary usable is absent.

What breaks when the consumer is a machine

This part is reasoning, not measurement.

A human consumer patches all three gaps by hand without noticing they’re doing it. No description, so infer from the path. No error responses, so send a bad request and see. Example dated 2019, so go and check. Then open Slack and ask someone who knows. Twenty years of API practice assumes the consumer will do every one of those repairs, for free, and mostly we have.

An agent doesn’t ask. It reads the contract literally, and where the contract is silent it produces the most plausible thing instead. Plausible is the problem. Wrong would be fine, wrong gets caught.

A change that a spec diff describes perfectly and still gets wrong

Made up in its details, but this is the shape of it.

You depend on an orders API. An agent maintains your integration; the API is a dependency and the spec is all the context it has about it.

The provider splits fulfilment out of the orders service. Good reason: fulfilment was the write-heavy part and it needed to scale separately. They publish the change. In the spec diff it is two lines. shipping_status is gone from GET /orders/{id}, and there is a new GET /fulfilments/{order_id} that has it.

Your agent handles this correctly. It notices the field is gone, finds where it went, calls both endpoints, merges the results. Types check. Tests pass. The integration works.

sequenceDiagram
    participant P as Orders API (provider)
    participant S as Published spec
    participant A as Agent (consumer side)
    P->>P: split fulfilment into its own service
    Note over P: internally: writes now async,<br/>status lags the order by seconds
    P->>S: shipping_status moved to /fulfilments
    S->>A: two-line diff, both shapes valid
    A->>A: call both, merge, tests pass
    Note over A: assumes the merged value is<br/>as fresh as it was yesterday
    A-->>P: intermittently shows stale status to users

Here is what it could not know. The split happened because fulfilment now has its own write path, and shipping_status is eventually consistent against the order. It can lag by seconds. That is not a side effect of the change, it is the reason for the change. The provider’s engineers discussed it for a week.

So your checkout page, which shows a customer their status immediately after they order, now intermittently shows them nothing or something stale. No test catches it. It shows up as a support ticket three weeks later, and whoever picks it up starts by reading a spec that describes the current state perfectly and explains nothing.

The agent did the semantic work faultlessly. It had no way to do the architectural work, because the architectural work requires knowing why, and why lives in the provider’s system.

You could argue the consumer should just read the provider’s context. But that is the wrong answer twice over. It is a volume of information nobody can filter, and most of it is deliberately not shared, which is why there’s an API between the two systems in the first place. The boundary exists so the consumer doesn’t need it.

Why the spec can’t fix this on its own

The provider could write the consistency note into the endpoint description, and the better providers do. But notice what that description can hold: what this endpoint is, now. There is no field in OpenAPI for what this used to guarantee and no longer does.

A spec is a snapshot. Impact is a delta. You cannot store a delta in a snapshot.

Which is why the 69% matters more than any of the findings. The changelog is the only artifact that can carry a delta, and the one place it could have said “status is now eventually consistent against the order, and if you were reading it inline you need to change that” said “spec release”.

The part that should worry you is the part I couldn’t measure

All of this is public specs, which is the only reason the survey exists: they’re published at addresses anyone can fetch.

Most specs aren’t public. They’re the internal ones describing the twelve services inside your own company, and nobody is running this check on those. No directory indexes them. No outsider fetches the URL to see whether it still returns a document. The specs I could measure are the ones that at least had the accountability of being visible.

I’d expect the internal numbers to be worse and I can’t prove it. But if 69% of public spec changes can’t say what moved, I wouldn’t bet on the ones nobody outside the team ever reads.

What a provider owes a consumer

None of this needs a product, and only the first item is about the file being valid.

Validate it. One minute. npx @redocly/cli lint your-spec.yaml resolves the references and names the dead ones. Catches everything in my syntax category, which is the least of your problems but also the cheapest to remove.

Check the URL still serves a spec. Fifteen seconds. curl it and look at the first line. You are checking that it is still JSON or YAML and not an HTML page. This is the failure I least expected to find twenty-nine times.

Write the change for the consumer, not for git. Two lines, every time you change the spec. Name what moved: which field, which operation. Then add the sentence the spec itself cannot hold, the one about what it means for somebody depending on the old behaviour. “We split fulfilment out, so shipping_status now lags the order by seconds. If you read it inline you need to change that.” That is the difference between an agent doing semantic work and an agent doing the right thing. “spec release” is the default because nobody downstream is in the room when it gets written.

Work out who actually gets told. One awkward meeting. Not who could find out from a git log. Who receives a notification. For most teams the honest answer is nobody, and that is what produced the 278.

If you do all four already, you’re ahead of nearly every company in this survey, and I’d like to know how you made it stick.

What I think is actually missing

I build API tooling for a living, so treat everything below as an interested party talking.

I started out thinking the problem was spec quality. The commit-message number moved me, and writing out that fulfilment example moved me further. Quality you can lint. A rationale nobody wrote down you cannot, because there’s no file to check and no format that would have held it.

What’s missing isn’t a better document. It’s a channel.

A provider ships a changelog to consumers it can actually identify, in a form something on the other end can read without a person in the middle. Over MCP, or a CLI that runs in the consumer’s build, so the agent maintaining an integration is told “this field moved and here’s what it means for you” at the moment it matters, instead of inferring it from a diff three weeks later.

That needs plumbing at both ends, and neither end is interesting work. On the provider side, something that makes the changelog get written properly rather than “spec release”, ideally at the moment the change is made and by whoever made it. On the consumer side, something that takes it in and puts it where your team actually records decisions, next to your ADRs or in whatever review catches architectural change.

None of that is hard to build. If you have a provider team and a consumer team who talk to each other, you could wire a usable version of it in a sprint, and I would much rather you did that than nothing. It doesn’t need us.

It’s easier to argue with something concrete than with a description, so here is one shape it can take. We mirror 42 public API specifications, the official ones their owners publish, and generate a changelog per version rather than waiting for anyone to write one: app.spec0.io/registry. A dozen of the specs in this survey are in there. Each listing carries a quality score with the structural findings separated from the stylistic ones, and a low score doesn’t keep an API out, because a record that hides the bad ones is a worse record.

I’m not sure that’s the right shape. The part I’m sure about is that the gap is real and nothing fills it by default.

Method

Two sources, both read through published interfaces rather than scraped: the APIs.guru directory, which exists to be read by machines, and GitHub’s REST API for code search and commit history. GitHub is there because APIs.guru stopped updating in April 2023 and can’t see anything published since. Then I fetched every spec from its own URL, one request per second per host, robots.txt respected, nothing behind a login. September 2026.

Three things a sceptic should know:

• “The URL the public directory lists” is carrying weight in that 278. For 260 the address is on the company’s own domain, so I think it’s fair, but some entries were recorded three years ago and I can’t separate “they moved it” from “the directory got it wrong”.
• The commit-message judgement is mine. I counted a change as informative if it named a path, a field, a method, or a change verb applied to something named. You’d draw that line slightly differently.
• I’m not naming companies. Every finding is a public file at a public URL, but I sell tooling in this space and a vendor listing other people’s defects by name is a poor way to argue. If you want to check my work rather than take it on trust, 42 of these specs are mirrored at app.spec0.io/registry with their official source, licence and a lint score on every listing. Run your own numbers and tell me where they differ from mine.

Every number here was re-derived by hand first, because an earlier version of my own tooling reported 41 broken references in a document that had none. YAML parses the key 422: as an integer. My resolver was looking for the string. It called every valid shared-response reference in every Swagger 2.0 file broken, and it was completely confident about it.

Which is the short version of the whole post. The tool was sure. Nobody had told it otherwise.

---

Written by Nakul Shukla. I’m building Spec0 as an attempt to put the above into practice: a home for the contract, changelogs that reach the consumers who depend on it, and the plumbing at both ends. The desktop client, Studio, is MIT and downloadable here, with no account and no telemetry. It doesn’t read Swagger 2.0 yet, which this survey says is 38% of the live public surface, so that’s next. The CLI and mock server are open source too.