Most teams test the happy path, then ship. The bug report arrives three weeks later: someone POSTed {"amount": -1} to the payment endpoint and the API cheerfully returned 201 Created. The database accepted it, the invoice generator tried to subtract a negative number from zero, and the customer got a refund email for an order that never existed. The contract said minimum: 1. The OpenAPI document was right. The code was wrong. Nobody caught it because nobody tried.
The API Contract Mutation Tester exists for that exact gap. You paste an OpenAPI 3.x document, point it at a staging backend, and it derives every documented operation, then mutates each request field into semantically risky variants — omitted required fields, negative numbers, enum violations, whitespace-only strings, suspicious special characters, scalar-vs-array shape mismatches. Each mutated request is sent against the real backend and classified as defended (rejected with a 4xx), accepted (the API returned 2xx anyway), or documented (the observed status matched what OpenAPI promised). Anything that lands in the accepted bucket is a real bug. Not a theoretical one — an actual successful request that violated the contract you wrote three months ago and forgot to enforce.
Mutation Testing Is Not Fuzzing
Fuzzers throw random garbage at an endpoint until something crashes. Mutation testing starts from the contract and asks a different question: given the schema you documented, how many of its own rules is your code actually enforcing? A fuzzer can find a crash in your JSON parser. A mutation tester finds a 201 Created for a negative refund amount — the kind of bug that doesn’t crash, just quietly corrupts an accounting ledger until quarter-end reconciliation.
The five mutation families the tool generates cover the realistic attack surface that hand-written happy-path tests miss:
- Required-field omission. Drop a documented required field entirely. If the backend returns 200, your
required: trueannotation is decoration. - Special-character and suspicious-string payloads. Empty string, whitespace-only string, control characters, SQL fragments, XSS fragments, emoji, NULL bytes. The contract says
type: string— does the code distinguish" "from"real value"? - Negative or oversized numeric values.
-1,0,Number.MAX_SAFE_INTEGER, scientific notation overflows. A documentedminimum: 1field that accepts0is a 0-dollar transaction waiting to happen. - Invalid enum members. Strings outside the documented
enum, case variations, leading whitespace, locale variants. - Wrong scalar-vs-array shapes. Documented as array, sent as object. Documented as string, sent as number. Documented as integer, sent as boolean.
Each mutation is generated deterministically from the schema, so the same OpenAPI document always produces the same mutation set. That makes regressions easy to spot — if a previously-defended mutation starts getting accepted after a refactor, the test suite catches it on the next CI run.
Reading the Report: Defended vs Accepted vs Documented
The output is a flat list with three labels per mutation, plus the HTTP status, response time, and a truncated response body. Reading it is the whole game:
- defended — the backend returned a 4xx or 5xx. Your validation is doing what the contract promised. Ship it.
- accepted — the backend returned a 2xx (or any code that isn’t a documented error response). The contract said one thing, the code did another. This is the bug list.
- documented — the observed status code matched an entry in the OpenAPI
responsessection, but the response body doesn’t look right (wrong shape, wrong field types, missing fields). The contract was honored at the status-code level but the payload is lying.
The ratio you want, asymptotically, is 100% defended, 0% accepted, 0% documented-but-broken. In practice, the first run on a real backend lands somewhere around 60-80% defended, with the rest concentrated in fields that have lax validation (free-text notes, optional metadata, optional numeric fields where zero is technically valid but semantically wrong). Those become your hardening backlog.
If you want to compare this against another defensive-validation pattern that handles a different surface — pure schema validation without the mutation engine — the API Response Contract Validator follows the same defended/accepted shape but works on responses rather than requests. Pair them: mutation tester for inputs, response validator for outputs. Together they bracket the entire contract surface.
Offline Mutation Plans vs Live Execution
The tool has two modes that change everything about how you use it. With the Base URL field empty, the tool runs in offline mutation-plan mode — it generates the full mutation set, labels each one with the field and value, but never sends a request. The output is a JSON document you can hand to QA, paste into a ticket, or feed into a property-based testing harness. This is the safe mode for production backends, shared staging environments, or anywhere you don’t want a flood of {"amount": NaN} requests hitting logs and downstream webhooks.
With a Base URL and the Execute Mutations checkbox enabled, the tool fires every mutation at the live backend and records the response. The Mutations Per Field cap (default 5, configurable) prevents combinatorial explosion on a schema with 30 fields — 30 fields × 5 mutations × 50 operations × per-request timeout = manageable. Without the cap, a 30-field schema with 50 operations would generate 7,500 requests per run, which is both slow and noisy in any real environment.
The Request Timeout (ms) field is the other safety valve. Default is 5,000 ms — long enough for a healthy backend, short enough that a hung request doesn’t poison the rest of the run. If your backend legitimately takes 30 seconds to respond to a malformed request, you’ve got a different bug to file.
If you want to look at how this fits next to boundary-value testing — which is the sibling pattern that asks “what does the code do at the edge of valid?” rather than “what does the code do on invalid input?” — the API Contract Stress Tester handles the boundary side. Mutation testing for what the contract forbids; boundary testing for what the contract barely tolerates.
What Mutation Testing Catches That Other Tests Miss
Unit tests on the validator layer usually mock the database and return true. Integration tests on the happy path cover the documented success cases. End-to-end tests click through the UI and rarely send malformed bodies. None of them catch the bug pattern that mutation testing specifically targets: the field that the schema marks as required but the controller treats as optional.
Concrete cases this pattern catches:
- The payment amount field documented as
required: true, minimum: 1that the controller reads withreq.body.amount ?? 0. The??convertsnullandundefinedto0, and the database happily accepts a zero-charge transaction. Mutation test: omit the field, expect 400, get 201. - The email field documented as
format: emailthat the controller validates only for presence. Mutation test: send"not-an-email", expect 400, get 201, and a malformed welcome email gets queued. - The status enum documented as
["pending", "active", "cancelled"]that the controller writes to the database without checking. Mutation test: send"approved", expect 400, get 201, and your reporting dashboard now has rows it can’t categorize.
These are the bugs that don’t show up in code review (the contract looks fine), don’t show up in unit tests (the validator is mocked), and don’t show up in happy-path integration tests (valid inputs only ship). They show up the first time someone — internal tooling, a frustrated customer, an attacker with a curl command — sends a value that wasn’t in the test fixtures. Mutation testing forces that “someone” to be your CI pipeline, on a schedule, before the production incident.
Putting It Together
The single best signal a mutation-testing run gives you is the accepted count after a defensive refactor. Before you tighten validation on the payment amount field, you record the baseline: 14 accepted mutations across the payment domain. You fix the controllers, redeploy, rerun the tool against the same OpenAPI document with the same Base URL. The new run shows 0 accepted mutations in the payment domain. That’s the metric — not “tests pass” but “previously-accepted mutations are now defended.”
If you’re building out an API hardening pipeline, the natural neighbors are the OpenAPI Diff Breach Detector (catches breaking changes between spec versions), the API Mock Server (gives the mutation tester a deterministic backend to attack in CI), and the JSON Schema Validator (handles the schema-only path when you don’t have OpenAPI 3.x). Each covers a different layer of the contract surface — mutation testing is the runtime, behavioral layer.
The contract you write is a promise. Mutation testing is the only automated way to verify that promise holds against the actual code, not against the unit-test mocks of the actual code. Paste your OpenAPI document, set the Base URL to staging, and read the accepted list. Whatever shows up there was a bug you didn’t know about. Now you do.
Try the API Contract Mutation Tester with one of your own endpoints. Browse the OpenAPI/Swagger samples for a starting document, or load up the Mock Service Worker samples if you want to test against a deterministic mock first. Explore more development tools at elysiatools.com.
