Check catalog
Every response apiprobe receives goes through the checks below (src/apiprobe/checks.py). A failure is
reported once per signature (check, operation, status, JSON pointer and a value-free version of the
message), then shrunk if the check is shrinkable.
| Check | Fails when | Shrunk | Example finding |
|---|---|---|---|
not_a_server_error |
status is 5xx | yes | server error 500 for {"name": "é", "age": 0} |
status_code_conformance |
status (below 500) matches no documented code, NXX range or default |
yes | status 409 is not documented (documented: 201, 400) |
content_type_conformance |
a documented response has content, the body is non-empty and Content-Type is missing or matches none of the documented media types (wildcards like application/* allowed) |
no | Content-Type 'text/html' is not documented (application/json) |
response_schema_conformance |
the JSON body is not valid JSON or violates the response schema; up to 5 errors with JSON pointers | yes | 'created_at' is a required property (at /created_at) |
response_headers_conformance |
a required: true response header is missing, or a present documented header fails its schema (values are coerced to integer/number/boolean first) |
no | required response header 'Location' is missing |
negative_data_rejection |
a request that violates exactly one constraint gets a status below 400 | yes | invalid request was accepted with status 201 (body /age: violates minimum (0)) |
response_time |
the response took longer than --time-budget ms (default 2000) |
no | response took 2350 ms (budget 2000 ms) |
Stateful checks (run only on create/read/update/delete sequences):
| Check | Fails when |
|---|---|
stateful:created_resource_not_retrievable |
the read operation linked from a successful create does not return 2xx |
stateful:updated_resource_not_found |
the update of a freshly created resource returns 404 (or a non-4xx error) |
stateful:deleted_resource_still_available |
after a successful DELETE, the read returns anything other than 404 or 410 |
Response lookup
For a status code, the documented response is chosen as: exact code, then the NXX range (4XX),
then default. Checks that need a response definition do nothing when there is none; the missing
definition is reported once by status_code_conformance.
Schema validation
validator.validate returns all violations so callers can count them (negative generation relies on
this). Supported keywords: type (string or list, 3.0 nullable), enum, const, format (date,
date-time, email, uuid, uri, ipv4), minLength, maxLength, pattern, minimum, maximum,
exclusiveMinimum/exclusiveMaximum (3.0 boolean and 3.1 numeric forms), multipleOf, items,
minItems, maxItems, uniqueItems, properties, required, additionalProperties,
minProperties, maxProperties, allOf, anyOf, oneOf, not.
Pointers follow RFC 6901 (/pets/1/a~1b). For a missing required property the pointer names the
missing member (/created_at), which is more useful in a report than the parent object.
Lint rules (apiprobe lint)
| Rule | Severity | Meaning |
|---|---|---|
duplicate-operation-id |
error | two operations share an operationId |
path-parameter-mismatch |
error | {var} without a path parameter, or a path parameter not in the template |
no-responses |
error | operation documents no responses |
missing-operation-id |
warning | operation has no operationId |
missing-response-schema |
warning | a 2xx (non-204) response has no schema |
missing-request-schema |
warning | request body without a schema |
missing-parameter-schema |
warning | parameter without a schema |
no-success-response |
warning | no 2xx/default response |
body-on-safe-method |
warning | GET/DELETE declares a request body |
inconsistent-operation-id-style |
warning | operationIds mix camelCase/snake_case/kebab-case |
inconsistent-property-naming |
warning | component schema properties mix naming styles |
inconsistent-path-naming |
warning | path segments mix naming styles |
no-error-response |
info | no 4xx response documented |
lint exits with status 1 when any error-level issue is found.