API REST & MCP evaluation

Valid JSON can still be wrong.

DocAI Bench scores structured output against versioned reference datasets. Review field verdicts and permitted diagnostic relationships.

Request access →Explore sample report

REST & MCPVersioned reference datasetsPrivate references stay private

docaibench.app · Product conceptProduct concept · Synthetic example

Illustrative financial statement

Two synthetic documents · six fields

DEV

Exact fields3 / 6synthetic example

Wrong2synthetic example

Missing1synthetic example

Documents2fictional

Field verdictsDEV

FieldOutputVerdict

total_assets120 000WRONG · ×1,000

equity15 000 000WRONG · sign

cash2 500 000EXACT

Source · synthetic

Statement

CZK thousands

Total assets120 000

Equity−15 000

×1,000Illustrative scale relationship; not a proven cause.

Explore product concept →

4versioned reference domains

35Czech filings in controlled holdout

86fields in the measured DEV run

$0.0063reported model cost of that run

Reference datasets

Start with a versioned set.

Separate DEV and controlled holdout access; availability depends on policy and grants.

Czech statements

35-document holdout · controlled access.

English statements

15 filings per version; access depends on the available version and entitlement.

Journey-log import

30 operator-owned synthetic documents.

Insolvency pairs

20 synthetic pairs; access depends on the available version and entitlement.

Versions, provenance and access →

SYNTHETIC EXAMPLE

The schema passed. The number was off by ×1,000.

A fictional statement, model output and verdict illustrate a scale mismatch.

01 Source

Statement

CZK thousands

Assets120 000

Equity−15 000

02 JSON

{
"assets": 120000
}

03 Verdict

WRONG · SCALE

×1,000

The model returned the printed number as whole CZK.

A numeric relationship is not a proven cause.

MEASURED · DEV

83 / 86 exact numeric fields

96.5% · 8 EN v1.2 DEV filings · Qwen3-14B, DeepInfra FP8 · 9 Nov 2026 · $0.0063 reported model cost.

XBRL-derived references; no cell-by-cell human review. The prompt converts printed scale, so this run does not test scale handling. This small DEV run is not a general accuracy estimate.

How it works

Evaluate in three steps.

01 · Select

Choose a versioned reference dataset.

02 · Run

Send structured output from your endpoint.

03 · Review

Inspect permitted verdicts and diagnostics.

REST API documentation · MCP documentation

Evaluate with a reference.

Get in touch to discuss access.

Email contact@docaibench.app →