Documentation
Evaluate an extraction endpoint.
Integration reference for the existing DocAI Bench service. Access and credentials depend on your entitlement.
How it works
With service access, an integration can select an available versioned set and submit an endpoint. The service sends documents to that endpoint and compares structured output with reference values. Exact, wrong, missing and invalid verdicts retain their original scoring meaning.
The existing scorer can report exact matches and selected scale or sign relationships. These relationships do not establish why an error occurred.
Reference sets
Versions, access and provenance. Holdout counts describe dataset composition; access depends on the available version and entitlement.
| Domain | Scope | Fields / task | Split & provenance |
|---|---|---|---|
| Czech financial statements | 35 documents | Assets, equity, revenue, net income and more | 35 holdout · model-adjudicated |
| English financial statements v1.1 | 15 filings | Set-specific financial schema | 8 DEV / 7 holdout · XBRL-derived references |
| English financial statements v1.2 | 15 filings | Set-specific financial schema | 8 DEV / 7 holdout · XBRL-derived references |
| Journey-log import | 30 synthetic documents | Date, distance, fuel and journey details | 12 DEV / 18 holdout · operator-owned synthetic |
| Insolvency claim–schedule pairs | 20 synthetic pairs | Amounts, match and discrepancy evidence | 10 DEV / 10 holdout · operator-owned synthetic |
Reference methods and licences vary by set. No current set has human cell-by-cell review. English v1.1 does not test scale handling: its prompt specifies the printed-scale conversion. v1.2 uses the same prompt; its scale handling has not been independently validated.
REST API
The existing REST service base is https://bench.cz-agents.dev. Paths below are relative to this base. Authenticate with a bearer token; access and document transfer require appropriate authorization.
GET /v1/reference-sets— list available setsPOST /v1/runs— start a benchmarkGET /v1/runs/{run_id}— check statusGET /v1/runs/{run_id}/report— get the report
MCP
The existing authenticated Streamable HTTP endpoint path is /mcp on the service base above. Tool names: list_reference_sets, run_benchmark, get_report, get_dev_outputs and diff_runs.
DEV and BLIND
DEV and BLIND access depend on the version and entitlement. Available report detail varies by configuration. Contact us to confirm the permitted use and available controls.
Reference provenance varies by set: model adjudication, XBRL derivation and synthetic construction are possible. No current set has human cell-by-cell review. English v1.1 does not test scale handling: its prompt specifies the printed-scale conversion. v1.2 uses the same prompt; its scale handling has not been independently validated. Set-specific access and provenance are provided on request.