The Audit Run and What It Costs in Credits

The estimate

Pressing Run audit on Overview or Questions opens a dialog that prepares an estimate without spending anything. The count is the project's approved buying questions times the enabled engines times the samples per question you chose, bounded by the same per-run cap a real audit obeys. If you pick more than one sample per question the dialog says so, and the button reads Confirm & run.

Through MCP, answer_audit_estimate does the same with an optional explicit prompt list and samples_per_prompt, and answer_usage_get tells you whether the account has that many credits left before you spend them.

Reservation, fan-out, aggregation

Confirming reserves budget for the estimated samples, then fans out one task per question per engine per geography per sample, each with a stable index in the run so a duplicate delivery is detectable everywhere downstream. Each task asks the engine through its official API with grounded web search and records the answer text, the citations, and the searches the engine issued while answering.

When the samples are in, the run aggregates them into gaps: buying questions where an engine recommended somebody else, or nobody. The run specification, prompts, engines, geographies, language and samples per prompt, is frozen at creation so the run can be reproduced exactly for re-verification.

Run states

A run is pending, running, partial, completed, failed, cancelled or budget_exceeded. Partial means some samples failed and the rest were measured; the failures stay visible and are excluded from metrics by grade, never converted into zero visibility. Budget_exceeded means a cap refused the run before or during fan-out. History lists every run with its status, start and finish, sample count and credits.

Open a run to see Answers and Citations sample by sample. Each sample names the engine that answered and an opaque instrument token, never a model id, so two runs on different model snapshots can be told apart without exposing the snapshot.

Credits, exactly

A credit is a true count of measured samples, read off the usage ledger, not a conversion from spend. Reservations are a flat rate times the sample count so the estimate divides back to that count exactly; charges are the count itself. Where a ledger row lacks a count it is reported as incomplete rather than summed as zero.

Usage shows what has been reserved and consumed today, the remaining daily allowance, the per-run cost cap, whether the operator kill switch is engaged, and the per-run size limits. A cap that reads null is not configured; that is not unlimited and it is not zero. A platform-wide daily cap you cannot see also applies, so an allowed estimate is not a guarantee.

• 10 questions, 4 engines, 1 sample each: 40 credits.
• 20 questions, 4 engines, 2 samples each: 160 credits.
• 50 questions, 4 engines, 1 sample each: exceeds the 200-sample per-run cap and is refused before it starts.

FAQ