Trust and Coverage

Evaluation card

Run over: the mount whose corpus file is sha256:0542b404f5188faaa2bc629e2712310c5b7659935d35ab4c9c1b5705131c44af, asked through the real handler; the cases are derived from its own Luxembourg and EU indexes

The same card for machines, as JSON

This card holds only the machine gates listed below (temporal_exactness; verdict_exact_match; anchor_ndcg_at_10; no_hit_accuracy; resolver_exactness). It does not hold the gates on the data (that every count matches the publisher's and that every admitted file yields text), on the quality of answers, on speed, or on two systems deriving the same index, and it tests none of the refusal codes no operation sends (advice_boundary; derivation_refused; not_transposable; upstream_unreachable).

Every machine gate on this card passes, and every shuffled control caught its shuffle.

Shuffled controls

• date_shuffle on temporal, as_of in eng: caught the shuffle, every expectation broke after the shift (seed 20260930, 1 cases, digest fbbeb194df09e84a4689151e00d2ef8bda1645848ba4bfb5b7d612255578a44e). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, in_force_on in eng: caught the shuffle, every expectation broke after the shift (seed 20260930, 1 cases, digest fbbeb194df09e84a4689151e00d2ef8bda1645848ba4bfb5b7d612255578a44e). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, as_of in fra: caught the shuffle, every expectation broke after the shift (seed 20260930, 609 cases, digest 1f31a84cbb4733c198876c4c089c66ce2e06594a54b984b7ed91945c95d103be). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, in_force_on in fra: caught the shuffle, every expectation broke after the shift (seed 20260930, 802 cases, digest b2c0bc1bbdc4aaada702bbbdde1fbb7743abc17a6746ecee348fbc0ea2b4ee8a). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, as_of with no language: caught the shuffle, every expectation broke after the shift (seed 20260930, 610 cases, digest e44398a3e5d3e25bfc7adff089919385b9723a809bd897991ff17908f36c0a78). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, in_force_on with no language: caught the shuffle, every expectation broke after the shift (seed 20260930, 803 cases, digest 7a2b4ca3ddcefc510b3346a35bd8ab892e87996ee3205caba3a1fa9852ce13e9). the control shifts every case forward by 129 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, EU as_of in eng: caught the shuffle, every expectation broke after the shift (seed 20260930, 302 cases, digest 953ed951c1a331e91679bb03701ac13f7bf5818e1a7d9630bf5581759d881b9f). the control shifts every case forward by 252 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, EU as_of in fra: caught the shuffle, every expectation broke after the shift (seed 20260930, 296 cases, digest 6b06b5f3e821870ca702ae47ec7e87347fbd6ab266bb40ff9db186f88380938a). the control shifts every case forward by 252 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• date_shuffle on temporal, EU as_of with no language: caught the shuffle, every expectation broke after the shift (seed 20260930, 295 cases, digest 7d1bb4e8f9ac0ff1e85f141c1bb5544b13053e987b79c31fd24af0a632b3cb0c). the control shifts every case forward by 252 days, the median gap between held dates; a case the shift cannot change (its timeline selects the same after it) is outside the control by its design.
• verdict_shuffle on refusal, the served operations, cases derived from the mount; not produced here: no_corpus_mounted (the mount holds an EU index, so an EU request is not refused for want of one); profiles_differ (no two held states of one work, each served alone on its own date, differ in rule profile): caught the shuffle, exact match fell to 0, within the base-rate ceiling 0.1923 (seed 20260930, 26 cases, digest 12e7327f61e3d4185002f2d5d5c057cd7bd228f438b8af73623f954667096ae0).
• qrels_shuffle on retrieval, search and verify, cases derived from the mount's own text: caught the shuffle, anchor nDCG@10 fell to 0 and both invariant gates stopped passing (seed 20260930, 77 cases, digest 549be02e21abb0d1e1190c8c9a4945043ce8d395b844aecb01cdcd426fd8972b).

Statistical rows

• D1, retrieval qrels v3: not yet labelled. Gates retrieval (section 1.4): anchor nDCG@10 against the baseline minus its MDE, by paired permutation test. Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D2, derivation audit set: not yet labelled. Gates derivation fidelity (section 1.2). Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D3, hard-negative and refusal set: not yet labelled. Gates typed absence and refusal correctness (section 4). Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D4, assistant verdict set: not yet labelled. Gates assistant contract (section 1.6): verdict exact match, with self-agreement. Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D5, span-support set: not yet labelled. Gates span-level support scoring (section 3). Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D6, crosswalk gold: not yet labelled. Gates the gated hybrid and crosswalk lane's activation against keyword. Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D7, temporal correctness set: not yet labelled. Gates as_of, timeline, diff and knowable_on correctness beyond the golden cases. Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.
• D8, grader audit set: not yet labelled. Gates grader-versus-human agreement; the grader never gates. Decision 92: the human-labelled datasets gate Stage 8 (S8-A03, S8-A11), after launch.

Negative results

• Hypothesis: hybrid retrieval serves the lay population it claims to serve better than the keyword lanes. Dataset: D6 crosswalk gold, not yet labelled. Result: not measured. Decision: not shipped: hybrid retrieval is not activated; search serves only the keyword lanes and refuses a ranked mode with retrieval_mode_unavailable. What would reverse it: passing the activation gate as 36-ideal-evaluation.md states it: on D6, recall@10 of the governing anchor at least 0.7 in English and German and at least 0.5 in Luxembourgish, with the keyword baseline printed beside it and the interpretation banner correct 100 percent of the time (section 2), and the published benchmark's conceptual nDCG +10 percent uplift condition (section 1.4, repair 6).