FindingsScience read/Diagnostics/AI & ML

3 of 1,357 Cleared AI Devices Have Public Outcome Trials. The Study Has No Non-AI Comparator.

Athithi Verma· 28 August 2026· 3 min read· Synopulse

What they did

A Review article in PLOS Digital Health, published 19 August 2026, which the authors describe as a regulatory evidence census and structured evidence-mapping analysis rather than a PRISMA-style meta-analysis. Abulibdeh and colleagues at Toronto, MIT, Johns Hopkins, Mbarara and Bergen took all 1,357 AI and machine-learning enabled devices cleared or approved by the FDA through 5 December 2025, using the FDA device database and the ACR Data Science Institute catalogue, then scraped ClinicalTrials.gov identifiers from 510(k) summary pages and linked those registrations to PubMed publications.

Findings

The attrition is steep at every stage. Of 1,357 cleared devices, 34 (2.5%) were linked to registered prospective trials, 12 (0.9%) posted results, 12 (0.9%) reached peer-reviewed publication, and 3 (0.2%) evaluated patient-centred outcomes such as mortality, morbidity or readmission.
The 34 trials that exist are small, domestic and industry-run. Roughly 73.5% enrolled fewer than 500 participants and a quarter fewer than 100. 68% ran only in the United States. 32 of 34 (94%) were industry-led. Nine reported any subgroup analysis, with race or ethnicity in 3 and language in none.
Specialty concentration is extreme and inversely related to evidence. Radiology accounts for 78% of cleared devices (1,059 of 1,357) and has prospective trials for fewer than 1% of them. Cardiovascular and neurology sit at 9.5% and 9.7%. Anaesthesiology has 22 cleared devices and no registered prospective trials at all.
Almost nothing tests treatment guidance. Among the 34 trials, 59% addressed diagnostic applications and 21% screening. Three studies, 9%, addressed therapeutic or treatment guidance.

Science note

The mechanism the authors identify is the predicate chain. A 510(k) submission demonstrates substantial equivalence to an existing device rather than independent clinical effectiveness, so a device cleared against a predicate that was itself never prospectively validated inherits that gap and passes it on. That is a structural property of the pathway rather than a failure of any individual clearance, and it explains why the shortfall concentrates in radiology, where the predicate population is largest and oldest. It also means the fix the authors propose, a staged framework requiring retrospective validation before clearance and prospective outcome trials at n of 500 and then 2,000, is a change to the pathway rather than to any company’s behaviour.

LimitationsThe authors are direct about the boundaries. The census counts publicly registered evidence, so it may undercount proprietary validation held internally by manufacturers, and it deliberately excluded unregistered validation studies that exist in the published literature. Not all 510(k) summary pages were accessible in full. Most consequentially, there is no non-AI device comparator, so the analysis cannot establish whether these gaps are specific to AI devices or reflect medical device regulation generally. Device class and regulatory pathway were not extracted for all 1,357 devices, which rules out risk-stratified analysis and leaves open whether low-evidence devices are low-risk ones where a trial would be disproportionate.

DisclosureThe authors declare that no competing interests exist. Separately, the acknowledgments record that one author serves as a course lead and paid consultant to the CITI Program, developing an online course on artificial intelligence for healthcare providers. The authors also disclose using a large language model for language refinement, grammar editing and drafting the Python scripts used for data retrieval, stating that all outputs were reviewed and validated by them. Funding is public and philanthropic, including NIH DS-I Africa and Bridge2AI, the NSF, the Korea Health Industry Development Institute, Johns Hopkins ICTR and a Norwegian university consortium.

SourceAbulibdeh R, Cajas Ordóñez SA, Celi LA, Gorijavolu R, Izath N, Markussen Lunde T. 1,357 AI medical devices cleared, 3 actually tested on patient outcomes. PLOS Digital Health 2026;5(8):e0001597. Open access.

AI Medical Devices 510(k) Clearance Clinical Validation Radiology Evidence Standards