English
A coronary narrowing can look different depending on the angle of the X-ray camera. Cardiologists therefore review several projections before deciding where a lesion is and how severe it appears. Earlier angiography AI models generally assessed individual frames or videos, leaving information from the rest of the examination unused.
DeepCORO-CLIP learns from angiography videos and the reports written about them, then combines multiple views for assessment of an examination. The study, published in Nature Cardiovascular Research, evaluates this approach for stenosis measurement and lesion characterization, and tests whether the learned representation can also support estimates of heart function and future cardiovascular events.
The development cohort included 203,808 videos from 32,473 studies in 28,117 patients at the Montreal Heart Institute, paired with 56,299 catheterization reports. Patients were kept separate across training, validation and test sets. Diagnostic acquisitions were selected before intervention, so the model learned from the arteries before angioplasty changed their appearance. Patients with previous coronary bypass surgery were excluded.
Training took place in two stages. First, contrastive learning aligned the representation of each angiography video with its corresponding report text. This taught the encoder to associate visual features with descriptions of coronary anatomy and disease. Second, the encoder was fine-tuned with attention pooling to combine information across videos and make predictions for 18 coronary segments.
The diagnostic model estimates stenosis severity and identifies obstructive disease, chronic total occlusion, intracoronary thrombus and calcification. Separate fine-tuning experiments adapted the pretrained encoder for left ventricular ejection fraction (LVEF) estimation and cardiovascular event prediction.
In the internal test set of 4,828 studies, DeepCORO-CLIP detected obstructive stenosis with a global AUROC of 0.89 (95% CI 0.88–0.89). At the Youden operating point, sensitivity was 0.85 (95% CI 0.84–0.86) and specificity was 0.70 (95% CI 0.70–0.71). The study used an obstructive threshold of at least 70% diameter stenosis, with a 50% threshold for the left main coronary artery.
The model also estimated the percentage narrowing and assessed other lesion features. All results below are from the internal test set; values in parentheses are 95% confidence intervals.
Combining views through attention pooling improved discrimination compared with using a single video or averaging video predictions. Video–text pretraining also improved performance over a video encoder pretrained on general video data. These comparisons support learning from both the clinical reports and the different projections within an angiogram.
DeepCORO-CLIP was externally evaluated on 4,249 studies from the University of California, San Francisco. Obstructive-stenosis discrimination was maintained, with an AUROC of 0.89 (95% CI 0.88–0.89). Continuous stenosis estimation was less accurate externally: mean absolute error was 12.1 percentage points (95% CI 11.9–12.3), compared with 9.2 (95% CI 9.1–9.4) internally. This distinction matters: detecting obstructive disease transferred more consistently than estimating its exact severity.
The investigators also compared predictions with an independent core laboratory using quantitative coronary angiography, rather than relying solely on the original clinical reports. Two interventional cardiologists reviewed 662 studies containing 965 lesions, with a third reader resolving large disagreements.
Against these adjudicated measurements, DeepCORO-CLIP had a mean absolute error of 13.6 percentage points (95% CI 12.5–15.2), compared with 19.0 (95% CI 17.6–20.5) for the original clinical reports. The model's correlation with the adjudicated measurements was 0.64 (95% CI 0.58–0.70). This was a separate evaluation with a different reference standard from the main test-set analysis.
The pretrained encoder was adapted to estimate LVEF using coronary angiography videos paired with echocardiograms obtained within 30 days. The held-out test set contained 2,530 studies.
DeepCORO-CLIP reduced estimation error compared with CathEF, an earlier model developed for this task. Values in parentheses are 95% confidence intervals.
These estimates came from the coronary angiography videos themselves. Echocardiography supplied the reference measurements for training and evaluation; it was not an input to the model at inference.
In a separate experiment, the encoder was fine-tuned to predict major adverse cardiovascular events (MACE) from diagnostic angiography. The study defined this composite as urgent revascularization, nonfatal myocardial infarction or complete coronary occlusion occurring after the first week and within one year of the examination.
The held-out test set included 350 patients, of whom 111 experienced a composite event. The model used angiographic videos without demographics, laboratory measurements or clinical risk factors as inputs. AUROCs and their 95% confidence intervals were:
Composite discrimination was higher than that of a clinical-variable baseline evaluated in the same cohort, and adding those variables to the model score did not improve discrimination. These are initial results from a relatively small cohort. Follow-up was often shorter than a year, and patients without documented follow-up were excluded, so the study does not establish a fully validated one-year risk estimate for every patient undergoing angiography.
The study also examined the quality of text reports produced from the model's predictions. These were generated using a reporting template that described stenosis severity and lesion features by coronary segment. They were not unrestricted answers from a language model.
Two interventional cardiologists reviewed 365 reports across 288 angiograms without knowing whether each report came from DeepCORO-CLIP or the operator. Both report sources were displayed in the same format to prevent wording from revealing their origin.
DeepCORO-CLIP reports were judged clinically usable in 75.9% of evaluations (132/174; 95% CI 69.0–81.6%), compared with 88.5% for operator reports (169/191; 95% CI 83.2–92.3%). The paper reported lower odds of usability for model reports: odds ratio 0.41 (95% CI 0.23–0.72).
The main shortfall was missed disease, particularly proximal obstructive stenoses. Hallucinated findings were flagged in 17.2% of model-report evaluations (30/174; 95% CI 12.4–23.5%) and 13.6% of operator-report evaluations (26/191; 95% CI 9.5–19.2%). The corresponding odds ratio was 1.32 (95% CI 0.75–2.34), with no statistically significant difference detected.
The percentage confidence intervals in this section are Wilson intervals calculated from the published evaluation counts. The odds ratios and their confidence intervals are reported in the paper. Some angiograms were assessed by both readers, so these percentages describe report evaluations rather than independent patient outcomes.
The reporting pipeline used a probability threshold of 0.50 to call a segment abnormal. This favored precision but left some obstructive lesions unreported. Lower thresholds recovered more lesions while increasing false-positive findings.
The released pipeline allows thresholds to be configured by coronary segment. However, the alternative thresholds in the paper were selected and assessed using the same test set. Their apparent gains therefore need confirmation with prespecified thresholds and new patients. The reader study supports further evaluation as a reporting aid with clinician review.
DeepCORO-CLIP was integrated with PACS-AI, which receives angiography studies, prepares the videos, runs inference and makes the analysis available in the PACS viewer. In the deployment experiment, this process completed within seconds, including preprocessing and template-based report generation.
This integration lets clinicians review the results alongside the original images. The study did not establish whether using the system changes treatment decisions or improves patient outcomes; those questions require prospective evaluation in clinical practice.
Performance was strongest in larger proximal vessels. Distal and side-branch disease remained harder to assess, and estimates in those segments require particular care. The study excluded patients with previous bypass surgery, and external validation was limited to one additional institution. LVEF and event-prediction results also need validation outside the development center.
DeepCORO-CLIP provides a way to learn from routine angiography videos and reports, combine the views within an examination, and adapt the resulting encoder to several clinical tasks. The released code and model weights allow other groups to test the approach on their own data and examine where it transfers well and where it needs further work.
The paper makes the weights available under Apache 2.0 and the code available for noncommercial research under the MHI research use agreement. Individual checkpoint access conditions are listed on Hugging Face. Patient datasets are not publicly released because of privacy restrictions.