A Novel Framework for End-Diastolic and End-Systolic Frame Localization in Contrast and Non-Contrast Echocardiography Without Manual Annotations

Sahaj Patel, BS, MS, PhD; Neeraj Kummaragunta, BS; Krishin Yerabolu, N/A; Abdulla Shahid, BS; Priyank Baria, BSN; Cynthia Li, N/A; Nehal Vekariya, MS; Akhil Pampana, MS; Pankaj Arora, MD; Garima Arora, MD

Paper (AHA / Circulation Abstract 4373195): https://www.ahajournals.org/doi/10.1161/circ.152.suppl_3.4373195

A fully automated pipeline for localizing end-diastolic (ED) and end-systolic (ES) frames in echocardiography cine loops using optional LV ROI detection (YOLO) and Robust PCA + SVD motion signal extraction—without requiring manually annotated ED/ES reference frames.

---

Table of Contents

• Project Description
• Why This Project
• Method Overview (Theory)
  • 1) ROI / Cropping
  • 2) Robust PCA Decomposition
  • 3) Motion Signal Extraction via SVD
  • 4) Cycle Selection (Spectral Dominance Ratio)
  • 5) ED/ES Localization (Peak/Valley Detection)
  • Exclusions / Failure Cases
• Results Summary
• Repository Layout
• Installation
• How to Run
  • Supported Input Modes
  • Crop Modes
  • Quick Start: Single AVI (Inference-only)
  • Quick Start: Batch from Filename List (Inference-only)
  • Quick Start: EchoNet CSV + AVI (Labeled)
  • Quick Start: UAB NPY + PKL (Labeled)
  • Column Mapping for PKL Tables
  • Batch Slicing
• Outputs
• Tips
• Citations / Acknowledgements

---

Project Description

This repository implements a generalizable ED/ES frame localization framework for echocardiography cine loops that works on contrast and non-contrast images. The approach uses:

• Optional LV ROI detection via a YOLO (v12) object detector (or fixed crop / full-frame),
• Robust principal component analysis (RobustPCA) to separate low-rank structure from sparse variations,
• SVD-derived temporal signals to capture pseudo-periodic cardiac motion,
• Spectral dominance + zero-crossing stability to select the best motion signal,
• Peak/valley detection to infer ED and ES frames.

The method is designed to avoid reliance on manually annotated ED/ES training labels, which can be costly, variable, and dataset-specific.

---

Why This Project

End-diastolic (ED) and end-systolic (ES) frames are critical for left ventricular (LV) volume measurements in echocardiography, but manual selection can have high inter- and intra-observer variability.

While deep learning methods have advanced ED/ES localization, many approaches require manually labeled ED/ES reference frames and may fail to generalize across different image types and acquisition conditions (especially contrast vs non-contrast).

This project focuses on a fully automated, annotation-free ED/ES localization method that can operate with or without a DL-based ROI detector.

---

Method Overview (Theory)

1) ROI / Cropping

The pipeline supports three crop modes:

• yolo: YOLO crop (select the single best-confidence LV box across all frames, and crop the entire cine loop to that ROI)
• full: no crop (use full frame)
• fixed: fixed crop rectangle (X1,Y1,X2,Y2)

Cropping helps reduce background clutter and stabilize the extracted motion signal, especially for contrast studies.

2) Robust PCA Decomposition

After cropping (or using full frame), the cine loop is reshaped into a matrix representation and decomposed using Robust PCA into:

• Low-rank (L): dominant structural content (stable anatomy/appearance)
• Sparse (S): transient variations / motion / outliers

This decomposition yields a more stable basis for extracting pseudo-periodic cardiac motion.

3) Motion Signal Extraction via SVD

SVD is applied to the low-rank matrix. The pipeline extracts the top three left singular vectors (U1, U2, U3), each serving as a candidate 1D temporal motion signal.

4) Cycle Selection (Spectral Dominance Ratio)

Each candidate U signal is analyzed to identify pseudo-periodic cardiac cycles using a Spectral Dominance Ratio (how strongly a dominant frequency stands out).

The algorithm then:

• detects cycles,
• computes zero-crossings and their variance,
• selects the U with lowest zero-crossing variance (and at least two cycles) as the representative cardiac motion signal.

5) ED/ES Localization (Peak/Valley Detection)

A peak detection step identifies local extrema in the selected motion signal:

• one extremum corresponds to ED (typically maximal filling),
• the opposite extremum corresponds to ES (typically maximal contraction).

This yields predicted ED/ES frame indices for each cine loop.

Exclusions / Failure Cases

If the selected motion signal contains only one cardiac cycle, the case is excluded (insufficient periodic structure for stable ED/ES localization). This is reported in the outputs so excluded cases are traceable.

---

Results Summary

Validated on:

• UAB dataset: N=984 (912 contrast, 72 non-contrast)
• EchoNet-Dynamic: N=10,030 (non-contrast) for external validation

YOLO model (trained exclusively on UAB data: 1394 train / 298 val / 300 test):

• mAP50 = 0.994
• mAP50–95 = 0.717

Frame localization mean absolute error (MAE):

• UAB: ED = 2.65 ± 2.95 frames (median 2), ES = 1.58 ± 1.49 frames (median 1)
• EchoNet: ED = 3.75 ± 4.02 frames (median 2), ES = 2.72 ± 2.81 frames (median 2)

Excluded due to only one cardiac cycle in the selected motion signal:

• 5 UAB cases
• 115 EchoNet cases

---

Repository Layout

Typical structure:

.
├── scripts/
│   └── run_edvesv.py
├── requirements.txt
└── README.md

Your local repository may include additional modules (RobustPCA utils, YOLO wrappers, helpers, etc.).

---

Installation

If your repo has a requirements.txt:

pip install -r requirements.txt

If your repo is packaged (has pyproject.toml or setup.py):

pip install -e .

If you run on GPU, ensure a CUDA-enabled PyTorch build is installed.

---

How to Run

The main entry point is:

python scripts/run_edvesv.py [args...]

Supported Input Modes

This repo supports labeled evaluation and inference-only workflows:

Inference-only (no ED/ES required)

• --dataset single_avi
  Run on a single .avi file and output predicted ED/ES frames.

• --dataset echonet_list
  Run on a batch of videos specified by a text file of filenames (one per line).
  This is useful when you only have filenames and want predictions without any ED/ES labels.

Labeled evaluation (ED/ES provided)

• --dataset echonet_csv
  AVI videos + a CSV that includes FileName (and optionally EDV, ESV for evaluation).

• --dataset uab_pkl
  NPY arrays + a PKL table (column names can be mapped).

Note: --output_csv is always required. Even in inference-only mode, results are written to a CSV.

---

Crop Modes

• --crop_mode yolo
  Uses YOLO weights from --yolo_weights and crops to the best LV ROI.

• --crop_mode full
  Uses full frame (no cropping).

• --crop_mode fixed --fixed_crop X1 Y1 X2 Y2
  Uses a constant rectangle crop.

---

Quick Start: Single AVI (Inference-only)

python scripts/run_edvesv.py   --dataset single_avi   --video_path /path/to/video.avi   --crop_mode yolo   --yolo_weights /path/to/yolo_weights.pt   --output_csv out_single.csv

Optional: set a custom identifier for the output row:

python scripts/run_edvesv.py   --dataset single_avi   --video_path /path/to/video.avi   --case_id MyCase123   --crop_mode full   --output_csv out_single.csv

---

Quick Start: Batch from Filename List (Inference-only)

Create a text file (e.g., filelist.txt) with one filename per line (with or without .avi).
Blank lines are ignored. Lines starting with # are treated as comments.

Example filelist.txt:

# Example filenames
CASE_001
CASE_002.avi
CASE_003

Run:

python scripts/run_edvesv.py   --dataset echonet_list   --video_dir /path/to/video_dir   --file_list /path/to/filelist.txt   --crop_mode full   --output_csv out_batch.csv

---

Quick Start: EchoNet CSV + AVI (Labeled)

1) YOLO crop

python scripts/run_edvesv.py   --dataset echonet_csv   --video_dir /path/to/video_dir   --input_csv /path/to/labels.csv   --yolo_weights /path/to/yolo_weights.pt   --crop_mode yolo   --output_csv out_yolo.csv

2) Full-frame (no crop)

python scripts/run_edvesv.py   --dataset echonet_csv   --video_dir /path/to/video_dir   --input_csv /path/to/labels.csv   --crop_mode full   --output_csv out_full.csv

3) Fixed crop

Replace with your experiment's constants: X1,Y1,X2,Y2 = 24,14,88,94

python scripts/run_edvesv.py   --dataset echonet_csv   --video_dir /path/to/video_dir   --input_csv /path/to/labels.csv   --crop_mode fixed   --fixed_crop 24 14 88 94   --output_csv out_fixed.csv

---

Quick Start: UAB NPY + PKL (Labeled)

python scripts/run_edvesv.py   --dataset uab_pkl   --npy_base_dir /path/to/npy_base_dir   --input_pkl /path/to/table.pkl   --yolo_weights /path/to/yolo_weights.pt   --crop_mode yolo   --output_csv out_uab.csv

---

Column Mapping for PKL Tables

If your PKL column names differ, map them with:

• --id_col (case identifier; default tries FileName, filename, Patient)
• --ed_col (default tries EDV / ED)
• --es_col (default tries ESV / ES)
• --npy_col (relative .npy path; default tries npy_path, Path, FilePath)

Example:

python scripts/run_edvesv.py   --dataset uab_pkl   --input_pkl /path/to/table.pkl   --npy_base_dir /path/to/npy_base_dir   --id_col PatientFolder   --ed_col ED   --es_col ES   --npy_col NPYRelPath   --crop_mode full   --output_csv out.csv

---

Batch Slicing

To process only a subset of rows (useful for batching on HPC):

python scripts/run_edvesv.py ... --start_idx 5000 --end_idx 7500

For single_avi, slicing is typically unnecessary.

---

Outputs

The pipeline writes a CSV specified by --output_csv.

Typical output includes:

• identifiers (e.g., FileName / Patient / ID)
• predicted ED frame index
• predicted ES frame index
• crop mode and crop coordinates used (if applicable)
• status flags (e.g., excluded due to single-cycle, YOLO failure fallback, read failure, etc.)

---

Tips

• Use crop_mode=yolo for best robustness when backgrounds vary or contrast introduces clutter.
• Use crop_mode=full as a baseline or if YOLO weights are unavailable.
• Use crop_mode=fixed when you need strict reproducibility across experiments or your dataset is consistently centered.

---

Citations / Acknowledgements

• EchoNet-Dynamic dataset is used for external validation.
• YOLO (v12) is used for optional LV ROI detection.
• Robust PCA + SVD is used for extracting a stable pseudo-periodic cardiac motion signal for ED/ES localization.