Repository navigation
fix(deformable-detr): preserve image aspect ratio - #1753
Conversation
Match the benchmark checkpoint, revision and input dimensions to the native manifest. Keep the accuracy reference usable after removal of the legacy qualification package by loading its optional performance harness only on that path. Preserve every existing numerical criterion. Refs: NVIDIA#1671 Signed-off-by: Yizhuo Zhang <yizhuoz@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Summary
Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Docstring Coverage |  |
Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 42 functions across 16 files. (2 skipped:… | Write docstrings for the functions missing them to satisfy the coverage threshold. |
| Benchmark Validation Integrity | |
The updated Deformable DETR benchmark compares different timed regions. The native worker starts timing immediately before IObjectDetection::detect and stops when it returns (`apps/benchmark/native/… |
Define one timing contract for the native and reference paths. Keep file decoding outside timing, but time equivalent image preprocessing, model execution, synchronization, and output decoding on both sides. For example, move the reference … |
-
Check name Status Explanation Title check The title clearly identifies the main production change: preserving image aspect ratio for Deformable DETR. Description check The description completes the required sections with background, exit criteria, implementation details, change categories, validation evidence, environment and revision data, remaining gaps, self-revi… Linked Issues check The PR addresses the coding requirements in #1671. It corrects the Deformable DETR checkpoint manifest, preserves aspect ratio with checkpoint resize bounds, builds dynamic TensorRT geometry, validate…Out of Scope Changes check The changes remain within #1671. Production changes are limited to the Deformable DETR family and implement its required preprocessing, dynamic engine geometry, manifest identity, and runtime validati…Family Ownership Boundary No cross-family dependency was introduced. The changed-file inventory contains only families/deformable_detr/**plus its documentation. Production imports inmodel.py:14-16are local family module…Shared Semantic Neutrality PASS. The diff contains no changed shared code that matches the check scope. All production and test changes are under the Deformable DETR family’s model-owned Python, runtime, C++ test, E2E, or famil… Shared Change Blast Radius The diff keeps production code, runtime contracts, benchmarks, and validation inside families/deformable_detr. The only shared-path change is a scoped addition to the generic AIPerf detection guide.…Full details: Description check
-
Explanation
The description completes the required sections with background, exit criteria, implementation details, change categories, validation evidence, environment and revision data, remaining gaps, self-review, notes, and risk rationale.
Full details: Linked Issues check
-
Explanation
The PR addresses the coding requirements in
#1671. It corrects the Deformable DETR checkpoint manifest, preserves aspect ratio with checkpoint resize bounds, builds dynamic TensorRT geometry, validates out-of-profile inputs, and keeps labels, scores, coordinates, and thresholds unchanged. It adds unit, C++, dynamic-geometry, and E2E coverage. The reported owning-case check passes. The PR also reports 100 native/HTTP comparisons and a real pip-installed AIPerf 0.13.0 run with separate HTTP and Task timing. It records the checkpoint revision, bundle digest, dataset, package versions, hardware, settings, warmups, and qualification limits. The nine previously qualified detectors and the remaining Deformable DETR checkpoint provide the reported ten-checkpoint coverage.
Full details: Out of Scope Changes check
-
Explanation
The changes remain within
#1671. Production changes are limited to the Deformable DETR family and implement its required preprocessing, dynamic engine geometry, manifest identity, and runtime validation. The benchmark manifest, accuracy harness import isolation, regression tests, qualification tests, and AIPerf guide support checkpoint qualification and reproducibility. No unrelated API, ABI, dependency, sibling-family, or custom-kernel change is shown.
Full details: Family Ownership Boundary
-
Explanation
No cross-family dependency was introduced. The changed-file inventory contains only
families/deformable_detr/**plus its documentation. Production imports inmodel.py:14-16are local family modules. New validation imports intests/dynamic_geometry.py:15-16are also local. The benchmark reference uses the sharedqualification_testsharness, not another model family. The priorPekingU/rtdetr_v2_r18vdcandidate was removed and replaced withSenseTime/deformable-detrin the benchmark diff. No central registry, switch, source list, or strategy map changed.
Full details: Shared Semantic Neutrality
-
Explanation
PASS. The diff contains no changed shared code that matches the check scope. All production and test changes are under the Deformable DETR family’s model-owned Python, runtime, C++ test, E2E, or family-tool paths. The only change outside those paths is Markdown documentation, which adds family-specific usage guidance and does not change shared code semantics.
Full details: Shared Change Blast Radius
-
Explanation
The diff keeps production code, runtime contracts, benchmarks, and validation inside
families/deformable_detr. The only shared-path change is a scoped addition to the generic AIPerf detection guide. The guide identifies the affected consumer commands and explains the compatibility impact: users must rebuild older Deformable DETR bundles because the engine is dynamic and preserves aspect ratio. The PR description and added family tests report configuration, native, dynamic-geometry, end-to-end, and AIPerf validation. The common guide is the correct shared consumer entry point; no shared implementation or contract was changed.
-
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 42 functions across 16 files. (2 skipped: 2 unsupported.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
Full details: Benchmark Validation Integrity
-
Explanation
The updated Deformable DETR benchmark compares different timed regions. The native worker starts timing immediately before
IObjectDetection::detectand stops when it returns (apps/benchmark/native/benchmark_worker.cpp:224-232, 687-710), so the PR's new aspect-ratio resize and preprocessing inruntime/pipeline.cpp:34-40are timed, while native box-to-JSON observation conversion is not. The reference prepares the image withprocessor(...)and transfers inputs to CUDA before its timedinvoke(tests/benchmark/reference.py:101-120), then times model execution, post-processing, device-to-host.cpu()transfers, list conversion, and output-map construction (:123-143). The corrected profile now activates this comparison for the SenseTime checkpoint (benchmark/deformable-detr-r50.yaml), and its 640x382 input exercises the new 796x1333 resize. Thus preprocessing is excluded on the reference side, while output conversion and host transfer/postprocessing are not accounted for equivalently.Resolution
Define one timing contract for the native and reference paths. Keep file decoding outside timing, but time equivalent image preprocessing, model execution, synchronization, and output decoding on both sides. For example, move the reference processor and input transfer into the timed callback and either move native detection observation/validation into the timed region or defer the reference
.cpu(),.tolist(), post-processing, and output-map construction until after timing. Add a regression that verifies the declared timing boundary and rerun the corrected Deformable DETR performance profile.
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
Use checkpoint resize bounds and dynamic spatial shapes for the native detector. Derive positions, reference points, normalization and attention geometry from the actual unpadded feature maps instead of stretching every image to a square. Add independent dynamic-geometry and resize regressions without changing existing correctness criteria. Older family bundles require rebuilding for the new resize metadata and dynamic engine. Refs: NVIDIA#1671 Signed-off-by: Yizhuo Zhang <yizhuoz@nvidia.com>
yizhuoz004
changed the title
Reject pathological aspect ratios that the checkpoint processor leaves outside the engine profile before invoking inference. Keep the builder optimization policy immediately after config creation as required by repository validation. Signed-off-by: Yizhuo Zhang <yizhuoz@nvidia.com>
github-actions
Bot
removed
the
run-internal-ci
Background
Stage 4 (#1671, epic #1667) has nine qualified object detectors from merged #1718 and one remaining supported checkpoint,
SenseTime/deformable-detr. Its benchmark profile incorrectly copied RT-DETR's checkpoint, revision and 640x640 size. The standalone accuracy reference also imported the qualification framework removed in #1742 before handling a request.Real corpus validation then exposed a native defect: stretching every image to 800x800 dropped mAP by 2.74 percentage points and mAP50 by 3.13 points versus the canonical aspect-preserving FP32 reference, exceeding both unchanged two-point gates. A diagnostic FP32 square-resize run closely matched native results, locating the first divergence at input geometry.
Exit Criteria
Qualify Deformable DETR's actual FP16 bundle through its unchanged owning checkpoint case, 100 original-image corpus/native/HTTP checks, existing mAP gates, and real published PyPI AIPerf. Reconcile every current-main detector and close stage 4 only after this PR merges. This finishes the ten canonical checkpoints for single-image object detection; other precisions and workloads retain separate status.
Implementation
Keep all production behavior and numerical checks in
families/deformable_detr. Read checkpoint shortest/longest-edge bounds and preserve image aspect ratio without padding. Build a dynamic TensorRT spatial profile and derive positions, reference points, GroupNorm reshapes and attention geometry from actual feature-map sizes. Reject invalid or out-of-profile geometry before inference. TensorRT still owns GPU lowering and tactics; no custom kernel, platform branch or sibling dependency is added.Correct benchmark identity and dimensions to match the existing pinned native manifest. Import the removed legacy performance harness only on that existing legacy mode so
--request/--outputaccuracy execution works independently. Add native resize/bounds cases, configuration tests and an independent NumPy/PyTorch dynamic-geometry regression inside the single owning E2E case. Update the detection guide with the model name, profile and rebuild instruction.The public Task API, ABI, bundle container format, dependencies, labels, coordinates, scores and every original correctness threshold remain unchanged. Family
runtime.jsongains required resize bounds and the plan becomes dynamic: rebuild older Deformable DETR bundles. There is no old-bundle fallback.Change categories
Validation
Commands and Results
On
cf93a041d8c0660f2e3e3ad0a4e4008b70cfd26cwith a clean worktree:PYTHONPATH=core/builder:. python -m pytest families/deformable_detr/tests/test_model.py -q: 18 passed.ctest --test-dir "$RUNTIME_ROOT" -R '^deformable_detr_preprocess_and_decode$' --output-on-failure: 1 passed, including aspect-ratio and engine-bound regressions.PYTHONPATH=core/builder:. python -m pytest families/deformable_detr/tests/test_e2e.py --e2e-model deformable-detr-r50 --e2e-testcase deformable-detr-r50 -q -s: one executed, non-skipped case passed. SetTRTMC_BINARY,TRTMC_RUNTIME_ROOT,TRTMC_DEFORMABLE_DETR_MODEL_DIRto the prepared CLI/runtime and exact pinned snapshot. This builds the actual bundle, checks dynamic positions/attention against independent NumPy/PyTorch for landscape/portrait/square shapes, then compares native checkpoint detections with the official reference. The fixture reports worst IoU 0.9953 and score delta 0.0074; original IoU >= 0.5 and score delta <= 0.05 rules are unchanged.PYTHONPATH=apps/aiperf_qual:apps/benchmark:apps/perf_serving:core/builder:. python -m trtmc_aiperf_qual endpoint-profile --environment "$ENVIRONMENT" --profile deformable-detr-r50 --bundle "$BUNDLE" --inputs "$INPUTS/inputs.jsonl" --out "$RESULTS": 100 complete native/HTTP input-output comparisons passed, followed by 50/50 measured requests and 5/5 warmups through real PyPI AIPerf; 463 complete boxes (2315 coordinate/confidence values) checked, zero client errors or missing request/metric joins.ENVIRONMENTselects the prepared native runtime, server interpreter and pip-installed client;BUNDLEis the exact owning-case artifact below. PrepareINPUTSusing the checked-inapps/aiperf_qual/examples/prepare_detection_inputs.pyas shown in the usage guide._box_iou,_detections,_coco_ap_at_iou,_coco_detection_metrics. The evaluator is kept outside source commits; this does not restore the removed framework.PYTHONPATH=core/builder:apps/benchmark:apps/aiperf_qual:. python -m pytest tools/tests/test_architecture.py tools/tests/test_family_impact.py apps/aiperf_qual/tests/test_endpoint_catalog.py apps/aiperf_qual/tests/test_endpoint_profile.py -q: 91 passed in a detached exact-head review checkout.PYTHONPATH=core/builder:apps/benchmark:. python -m tools.model_ci validate,python tools/test_impact.py --validate, Ruff, clang-format andgit diff --check github/main...HEAD: pass. Impact selects onlydeformable_detrplus generic CPU/docs checks.npm --prefix website run test:model-support: 14 passed;npm --prefix website run build: pass, including all 34 diagram checks.Ten-detector benchmark results
All rows use one complete original RGB image per request, FP16/TP1, one persistent replica, concurrency 1, five warmups and 50 measured requests, published AIPerf 0.13.0,
/v1/inferwith endpoint typeimage_retrieval. HTTP latency andpublic_task_call_wallare separate; native Task time includes family preprocessing/inference/postprocessing and excludes transport decoding. Build/load/startup, correctness/verification calls and warmup are excluded from measured timing. These short integration runs establish no capacity or performance gate.The nine existing rows retain their original tested source
ad6ca2e0c60b2327c2c7a62350a54dda5b5f3f9band bundle/checkpoint identities in #1718. Their families and serving/runtime paths are unchanged by this PR and intervening main changes; they were not remeasured on this head. Deformable DETR is newly measured on the head above. The development host is shared, so the observed timings are not controlled speedup comparisons.deformable-detr-r50detr-resnet-50rtdetr-v2-r18yolo11nyolos-smallyolov10nyolov10syolov10xyolov5nyolov8nCorpus values below are percentages on the same 100 original COCO images. Preserve each family's label space and filtering. YOLOS has no existing dataset benchmark gate; its unchanged owning reference case passes separately.
deformable-detr-r50detr-resnet-50rtdetr-v2-r18yolo11nyolos-smallyolov10nyolov10syolov10xyolov5nyolov8nHardware, Environment, and Revisions
Base
268e03e56c9206d21611e5943c656ab97c4a08e1; tested sourcecf93a041d8c0660f2e3e3ad0a4e4008b70cfd26c. One NVIDIA GB300, Linux aarch64, Python 3.12.3, CUDA toolkit 13.3, driver 580.105.08, TensorRT 11.1.0.106, PyTorch 2.12.0+cu130, Transformers 5.2.0, timm 1.0.28, Pillow 12.3.0, published AIPerf 0.13.0. The strict FP32 micro-regression disables TF32 only for its probe; production FP16 arithmetic is unchanged.Checkpoint
SenseTime/deformable-detr@83ecd26945199939cb82806f988debdb71e6f43e. Profile: FP16, TP1, 800-pixel shortest-edge target / 1333-pixel longest-edge bound, confidence threshold 0.3, top 100. Datasetdetection-datasets/coco@cf0b22332314a937e9dc8a1957b21725430bb41d, validation split, the same pinned first-100-eligible original RGB selection as #1718, with complete dimensions, byte hashes and annotations retained. Reference is canonical FP32.Actual native-correctness/served bundle SHA-256:
4001c40acaaac239e6c65f2f8eb6f38985c07f943bee200447a2311520f7879e(89283054 bytes). All generated bundles, selections, records and reports are preserved outside commits.Not Run / Remaining Gaps
Other precisions, hardware targets, TP variants, batching, multiple images, streaming, document understanding outside the detection Task, and capacity sweeps are unqualified by these runs. The legacy performance CLI still depends on the upstream-deleted framework and is not restored or claimed here. Current-head CI must finish before merge. No additional canonical checkpoint gap remains in the refreshed ten-detector inventory.
Contributor Self-Review
Notes For Future Readers
Review checkpoint geometry first, then dynamic graph layout and runtime bindings, then the independent regression and corpus evidence. Rebuild older Deformable DETR artifacts with the corrected family. All production changes are family-local; shared serving code preserves the family's numeric labels, complete boxes/scores and coordinates without reinterpretation. No new third-party dependency, vendored source, binary, weight, fixture or dataset is redistributed. The pinned checkpoint was already owned by the native manifest; benchmark metadata now matches it.
Closes #1671. Other epic workloads keep their own stages and evidence.
Risk level
The family now uses aspect-preserving preprocessing, dynamic spatial tensors and mandatory resize metadata. Exact-bundle model/corpus/client proof passes without relaxing any gate; older bundles require rebuilding, and additional targets/configurations remain untested.