Skip to content

layout_parsing: unguarded int() crop of layout blocks crashes the whole page on degenerate boxes and silently wraps on negative coordinates #5233

Open

Description

@rpvelloso

Title:

layout_parsing: unguarded int() crop of layout blocks crashes the whole page on degenerate boxes and silently wraps on negative coordinates

Body:

Describe the bug

In _LayoutParsingPipelineV2.get_layout_parsing_objects()
(paddlex/inference/pipelines/layout_parsing/pipeline_v2.py, lines 767-772 in
v3.7.2; still present on develop as of 2026-10), every layout block with a
visual/table label gets its auxiliary image crop via:

if (
    label
    in ["seal", "table", "formula", "chart"]
    + BLOCK_LABEL_MAP["image_labels"]
):
    x_min, y_min, x_max, y_max = list(map(int, block_bbox))
    img_path = (
        f"imgs/img_in_{block.label}_box_{x_min}_{y_min}_{x_max}_{y_max}.jpg"
    )
    img = Image.fromarray(image[y_min:y_max, x_min:x_max, ::-1])
    block.image = {"path": img_path, "img": img}

block_bbox comes straight from the layout detection model
(box_info["coordinate"], float coordinates) with no validation or clamping.
Both int() (truncates toward zero) and the raw numpy slice are unsafe
against the fractional, degenerate, or negative boxes the detector actually
emits in production.

Reproduction

No model, GPU, or input document is needed — the arithmetic itself fails:

import numpy as np
from PIL import Image

page = np.zeros((300, 400, 3), dtype=np.uint8)  # H=300, W=400 page

# 1. Sub-pixel box: int() truncation turns it into a zero-area crop
x_min, y_min, x_max, y_max = list(map(int, [100.4, 200.6, 100.9, 210.9]))
Image.fromarray(page[y_min:y_max, x_min:x_max, ::-1])
# ValueError: tile cannot extend outside image

# 2. Fully-negative box: no crash — numpy slicing wraps and silently returns
#    pixels from the opposite edges of the page:
page = np.arange(100 * 200 * 3, dtype=np.uint8).reshape(100, 200, 3)
x_min, y_min, x_max, y_max = list(map(int, [-30.5, -20.5, -10.2, -5.1]))
crop = page[y_min:y_max, x_min:x_max]
# crop.shape == (15, 20, 3), starting at page row 80 — nowhere near the box

# 3. Partially-negative box [-5.5, 10.2, 50.9, 30.1]: x_min = -5 wraps the
#    column axis; combined with the valid x_max the slice comes out
#    zero-width -> same ValueError as case 1.

We hit this in a production PP-StructureV3 deployment (served through the
official HPS/Triton image): the layout model emitted a degenerate/out-of-page
table box and the whole request died with ValueError: tile cannot extend outside image. The two case-1/case-3 shapes above reproduce it through the
installed get_layout_parsing_objects() with fabricated layout/OCR results.

Impact

  • One degenerate box aborts the whole page. The crop runs inside the
    predict() generator, so the exception propagates out of the pipeline: no
    markdown, no imgs/... assets, and all already-recognized content for the
    page (including the table HTML in block.content) is lost. In serving this
    fails the entire document job, and because the error surfaces as a generic
    backend failure, clients typically retry it pointlessly.
  • Not disable-able by configuration. The crop runs for every table block
    when use_table_recognition=True, and for image/seal/chart/formula blocks
    whenever those features are on. layout_threshold/layout_nms only change
    the odds, not the possibility.
  • The failure mode varies with the box: zero-area after truncation or
    inversion -> ValueError; fully-negative -> no error at all, just a crop
    of unrelated page pixels embedded as the block image and referenced as
    imgs/img_in_*.jpg in the Markdown (silently wrong document content).
  • Even valid fractional boxes are truncated inward at both ends, so the crop
    can drop up to one boundary pixel per edge relative to the reported
    block_bbox, and the imgs/img_in_*_box_*.jpg filename coordinates can
    disagree with the true box.

Notes

  • The sibling image path for the same page, gather_imgs()
    (paddlex/inference/pipelines/layout_parsing/utils.py, line 353), already
    clamps to the page and skips zero-area boxes via
    _crop_image_region_for_pil() (line 338); the block-crop path in
    get_layout_parsing_objects() has no equivalent handling, so the two paths
    can also disagree about the same physical region.
  • We searched for the error string and found no existing issue.
  • Environment: PaddleX 3.7.2 (official paddlex/hps GPU image, also verified
    against current develop source), PP-StructureV3 pipeline with table
    recognition enabled.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions