Skip to content

Django production image regressed to 2.4 GiB; size budget raised to 3 GiB as a stopgap (follows #1494) #2236

Description

@JSv4

Summary

The production Django/Celery image has grown back past the size bar set by #1494. The v3.0.0 release build measured it at 2,524,107,921 bytes (2.4 GiB) against the 1.5 GiB acceptance criterion — ~871 MiB over.

Because the Enforce Django image size budget step in .github/workflows/docker-build-release.yml runs before the push step, the gate failing meant no Django image was published for the v3.0.0 stable release while frontend, postgres, and traefik all published normally.

To unblock the release, DJANGO_IMAGE_BUDGET_BYTES was raised from 1.5 GiB to 3 GiB in 93dc9a1. That is a deliberate stopgap, not an acceptance of 2.4 GiB. This issue tracks getting the image back down and re-tightening the gate.

How this got past us

#1494 / #1500 cut the image from ~6.3 GB to under 1.5 GiB and added the budget gate at the same time (1236d5767, 2026-05-03). The gate has never actually run in a release build until now. Every release tag since the gate landed is either older than it (b2, b3, b4 all predate 2026-05-03) or died earlier in the job:

Release Django job Cause
v3.0.0.b1 (2025-08) ✅ success — (predates the gate)
v3.0.0.b2 (2025-10) ❌ failure No space left on device on the runner
v3.0.0.b3 (2025-12) ❌ failure pip wheel build failure (exit 123)
v3.0.0.b4 (2026-02) ❌ failure No space left on device on the runner
v3.0.0 (2026-08) ❌ failure size gate: 2.4 GiB > 1.5 GiB

So v3.0.0 is the first build to get all the way to the size check — the gate did its job on its first real opportunity. It also means no Django image has been published to ghcr since v3.0.0.b1, roughly a year.

Leading suspect

en_core_web_sm and en_core_web_lg spaCy model wheels are fetched into the wheel cache and installed into the runtime image. Both were added in 82f974870 (#1524) — the review follow-up to #1500, i.e. after the sub-1.5 GiB measurement was taken, and never re-measured in a release build. en_core_web_lg is several hundred MB on its own.

See compose/production/django/Dockerfile, the wget ... spacy-models step in the build stage.

Worth confirming before acting — the dependency delta since #1500 otherwise trends smaller (pandas, tiktoken, graphene-django, pytesseract, jsonschema, django-extensions all removed; only strawberry-graphql, graphql-relay, and pydantic-ai-slim added). Note also that the build installs every requirements/*/*.txt recursively, so optional-component requirements (analyzers/, ingestors/, postprocessors/) land in the production image too.

Reproduce / measure

docker build -f compose/production/django/Dockerfile . \
  --build-arg BUILD_ENVIRONMENT=production -t oc-django-size-probe
docker image inspect --format='{{.Size}}' oc-django-size-probe
docker history --no-trunc --format '{{.Size}}\t{{.CreatedBy}}' oc-django-size-probe | sort -h | tail -20

Acceptance criteria

  • Attribute the ~871 MiB regression to specific layers (docker history), not estimates
  • Decide whether en_core_web_lg belongs in the base production image or behind an optional requirements file / separate model-preloader image
  • Decide whether optional requirements/*/ component deps should ship in the default production image at all
  • Get the image back under a defensible target and lower DJANGO_IMAGE_BUDGET_BYTES to match, with headroom
  • Publish a Django image for the current release tag once it fits

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions