Enable PostgreSQL as the default database for rhdh-local

[NiallTwomey2]

Description

Enable PostgreSQL as the default database for RHDH Local (RHIDP-16882 / RHIDP-16883).

• db is in compose.yaml, image ${POSTGRES_IMAGE:-quay.io/fedora/postgresql-18:latest}
• Anonymous volume at /var/lib/pgsql/data
• compose stop/start/restart keep data;
• compose down then compose up starts a new empty volume (RHDH does not support DB downgrades)
• rhdh depends_on: db (healthy)
• Default app-config.yaml uses client: pg and POSTGRES_* from default.env
• Removed WITH_POSTGRES,app-config.db.yaml,compose-with-db.yaml
• wait-for-plugins-and-start.sh waits for $POSTGRES_HOST:$POSTGRES_PORT before starting node
• Corporate-proxy overlay sets NO_PROXY=localhost,127.0.0.1
• SQLite remains an optional app-config.local.yaml override
• RHEL Postgres is opt-in: POSTGRES_IMAGE=registry.redhat.io/rhel10/postgresql-18:latest in .env plus registry login

Which issue(s) does this PR fix or relate to

• Relates to RHIDP-16882

• Relates to RHIDP-16883

• Follow up issue RHIDP-16884 - CI check that the effective RHDH config is client: pg (Postgres, not SQLite) after the stack is healthy.

PR acceptance criteria

• Tests updated and passing
• Documentation updated
• Built-in TechDocs updated if needed

How to test changes / Special notes to the reviewer

Postgres uses an anonymous volume at /var/lib/pgsql/data.

Command	Catalog / plugin DB data
podman compose stop then start (or restart)	Kept — same containers, same volume
podman compose down then podman compose up -d	Wiped — PostgresSQL catalog is wiped and replaced with a new empty volume (use this when switching RHDH versions) Named volumes such as plugins and RAG are kept.
podman compose down --volumes then podman compose up -d	PostgreSQL is Wiped, and named volumes (plugins, RAG) are deleted too. A full reset

# Fresh start
podman compose down --volumes --remove-orphans
podman compose up -d
# http://localhost:7007 — rhdh logs: waited for Postgres at db:5432

podman exec -e PGPASSWORD=postgres db \
  psql -U postgres -d postgres -c '\l'
# expect backstage_plugin_catalog and other backstage_plugin_* DBs

# Keep data
podman compose stop
podman compose start

# Wipe Postgres only (new anonymous volume)
podman compose down
podman compose up -d

# Optional proxy:
CORPORATE_PROXY_IMAGE=docker.io/ubuntu/squid:latest \
  podman compose -f compose.yaml -f compose-with-corporate-proxy.yaml up -d
# Squid access.log should have no CONNECT to db:5432

[rm3l]

@NiallTwomey2 Looks like CI is failing on this PR. Could you please take a look?

[NiallTwomey2]

@rm3l The CI has passed. Podman does not cause the CI checks to fail anymore.
CI checks could not identify any database inside nested podman containers and exits 125.
Now, after compose exec, the CI checks will use the same path as podman compose up, the database is now acknowledged by the checks - backstage plugin databases are identifiable now.

[Fortune-Ndlovu]

Thanks for the RHIDP-16882 work. The overall approach looks good: Postgres in the default compose file, pg client in app-config, startup wait, and NO_PROXY for db behind the corporate proxy overlay.

Before merge, please align the PR description with the code, add (or explicitly defer) the CI check for plugin databases, and pin the default Postgres image by digest instead of :latest.

Details are on the inline comments.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[benwilcock]

We never wanted PostgreSQL to be on by default so we could keep the resource overheads low. Why do we need this now?

[NiallTwomey2]

@benwilcock In memory SQLite is definitely lower cost, and resource overheads are lower than of a PostgreSQL container, also due the fact that PostgreSQL adds an additional db container to rhdh-local, which contributes to resource overheads being higher.
We are looking into enabling PostgreSQL as the default database for rhdh-local, to address the loss of catalog and plugin data when rhdh-local restarts - podman compose stop/start/restart, which is currently unavailable when memory for rhdh-local is configured with better-sqlite3.

[rm3l]

We never wanted PostgreSQL to be on by default so we could keep the resource overheads low. Why do we need this now?

@benwilcock After some issues affecting only the SQLite backend in the core RHDH image (e.g.: RHDHBUGS-3753), we realized that we actually only support RHDH against PostgreSQL in all our other install methods. So we thought it might make sense to stay consistent here. Otherwise, that would mean that:

• SQLite-specific issues burn time investigating and fixing a code path no supported install method runs.
• the dev-to-prod path might have a hidden gap. Users prototype/develop in rhdh-local with SQLite, then eventually move to Helm or the Operator on Postgres. Behavior differences between the two backends might surface only after promotion.

On resource overhead, the Postgres container uses ~170-190 MB of memory and CPU stays around ~1%. This seems negligible next to the default RHDH container (~413 MB mem / ~7% CPU) and lightspeed-core one (~337 MB mem / ~5% CPU). See the podman stats with Postgres running for some time:

ID            NAME             CPU %       MEM USAGE / LIMIT  MEM %       NET IO             BLOCK IO           PIDS        CPU TIME    AVG CPU %
809697e3f067  db               1.14%       188MB / 66.82GB    0.28%       14.68MB / 8.213MB  83.96MB / 783.7MB  49          8.281529s   1.14%
5288ce5bafec  rhdh             3.56%       412.7MB / 66.82GB  0.62%       10.04MB / 14.77MB  169.6MB / 14.39MB  12          23.989255s  6.56%
54112098b5a3  lightspeed-core  1.50%       337.1MB / 66.82GB  0.50%       10.04MB / 14.77MB  187.1MB / 7.094MB  2           10.065334s  5.50%

We thought 2.1 might be a good opportunity for this change. And that being said, if needed, users can always switch back to SQLite if needed by updating their app-config.local.yaml/compose.override.yaml accordingly.