Skip to content

Support external PostgreSQL for NetBox and Keycloak - #1290

Open
sylvesterkaczmarek wants to merge 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:feat/632-external-postgres-20261106
Open

sylvesterkaczmarek wants to merge 1 commit into
cisagov:contributor-prsfrom
sylvesterkaczmarek:feat/632-external-postgres-20261106

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown
Contributor

Summary

Make the existing external PostgreSQL configuration work consistently for NetBox and embedded Keycloak. The application connection settings already accept an external host, but Keycloak readiness queries the maintenance database, NetBox backups run in the embedded PostgreSQL container, and restore SQL defaults to the username's database rather than the configured database.

Closes #632.

Changes

  • Check Keycloak's application database directly, so normal startup works with a role denied access to the postgres maintenance database. Preserve JDBC URL properties for verified TLS and bracket IPv6 hosts.
  • Run NetBox dumps inside the NetBox container using its endpoint, credentials and TLS settings. Support both Compose and the existing Kubernetes backup path, keep credentials out of command arguments, and preserve a requested backup filename.
  • Honor PGSSLMODE in NetBox unless DB_SSLMODE explicitly overrides it. Apply that same override during backup and restore.
  • Target the configured database explicitly in all restore SQL calls and enable ON_ERROR_STOP.
  • Document provisioning, separate application roles, libpq/JDBC TLS configuration, migration, credential handling and restore privileges. Existing embedded-server defaults remain unchanged.

Validation

  • 19 focused regression cases passed. Focused and adjacent NetBox/helper coverage: 254 tests and 7 subtests passed, with no skips.
  • Against the unchanged base, the 14 cases covering pre-existing code paths report 9 failures and 5 passes. The five tests for the new backup helper were excluded from that comparison because the helper does not exist on the base.
  • Built the complete Keycloak Dockerfile on Linux ARM64 and exercised Keycloak 26.8.2 against a disposable PostgreSQL 17.11 server on a non-default port with separate application databases and roles.
  • Verified verify-full TLS, rejection of an incorrect hostname and untrusted CA, actual NetBox configuration connectivity, real pg_dump, and a real restore through netbox_init.execute_restore_commands using a custom database name.
  • Keycloak startup and schema migrations succeeded while its application role was explicitly denied maintenance-database access.
  • Restore privileges were granted separately for the restore test. Restored rows and the post-restore user-table cleanup were checked in the intended database.
  • Python syntax, focused Ruff checks, Black on new tests and the modified initializer, Bash syntax, ShellCheck and whitespace checks passed.
  • These changes and the user-agent normalization change merge cleanly with the 29 existing active topic PRs, excluding reference-only Preserve saved installer configuration during bulk environment import #1222. The same focused/adjacent test selection passes on that combined checkout.
python -m pytest -q   shared/tests/test_external_postgresql.py   shared/tests/test_netbox_device_type_import_cache.py   shared/tests/test_netbox_plugin_discovery.py   shared/tests/test_malcolm_utils_helpers.py

Limits

Configuration uses the existing environment files; this does not add installer menus or provision an external database automatically. The full Malcolm stack and full NetBox web service were not started. NetBox verification loaded its actual configuration and connected with psycopg. No managed cloud database provider was tested.

The built-in NetBox restore drops and recreates its selected database and still requires ownership, CREATEDB and maintenance-database access. Restricted managed-service roles require an administrator/provider restore instead. Kubernetes command routing is unit-tested; a Kubernetes cluster was not provisioned. Existing limitations on Kubernetes media backup remain unchanged.

The temporary database, Keycloak container, isolated network, certificate volume and test credentials were removed after validation.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants