Repository navigation
fix(operational): branch picker, container limits + kill reason, GitLab token refresh - #70
Merged
Merged
Conversation
…ab token Five operational bugs in one branch. GitLab branch pagination: - list_branches/list_tags walked only the first 20 results because the authenticated GitLab provider didn't paginate. Added per_page=100 page loop with a 1000-item safety cap (matches public_repo.rs). - BranchSelector also injects the currently-selected branch when the API doesn't return it (e.g., renamed/deleted upstream) so the dropdown never falls back to the empty placeholder. CPU/memory limits per project/environment: - WorkflowExecutionService never called .resources() on DeployImageJobBuilder, so configured limits were silently dropped and every container ran with the hardcoded 1000m/512Mi defaults. Now resolves env→project deployment_config and threads them through. - Even when set, deploy_image.rs parsed "1000m" with parse::<f64>() which always returned None. Replaced with parse_cpu_cores / parse_memory_mb that handle m/Mi/Gi/Ki/K/M/G/T. Unit tests added. Container kill reason + status: - New deployment_containers columns: exit_code, exit_reason, oom_killed, error_message, finished_at (migration m20260430_000001). - Deployer captures Docker's state.exit_code, oom_killed, error, finished_at on inspect; build_exit_reason() turns these into human-readable strings (OOMKilled, "Killed by SIGKILL (exit code 137)", "Exit code 1"). - Container health monitor persists the exit metadata to the row and picks ContainerCrash vs ContainerOomKilled alarm based on the OOMKilled flag. - ContainerInfoResponse and ContainerDetailResponse expose all five fields so the UI can render the kill reason. GitLab token refresh: - Bumped expiry buffer 60s -> 5min so a slow clone over flaky network no longer races the validate-then-use window. - Added force_refresh_connection_token bypassing expires_at, plus retry-on-auth wrappers around clone_repository and download_archive in GitProviderManager. On HTTP 401/403/"authentication failed", we force-refresh once, wipe partial clone artifacts, and retry. Test for is_auth_failure heuristic. Logs on redeploy: - Verified d6d1fcf already shipped the fix (deployment.id in useEffect deps, exponential backoff, dedup-by-line, state preservation). No additional change needed. Misc carried in this branch: - temps-providers: add base64 = workspace dep (mongodb.rs uses it for keyfile generation; main commit was missing the manifest entry). - temps-providers: mongodb replica_set support (single-node replica set for transactions / change streams) — this came in via merge conflict resolution and is being kept intentionally. - All entity literals updated for the new deployment_containers fields.
The container detail header showed bare current usage with no point of
reference: "CPU 0.0% · Mem 126 MB / 99%". A 99% memory number against
the *host's* total RAM (Docker's default "limit" when none is set)
looks like an emergency every time. Now we surface what's actually
configured and how long the current process has been running.
Backend:
- ContainerInfo gains started_at and cpu_limit_cores. Pulled from
container.host_config.nano_cpus and state.started_at on the same
inspect_container call we already issue.
- ContainerStats gains cpu_limit_cores, memory_limit_bytes (already
there), restart_count, and started_at — all the pieces the live SSE
stream needs to render a useful header without a second round-trip.
- /metrics/stream emits the new fields. Default impls added so test
doubles continue to compile.
Frontend (ContainerHeaderBar):
- Uptime is now derived from started_at, falling back to created_at.
After a restart it resets, which is the behavior users expect.
- "Restarted N×" chip when restart_count > 0. Amber for 1–2, red at
3+ so a crash loop is visible at a glance.
- "CPU x.x% / 1.0 cores" instead of bare percent. formatCores prints
millicores under one core ("500m") and cores above ("2 cores").
- "Mem 126 MB / 512 MB (24%)" instead of "Mem 126 MB / 99%". When
Docker reports the host's RAM as the limit (no real cap configured)
we hide the limit entirely — values above 100 GiB are treated as
unset so we don't show "Mem 126 MB / 128 GB".
- Replaced setState-in-useEffect uptime ticker with a tick-counter
driving useMemo, satisfying the react-hooks/set-state-in-effect rule
the project lints for.
The dev server's API proxy was targeting http://localhost:80, which matches an old setup before the temps backend moved to 8080. With the old target, every API request from `bun dev` either hit nothing or got served by whatever happened to be on port 80, surfacing as cryptic network errors during local frontend work.
DB stores CPU as microcores (1_000_000 = 1 core), matching legacy data
already in production. The form now displays/edits in cores ("1", "0.5",
"2") and converts at the boundary, eliminating the millicores UX trap
that produced impossible Docker CPU limits (e.g., NanoCpus 2_000_000_000_000).
Backend:
- parse_cpu_cores accepts a "u" suffix for microcores alongside the
existing "m" (millicores) and bare-cores forms
- workflow_execution_service formats stored i32 as "{}u" so Docker gets
the correct nano_cpus value
- ResourceUsage default switched to microcore strings for consistency
Frontend:
- EnvironmentConfigurationCard and GeneralSettings divide DB values by
1_000_000 on load and multiply by 1_000_000 on save
- Inputs are number/step="any"/min="0.01" with cores-friendly placeholders
Also fix pre-existing breakage encountered en route:
- docker.rs: replace manual range check with RangeInclusive::contains
- on_demand.rs test fixture: add new deployment_containers Model fields
Backwards compatible: API field types unchanged (i32), DB schema
unchanged, legacy rows render correctly without migration.
Drops the inline "Secret" toggle from each environment-variable row in ProjectConfigurator. Secret marking now lives in the dedicated secret flow, keeping the env vars form focused on key/value entry.
Builds on the previous container exit-reason work so the container
detail page is useful even after a container has stopped.
Backend:
- Migration m20260430_000001 extended with two more columns on
deployment_containers: started_at and cpu_limit_cores. The exit-info
columns alone weren't enough — we also want the running-state metadata
to survive after a stop, so the configuration tab can show "uptime"
and "CPU limit (observed)" without depending on the live SSE stream.
- ContainerHealthMonitor.persist_runtime_info writes started_at and
cpu_limit_cores onto the row whenever they change. Diff-checked, so
this is a no-op once the row is up to date.
- Eq dropped from deployment_containers::Model — f64 doesn't impl Eq.
PartialEq is enough for the diff checks the monitor does.
- ContainerInfoResponse gains started_at, restart_count, cpu_limit_cores
for the listing endpoint, so the header bar can render the restart
chip for stopped containers too (once SDK regenerated downstream).
- ContainerDetailResponse gains started_at + cpu_limit_cores. Detail
handler now resolves resource_limits from environment.deployment_config
with project fallback (was hardcoded to None), so the configuration
tab can show the configured CPU/memory caps the workflow applies.
Frontend:
- ContainerConfiguration grows two new sections:
Resource limits — CPU/memory request/limit from the deployment
config plus the observed CPU limit from Docker, which makes drift
visible if a container was started before a config change.
Exit details — exit reason, exit code, error, and finished_at,
visible only when the container is in a terminal state.
- Uptime in the configuration tab now reads started_at first, falling
back to created_at — matches the behavior we already wired into the
header bar so an in-place restart resets the counter.
The frontend uses two `as` casts on started_at and cpu_limit_cores
because the OpenAPI client wasn't regenerated for these specific fields
in this commit (the running dev server is still on the previous binary).
Once the server is restarted and bun run openapi-ts is run, the casts
can be dropped.
Web SDK regen included (140 lines) covers the multi-host network /
HA cluster types pulled in from main.
The runtime log viewer kept tailing the old container after a redeploy, producing endless "Failed to get container logs / Deployment not found" frames and the "Connection lost" banner. Three fixes, all in web/src/components/runtime-logs/log-viewer.tsx: 1. Per-environment redeploy signal. Watch the env's `current_deployment_id` (already exposed by getEnvironments) instead of project-wide getLastDeployment, so a redeploy in another environment can't bump us off live logs. When the id flips, invalidate the containers list and let the reconciliation effect atomically swap selectedContainer once the new container appears in the fresh list — never clear the selection mid-flight. 2. Reconciliation effect. If the selectedContainer is no longer in the container list (e.g. it was destroyed by a redeploy), fall back to the first available so the WebSocket stops trying to tail a dead container. 3. Stale-socket hardening. When reconnecting, detach all handlers on the previous socket before close() so its in-flight onmessage frames can't bleed into the new connection's log buffer. Every handler double-checks `ws === wsRef.current` before touching state. Removed the early-return on isConnectingRef that was silently swallowing reconnect attempts mid-handshake. Net effect: redeploy → env's current_deployment_id flips → containers refetch → reconciliation swaps selectedContainer → WS effect cleanly reconnects exactly once. Old socket's late frames are dropped.
…nges - web(sidebar): pin current project at top of default nav so users can return to the project sidebar in one click after pressing back - web(service-data-browser): minor adjustments - providers(mongodb): expand connection + parameter handling - migrations: add deployment_container runtime info migration; tweak existing exit info migration
GitLab webhook lifecycle (backend):
- Auto-install on project create + repo change via POST /projects/:id/hooks
- Auto-delete on repo change/disconnect (idempotent on 404)
- Per-project signing_token in whsec_{base64} format (32-byte key)
- Receiver at /api/webhook/git/gitlab/events validates Standard-Webhooks
HMAC-SHA256 over {webhook-id}.{timestamp}.{body}, multiple sigs supported
- Pre-flight access-level check (>= 40 Maintainer); install fails soft
- POST /projects/{id}/gitlab/reinstall-webhook for stale-hook recovery
- OAuth tokens use Authorization: Bearer; PAT/App use PRIVATE-TOKEN
- Schema: projects.gitlab_webhook_id, gitlab_webhook_signing_token (encrypted)
Git settings UI redesign:
- Inline-edit field rows replace global Edit Settings mode
- Click-to-edit branch / framework / directory / dockerfile / compose paths
- Repo card surfaces description, pushed_at, default branch, connection
- Auto-deploy now reads project.deployment_config.automaticDeploy correctly
- Webhook status row (GitLab only) with Reinstall button
- Change repository moved from cramped modal to dedicated page
Project sidebar restructure:
- Promote Environment Variables, Domains, Git, Logs, Environments to top-level
- Fold Error Tracking + Request Logs into Observability submenu
- Fold Revenue into Analytics submenu
- Merge Workspace + AI Workflows under AI
- Clean URLs: /domains and /git instead of /settings/domains, /settings/git
Misc:
- Notifications: exponential backoff for repeated alerts
- Proxy: simplify HTML-only logging branch
- Command palette: dedupe Request Logs, rename Runtime Logs to Logs
- log viewer: cap, batch, and gate WS by mode (Live/Pause/Interval) with per-project persistence; Pause closes the socket; bounded pending buffer drops oldest with a visible counter so high-volume streams can't bloat memory. - container logs (useLogStream): same cap + rAF batching to stop per-line setState rerenders on chatty containers. - command palette: disable cmdk's built-in filter so Fuse alone decides matches — fixes "Logs" disappearing from Cmd+K when typing "logs".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five operational bugs in one branch + a UX upgrade to the container detail header.
Fixes
1. GitLab branch picker hides selected branch on busy repos
gitlab_provider.list_branches/list_tagsreturned only the first 20 results because the authenticated path wasn't paginating. Added aper_page=100page loop with a 1000-item safety cap, matching the public-repo pattern.BranchSelectoralso injects the currently-selected branch into the dropdown when the API didn't return it (renamed/deleted upstream), so the trigger never falls back to the empty placeholder.2. CPU/memory limits not applied to new containers
Two breaks in series:
WorkflowExecutionServicenever called.resources()onDeployImageJobBuilder, so configuredcpu_limit/memory_limitfromenvironment.deployment_config(withproject.deployment_configfallback) were silently dropped. Every container ran with the hardcoded1000m / 512Midefaults.deploy_image.rsparsed"1000m"withparse::<f64>()which always returnedNone. Replaced withparse_cpu_cores/parse_memory_mbhelpers handlingm/Mi/Gi/Ki/K/M/G/T. Unit tests added.3. Container kill reason + status
deployment_containerscolumns:exit_code,exit_reason,oom_killed,error_message,finished_at(migrationm20260430_000001).state.exit_code,oom_killed,error,finished_aton inspect;build_exit_reason()turns these into human-readable strings (OOMKilled,Killed by SIGKILL (exit code 137),Exit code 1).ContainerCrashvsContainerOomKilledalarm based on the OOMKilled flag.ContainerInfoResponseandContainerDetailResponseexpose all five fields so the UI can render the kill reason.4. Logs not working on redeploy
Verified
d6d1fcfalready shipped the fix (deployment.idinuseEffectdeps, exponential backoff, dedup-by-line, state preservation). No additional change needed.5. GitLab access token sometimes invalid when downloading a repo
force_refresh_connection_tokenbypassingexpires_at, plus retry-on-auth wrappers aroundclone_repositoryanddownload_archiveinGitProviderManager. On HTTP 401/403/authentication failed, we force-refresh once, wipe partial clone artifacts, and retry. Test for theis_auth_failureheuristic.UX: container detail header
Before:
CPU 0.0% · Mem 126 MB / 99%— that 99% was against the host's total RAM (Docker's "limit" when none is configured), making every container look like it was about to OOM.Now:
CPU 0.0% / 1.0 cores— shows the configured CPU limit alongside current usage.Mem 126 MB / 512 MB (24%)— shows the configured memory limit. When Docker reports the host's RAM as the limit (no real cap configured), the limit is hidden entirely so we don't showMem 126 MB / 128 GB.started_at(process start), notcreated_at— after an in-place restart it resets, which is the behavior users expect.Restarted N×chip whenrestart_count > 0. Amber for 1–2, red for 3+ so a crash loop is visible at a glance.ContainerStatsand/metrics/streamwere extended withcpu_limit_cores,restart_count,started_at, and the existingmemory_limit_bytesso the live SSE stream carries everything the header needs without a second round-trip.Plus
chore(web): dev proxy was pointed at:80, fixed to matchtemps serveon:8080.temps-providers:base64workspace dep added (was missing, brokemongodb::generate_keyfile_content).temps-providers/mongodb: optional replica-set support carried in via merge — single-node replica set for transactions / change streams.Test plan
cargo check --libclean across the workspacecargo test --lib -p temps-deployments parse_— 13 passed (incl. new parser tests)cargo test --lib -p temps-git is_auth_failure— passedbunx eslinton touched frontend files — 0 errors, 0 warningsdocker inspectexit_reasonlands on the row + UI