Repository navigation
feat(serve-logging): reproducible orca-serve logging & diagnostics pack - #39
Merged
Merged
Conversation
Port the headless `orca serve` logging/diagnostics pack from revive_labs: - logging architecture guide + troubleshooting matrix - reproducible templates (systemd unit, config/env layers, installer, client diagnostics) Link the guides from README next to the headless-linux-server guide.
…stream surface Rewrite the revive_labs port so it references only what upstream orca actually ships: real `orca serve` flags, the `orca_server_ready` ready contract, the `runtime.sock` local transport, real env vars (ORCA_ENVIRONMENT, ORCA_VERSION, ORCA_USER_DATA_PATH), and journald/logrotate logging. Drop fork-only tooling (orca-serve-fg/-diag/-version, OSV_*/ORCA_SERVE_* vars, /data/opt/revive, svc_orca).
…eyMurfin/orca into feature/serve-logging-setup
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Advanced Run ID: ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…y, no prod touch) + README test section
…or orca-serve-troubleshoot
…pansion) + missing-artifact test guard + root-relative test path
…sh-escape mangling)
…d docs links to README
…inting, and container test docs
Decouple stale Xvfb lock cleanup from socket existence in ensureVirtualDisplayForHeadlessServe.
Add isStaleDisplayLock to inspect /tmp/.X${display}-lock and probe PID liveness via process.kill(pid, 0).
Fixes Fork Issue #23 (PRB-0015).
Verified with unit tests and Docker Ubuntu 24.04 containerized test harness.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reproducible Orca Serve logging & diagnostics pack — one script (
install-logging-setup.sh) provisions the systemd instance unit, config layer, per-instance state tree, and log rotation + journal retention from templates. Plus theorca-serve-troubleshootskill (install + 4-bucket triage). It arranges the observability surface only — it does not download, build, or start the serve.In addition, this PR resolves Fork Issue #23 / PRB-0015: Xvfb stale display lock self-healing on headless Linux serve, fixing browser pane crashes caused by leftover
/tmp/.X99-lockfiles.Current State
orca servelogging/troubleshooting was hand-rolled per host; no repeatable installer, no shared triage procedure.apply()nevermkdir'dSYSTEMD_DIR; conf/env templates retained un-substituted@PREFIX@in comments;SERVICE_USER/SERVICE_GROUPdidn't flow into the unit's hardcodedUser=orca./tmp/.X99-lockwith the socket removed or dead.ensureVirtualDisplayForHeadlessServepreviously only purged stale artifacts when a unix socket was present, causing subsequent Xvfb rebinds on display:99to fail and crash headless serve browser panes with SIGSEGV (Fork Issue [Bug]: Headlessserveself-spawned Xvfb doesn't reap a stale/tmp/.X99-lockafter an unclean exit (reaper is socket-gated) → serve can't get its virtual display and crash-loops #23 / PRB-0015).Future State
--dry-run-first, idempotent installer reproduces the whole observability surface on any Linux/systemd host.SERVICE_USER/SERVICE_GROUPtemplating propagates to the unit.--verboseflag & cross-repo portability: Installer supports detailed preflight diagnostics and customizable source repository/branch URLs viaORCA_RAW_REPO_BASE/SKILL_RAW_URLto allow seamless installation across forks and upstream mirrors.tests/docker/Dockerfile.serve-logging-testandrun-test-in-container.sh: verifies sandbox installation, dry-run auditing, idempotency, logrotate syntax, and systemd-analyze verification.tests/docker/Dockerfile.xvfb-testandrun-xvfb-test.sh: verifies Xvfb stale display lock cleanup, orphan lock reap on dead PID / ESRCH, and self-healing headless serve startup under Ubuntu Linux container conditions.serveself-spawned Xvfb doesn't reap a stale/tmp/.X99-lockafter an unclean exit (reaper is socket-gated) → serve can't get its virtual display and crash-loops #23 / PRB-0015):isStaleDisplayLockdecouples stale lock detection from unix socket presence, inspecting/tmp/.X${display}-lockand testing PID liveness viaprocess.kill(pid, 0). Stale locks from dead/corrupt PIDs are cleanly unlinked prior to spawning Xvfb.Testing
Tested end-to-end across environments:
bash -n,--help,--dry-run, and--verbose— pass/tmp) — pass, no leftover@..@in any artifactsystemd-analyze verify— exit 0logrotate -d— exit 0keep (already present), exit 0orca-serve@factory— active,:6768LISTENtests/docker/(serve logging & Xvfb stale lock suites)vitest run src/main/startup/ensure-virtual-display.test.ts— 38/38 passingoxlint src/main/startup/ensure-virtual-display.ts— 0 errorstypecheck:node(tsc -p config/tsconfig.node.json) — clean (0 errors)Provenance
lesleyMurfin/orca— branchfeature/serve-logging-setupmtl-02-dev-001,Lesleys-PC(WSL2), Docker containerDocumentation
docs/reference/serve-logging/*(guide, matrix, templates README)skills/orca-serve-troubleshoot/SKILL.mdspecs/23-serve-xvfb-stale-lock.md(SSSF spec for Fork Issue [Bug]: Headlessserveself-spawned Xvfb doesn't reap a stale/tmp/.X99-lockafter an unclean exit (reaper is socket-gated) → serve can't get its virtual display and crash-loops #23 / PRB-0015)specs/serve-troubleshoot-skill-guide.mdUpstream Issue & PR Linkages
6d6af7af1a(Support browser panes on headless remote Orca servers stablyai/orca#5412) - socket-gated Xvfb reaper bugorca serve stats [--json]— live headless-serve runtime stats stablyai/orca#10608 & feat(cli): orca serve stats [--json] — live headless-serve runtime stats stablyai/orca#10612 (orca serve stats)Follow-ups
orcaservice account on hosts before starting the unit (installer soft-warns; full serve installer provisions it).systemd-analyze verifyflags the missingorcabinary — expected (installer does not provision it).systemctl restart systemd-journald.Refs: serve-logging standardization, Fork Issue #23, PRB-0015
Ticket: INFRA