Skip to content

feat(settings): hot-reload the settings directory at boot and runtime - #508

Open
taitelee wants to merge 10 commits into
mainfrom
settings-reload
Open

feat(settings): hot-reload the settings directory at boot and runtime#508
taitelee wants to merge 10 commits into
mainfrom
settings-reload

Conversation

@taitelee

@taitelee taitelee commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

The server now consumes the settings directory #500 only validated. settings.dir / WH_SETTINGS_DIR is required: boot validates and adopts it (missing or invalid refuses to start, same contract as policy.file_path), and a running instance re-validates and re-adopts on a directory watch (fsnotify with debounce; survives a delete-and-recreate), SIGHUP, or admin-gated POST /v1/ops/settings/reload, all through one serialized reload path. A rejected reload, a deleted file, or a vanished directory keeps the previous good snapshot.

  • Tenant tunables move out of boot config into config.json: dedupe.id_field / dedupe.require_id (with per-table overrides, Per-table dedupe id_field (+ fix cross-table dedupe keyspace collision) #222), query.default_max_rows, schema.refresh_interval, cors.allowed_origins. The YAML/env keys for these are removed; dedupe.enabled stays boot config since it owns Pebble's lifecycle.
  • Every config.json key is required and the binary carries no compiled defaults, so the adopted snapshot is what the files say. Defaults live in one checked-in seed (internal/settings/seed/, go:embedded), shipped three ways: the new wavehouse init-settings <dir> writes it (refusing a non-empty directory), both container images bake it at /app/settings with WH_SETTINGS_DIR preset, and the dev config.yaml and e2e fixture point at it. The seed ships no policy (fail-closed, warned).
  • Consumers take functions instead of values (IngestHandler.DedupeSettings, the structured-query defaultMaxRows getter, corsMiddleware origins getter, SchemaRegistry.SetIntervalSource) so a reload applies without a restart and internal/api stays testable without a directory.

Known gap: only config.json changes affect a running server. roles.json, policies.json, and pipes.json are validated and loaded into the snapshot, but editing them does nothing yet — access control still comes from policy.file_path + NATS KV, and pipes from the /v1/ops/pipes API + NATS KV. The next PR makes the snapshot the runtime authority for all three and deletes the PUT /v1/ops/policy and PUT/DELETE /v1/ops/pipes/{name} endpoints and the NATS KV stores behind them: without KV there is no bundled node sync, so a node-local write path would let distributed nodes drift. Files (edited directly on standalone, written by the control plane in cloud) become the only write path.

Related Issues

Advances #48 (reload wiring; reverses the #48-era runtime-settings direction) and #222 (per-table id_field; the keyspace fix stays open there). Follow-up surfaced in review: #510 (distroless digest pin).

@taitelee
taitelee requested review from a team and EricAndrechek August 21, 2026 01:35
@github-actions github-actions Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file go Pull requests that update go code area/api HTTP handlers, routing, middleware area/query Structured query AST, SQL builder area/docs Documentation, site/, README area/infra CI, build, deploy, Docker, release labels Aug 21, 2026
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added required settings-directory initialization through wavehouse init-settings.
    • Added hot reload via file changes, SIGHUP, and the admin reload endpoint.
    • Added runtime configuration for CORS, query limits, schema refresh, and deduplication.
    • Added per-table deduplication overrides and safer deduplication lifecycle handling.
  • Documentation

    • Updated setup, deployment, configuration, API, and settings-directory guidance.
    • Documented validation, reload behavior, defaults, and migration from previous configuration options.
  • Bug Fixes

    • Invalid reloads now preserve the last valid settings snapshot.

Walkthrough

WaveHouse moves tenant tunables into a required JSON settings directory. The application validates and hot-reloads settings from file changes, SIGHUP, or an admin endpoint. Deduplication, query limits, schema refresh, and CORS now use live settings.

Changes

Settings contract and reload store

Layer / File(s) Summary
Settings validation, seeds, storage, and watching
internal/settings/*, go.mod
Required JSON blocks and fields are validated. Seed files can initialize a directory. The store adopts valid snapshots atomically, retains the previous snapshot after rejection, exposes typed accessors, and watches filesystem changes with debouncing.
Settings validation coverage
internal/settings/*_test.go
Tests cover seed creation, validation, reload adoption and rejection, snapshot retention, typed accessors, callbacks, and directory recreation.

Boot, command, and deployment wiring

Layer / File(s) Summary
Startup and initialization command
cmd/wavehouse/*, internal/config/*
Startup requires a valid settings directory. The init-settings command writes default files and returns distinct usage and operational exit codes. Boot configuration removes tenant-owned settings.
Container and local configuration
config.yaml, deployments/*, Makefile
Container images and Compose provide /app/settings. Configuration comments describe initialization and reload behavior. Local template updates now warn when existing files are stale.

Runtime consumers and reload endpoint

Layer / File(s) Summary
Managed deduplication lifecycle
internal/dedupe/*, internal/api/ingest.go
dedupe.Managed opens and closes Pebble storage as dedupe.enabled changes. Ingest resolves per-table settings and handles disabled or unavailable deduplication states.
Dynamic API and query consumers
internal/api/*, internal/discovery/discovery.go
CORS origins, query row limits, and schema refresh intervals are evaluated from live settings. The admin reload endpoint reports adoption and validation findings. Tests cover dynamic CORS and reload behavior.

Documentation and fixtures

Layer / File(s) Summary
Settings directory documentation
docs/src/content/docs/settings-directory.mdx, docs/src/config/sidebar.ts, docs/src/content/docs/index.mdx
The new documentation describes required files, initialization, validation, reload triggers, configuration keys, ownership, CORS, and deduplication.
Configuration and deployment references
docs/src/content/docs/*, README.md, AGENTS.md, CHANGELOG.md
Documentation and project notes replace obsolete YAML and environment settings with settings-directory references.
End-to-end fixtures and coverage configuration
tests/e2e/fixtures/*, .testcoverage.yml
End-to-end fixtures provide the required settings files and tenant configuration. Coverage exclusions document validation-only paths.

Estimated code review effort: 4 (Complex) | ~60 minutes

Merge Risk: 🟠 High · up to 50c63

This PR makes the settings directory authoritative at boot and during runtime reloads, but the current head can still silently discard valid records, miss settings updates after filesystem event overflow, and misconfigure the container settings path; its floating runtime image also weakens deployment reproducibility. Merge should be blocked until these concrete data, runtime, and deployment risks are fixed or explicitly accepted.

Sequence Diagram(s)

sequenceDiagram
  participant Operator
  participant WaveHouse
  participant SettingsStore
  participant DedupeManaged
  participant APIConsumers
  Operator->>WaveHouse: Edit settings, send SIGHUP, or call reload endpoint
  WaveHouse->>SettingsStore: TriggerReload(trigger)
  SettingsStore->>SettingsStore: Validate and adopt valid snapshot
  SettingsStore->>DedupeManaged: Apply dedupe.enabled
  SettingsStore-->>APIConsumers: Updated query, schema, and CORS accessors
  WaveHouse-->>Operator: Adoption status and findings
Loading

Suggested reviewers: ericandrechek

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 90 functions across 28 files. (9 skipped:… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description accurately covers the required settings directory, boot validation, hot reload triggers, tenant configuration migration, seed behavior, dynamic consumers, and known follow-up scope.
Title check ✅ Passed The title clearly identifies the main change: hot-reloading the settings directory during boot and runtime.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 90 functions across 28 files. (9 skipped: 9 unsupported.)

✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch settings-reload
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch settings-reload
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch settings-reload

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@taitelee taitelee moved this from Backlog to In progress in WaveHouse Task Board Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown

📚 Docs preview is livehttps://14883740-wavehouse-docs.wave-rf.workers.dev

  • Commit5646dab: fix(settings): re-watch a recreated settings directory and add CORS reload test
  • Author@taitelee
  • Committed — 2026-08-21 12:01 (UTC-04:00)
  • Deployed — 2026-08-21 12:14 EDT

@github-code-quality

github-code-quality Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: Go

Go

The overall line coverage in commit 5646dab in the settings-reload branch is 90%. The line coverage in commit 9ebd10c in the main branch is 91%.

Show a line coverage summary of the most impacted files.
File main 9ebd10c settings-reload 5646dab +/-
internal/discov...ry/discovery.go 99% 97% -2%
cmd/wavehouse/main.go 65% 64% -1%
internal/settings/validate.go 96% 96% 0%
internal/config/config.go 94% 94% 0%
internal/api/ingest.go 97% 97% 0%
internal/settings/watch.go 0% 67% +67%
internal/settings/seed.go 0% 69% +69%
internal/settings/store.go 0% 82% +82%
internal/api/settings.go 0% 88% +88%
cmd/wavehouse/init_settings.go 0% 97% +97%

Updated August 21, 2026 16:14 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 8


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: e472820d-ea24-4cec-91dd-ebe59cdf26b7

📥 Commits

Reviewing files that changed from the base of the PR and between 9ebd10c and 9ad2d00.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (50)
  • .goreleaser.yaml
  • .testcoverage.yml
  • AGENTS.md
  • CHANGELOG.md
  • cmd/wavehouse/init_settings.go
  • cmd/wavehouse/init_settings_test.go
  • cmd/wavehouse/main.go
  • cmd/wavehouse/validate_test.go
  • config.yaml
  • deployments/Dockerfile
  • deployments/Dockerfile.goreleaser
  • deployments/compose/standalone.yaml
  • docs/src/content/docs/api.md
  • docs/src/content/docs/configuration.mdx
  • docs/src/content/docs/deployment.md
  • docs/src/content/docs/development.md
  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/reverse-proxy.mdx
  • docs/src/content/docs/sdk/index.mdx
  • go.mod
  • internal/api/ingest.go
  • internal/api/ingest_test.go
  • internal/api/router.go
  • internal/api/router_test.go
  • internal/api/settings.go
  • internal/api/settings_test.go
  • internal/api/structured_query.go
  • internal/api/structured_query_test.go
  • internal/config/config.go
  • internal/config/config_test.go
  • internal/discovery/discovery.go
  • internal/settings/finding.go
  • internal/settings/seed.go
  • internal/settings/seed/config.json
  • internal/settings/seed/pipes.json
  • internal/settings/seed/policies.json
  • internal/settings/seed/roles.json
  • internal/settings/settings.go
  • internal/settings/store.go
  • internal/settings/store_test.go
  • internal/settings/validate.go
  • internal/settings/validate_test.go
  • internal/settings/watch.go
  • internal/settings/watch_test.go
  • tests/e2e/fixtures/config.yaml
  • tests/e2e/fixtures/settings/config.json
  • tests/e2e/fixtures/settings/pipes.json
  • tests/e2e/fixtures/settings/policies.json
  • tests/e2e/fixtures/settings/roles.json
  • tests/integration/query_limits_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout. (2)
  • GitHub Check: E2E tests
  • GitHub Check: Coverage
🧰 Additional context used
📓 Path-based instructions (9)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: - No global state: Dependencies are passed explicitly (constructor injection).

  • Go 1.26, strict formatting (gofumpt, enforced by CI)

Files:

  • tests/integration/query_limits_test.go
  • cmd/wavehouse/validate_test.go
  • internal/settings/finding.go
  • cmd/wavehouse/init_settings.go
  • internal/api/settings_test.go
  • internal/settings/watch_test.go
  • internal/api/structured_query_test.go
  • internal/settings/seed.go
  • internal/api/structured_query.go
  • cmd/wavehouse/init_settings_test.go
  • internal/api/ingest.go
  • internal/settings/watch.go
  • internal/api/settings.go
  • internal/settings/settings.go
  • internal/settings/validate.go
  • internal/discovery/discovery.go
  • internal/settings/validate_test.go
  • internal/api/router.go
  • internal/api/ingest_test.go
  • internal/settings/store_test.go
  • internal/config/config.go
  • internal/settings/store.go
  • internal/api/router_test.go
  • internal/config/config_test.go
  • cmd/wavehouse/main.go
tests/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

  • Table-driven tests: Use tests := []struct{ name string; ... } with t.Run(tt.name, ...) for test cases.

Files:

  • tests/integration/query_limits_test.go
**/*_test.go

📄 CodeRabbit inference engine (AGENTS.md)

  • Every new function should have corresponding test cases. Run make lint and make test before considering work complete.

Files:

  • tests/integration/query_limits_test.go
  • cmd/wavehouse/validate_test.go
  • internal/api/settings_test.go
  • internal/settings/watch_test.go
  • internal/api/structured_query_test.go
  • cmd/wavehouse/init_settings_test.go
  • internal/settings/validate_test.go
  • internal/api/ingest_test.go
  • internal/settings/store_test.go
  • internal/api/router_test.go
  • internal/config/config_test.go
internal/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

  • Package naming: Lowercase, single word (or abbreviated). internal/ enforces module privacy.

Files:

  • internal/settings/finding.go
  • internal/api/settings_test.go
  • internal/settings/watch_test.go
  • internal/api/structured_query_test.go
  • internal/settings/seed.go
  • internal/api/structured_query.go
  • internal/api/ingest.go
  • internal/settings/watch.go
  • internal/api/settings.go
  • internal/settings/settings.go
  • internal/settings/validate.go
  • internal/discovery/discovery.go
  • internal/settings/validate_test.go
  • internal/api/router.go
  • internal/api/ingest_test.go
  • internal/settings/store_test.go
  • internal/config/config.go
  • internal/settings/store.go
  • internal/api/router_test.go
  • internal/config/config_test.go
internal/api/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

  1. Bearer-token-only CORS posture (security) — Bearer JWT on every request, no cookies/sessions; corsMiddleware deliberately never emits Access-Control-Allow-Credentials (not needed, and * + credentials is a spec violation browsers reject).

Files:

  • internal/api/settings_test.go
  • internal/api/structured_query_test.go
  • internal/api/structured_query.go
  • internal/api/ingest.go
  • internal/api/settings.go
  • internal/api/router.go
  • internal/api/ingest_test.go
  • internal/api/router_test.go
**/*.mdx

📄 CodeRabbit inference engine (AGENTS.md)

  • In MDX, leave a blank line between a JSX tag and a code fence.

Files:

  • docs/src/content/docs/sdk/index.mdx
  • docs/src/content/docs/reverse-proxy.mdx
  • docs/src/content/docs/configuration.mdx
docs/src/content/**/*.mdx

📄 CodeRabbit inference engine (AGENTS.md)

docs/src/content/**/*.mdx: - Opt a page into the Cloud CTA with cloudCta frontmatter, not by importing the component.

  • Never hand-write ® or in prose.
  • Never hand-write utm_* params or rel on a link to wavehouse.cloud or wave-rf.com. Use cloudLink() / relFor() from docs/src/config/outbound.ts.

Files:

  • docs/src/content/docs/sdk/index.mdx
  • docs/src/content/docs/reverse-proxy.mdx
  • docs/src/content/docs/configuration.mdx
**/*.md

📄 CodeRabbit inference engine (AGENTS.md)

  • Never hard-wrap prose. One paragraph is one line. No wrapping at 72/80 columns, no "semantic linefeeds" splitting a paragraph at sentence boundaries.

Files:

  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/development.md
  • AGENTS.md
  • docs/src/content/docs/api.md
  • CHANGELOG.md
  • docs/src/content/docs/deployment.md
**/*.{yml,yaml}

📄 CodeRabbit inference engine (AGENTS.md)

New workflows must follow the same pattern — never @main or floating tags on third-party actions.

Files:

  • deployments/compose/standalone.yaml
  • tests/e2e/fixtures/config.yaml
  • config.yaml
🧠 Learnings (2)
📚 Learning: 2026-06-26T12:23:22.696Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 346
File: internal/stream/subscriber_test.go:9-28
Timestamp: 2026-06-26T12:23:22.696Z
Learning: In this Go repository, prefer table-driven tests (e.g., `[]struct{...}` with `t.Run(...)`) only for tests that cover multiple scenarios/inputs and can be cleanly enumerated. Do not artificially rewrite a clear single-scenario sequential behavioral-flow test into a table-driven form just to fit the pattern; if there’s only one meaningful scenario, keep the test as a straightforward linear flow (as in `TestSubscriber_SendDeliversThenDropsWhenFull`).

Applied to files:

  • internal/settings/validate_test.go
  • internal/settings/store_test.go
📚 Learning: 2026-08-13T12:17:52.620Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 470
File: docs/src/content/docs/reverse-proxy.mdx:137-144
Timestamp: 2026-08-13T12:17:52.620Z
Learning: For Wave-RF/WaveHouse documentation, verify claims about implementation control flow against the authoritative implementation source (for example, internal/auth/auth.go) rather than relying solely on docs/** content. Documentation may lag behind or paraphrase behavior, so control-flow claims should be confirmed in source code.

Applied to files:

  • docs/src/content/docs/configuration.mdx
🪛 Checkov (3.3.10)
deployments/Dockerfile

[low] 42-42: Ensure the base image uses a non latest version tag

(CKV_DOCKER_7)

🪛 Hadolint (2.15.1)
deployments/Dockerfile

[warning] 42-42: Always tag the version of an image explicitly

(DL3006)


[info] 45-45: Non-numeric user-id may not be resolvable by host system

(DL3066)

🪛 LanguageTool
CHANGELOG.md

[style] ~13-~13: Consider an alternative for the overused word “exactly”.
Context: ...issing one), so the adopted snapshot is exactly what the files say, and once adopted it...

(EXACTLY_PRECISELY)


[style] ~13-~13: Since ownership is already implied, this phrasing may be redundant.
Context: ...et (bare docker run boots; bind-mount your own directory over it), and the dev `config...

(PRP_OWN)

docs/src/content/docs/configuration.mdx

[style] ~188-~188: Consider an alternative for the overused word “exactly”.
Context: ...hey live, and what the server adopts is exactly what the files say. The container image...

(EXACTLY_PRECISELY)


[style] ~188-~188: Since ownership is already implied, this phrasing may be redundant.
Context: ...o a bare docker run boots; bind-mount your own directory over that path to edit. Chec...

(PRP_OWN)


[style] ~204-~204: Consider using the typographical ellipsis character here instead.
Context: ...ttings/reload** (admin-only) — returns {"adopted": bool, "findings": [...]}; 200when adopted,422` when reje...

(ELLIPSIS)

🪛 Trivy (0.73.0)
deployments/Dockerfile

[warning] 42-42: ':latest' tag used

Specify a tag in the 'FROM' statement for image 'gcr.io/distroless/static-debian12'

Rule: DS-0001

Learn more

(IaC/Dockerfile)

🔇 Additional comments (41)
AGENTS.md (2)

61-62: LGTM!

Also applies to: 65-65


57-57: 🗄️ Data Integrity & Integration | 🏗️ Heavy lift

Make the Issue #222 table-isolation contract explicit and executable.

The changes mention per-table id_field selection but do not represent the required table-qualified deduplication keyspace in the invariant or the e2e settings fixture.

  • AGENTS.md#L57-L57: state that deduplication keys include table identity and that writes and lookups use the same namespace.
  • tests/e2e/fixtures/settings/config.json#L1-L16: add table overrides and test equal IDs in different tables, or link an existing test that proves both behaviors.
.testcoverage.yml (1)

75-86: LGTM!

config.yaml (1)

53-55: LGTM!

Also applies to: 85-99

tests/e2e/fixtures/config.yaml (1)

8-14: LGTM!

Also applies to: 31-36

tests/e2e/fixtures/settings/pipes.json (1)

1-1: LGTM!

tests/e2e/fixtures/settings/policies.json (1)

1-3: LGTM!

tests/e2e/fixtures/settings/roles.json (1)

1-3: LGTM!

docs/src/content/docs/api.md (1)

597-597: LGTM!

Also applies to: 799-813

docs/src/content/docs/configuration.mdx (1)

26-27: LGTM!

Also applies to: 42-43, 70-70, 116-116, 131-137, 188-236, 303-304, 328-331, 392-392

docs/src/content/docs/deployment.md (1)

62-62: LGTM!

Also applies to: 157-171, 369-369

docs/src/content/docs/development.md (1)

147-147: LGTM!

Also applies to: 231-238

docs/src/content/docs/getting-started.md (1)

98-98: LGTM!

Also applies to: 116-116

docs/src/content/docs/reverse-proxy.mdx (1)

25-25: LGTM!

Also applies to: 199-199

docs/src/content/docs/sdk/index.mdx (1)

404-404: LGTM!

go.mod (1)

23-23: LGTM!

internal/settings/finding.go (1)

18-24: LGTM!

internal/settings/store_test.go (1)

13-157: LGTM!

cmd/wavehouse/validate_test.go (1)

22-22: LGTM!

.goreleaser.yaml (1)

85-88: LGTM!

internal/api/settings.go (1)

11-51: LGTM!

internal/api/structured_query.go (1)

30-35: LGTM!

Also applies to: 52-52, 122-126

internal/settings/seed.go (1)

1-61: LGTM!

internal/settings/seed/config.json (1)

1-16: LGTM!

internal/settings/seed/pipes.json (1)

1-4: LGTM!

internal/settings/seed/policies.json (1)

1-2: LGTM!

internal/settings/seed/roles.json (1)

1-4: LGTM!

internal/settings/validate_test.go (1)

4-4: LGTM!

Also applies to: 30-64, 96-96, 264-271, 335-335, 358-368, 378-378

tests/integration/query_limits_test.go (1)

102-102: LGTM!

internal/settings/settings.go (1)

63-84: LGTM!

Also applies to: 98-118

internal/settings/validate.go (1)

286-309: LGTM!

Also applies to: 338-362

internal/settings/store.go (1)

10-131: LGTM!

internal/config/config.go (1)

13-32: LGTM!

Also applies to: 45-52, 108-112, 142-147, 222-224

internal/config/config_test.go (1)

14-34: LGTM!

Also applies to: 101-107, 189-190, 203-216, 227-228, 239-241, 263-264, 282-283, 302-303, 319-320, 344-345, 372-373, 391-392, 427-428, 445-446, 458-459, 472-473, 487-488

cmd/wavehouse/init_settings.go (1)

14-50: LGTM!

cmd/wavehouse/main.go (1)

31-31: LGTM!

Also applies to: 102-116, 136-137, 177-188, 295-331, 473-473, 531-537

cmd/wavehouse/init_settings_test.go (1)

12-42: LGTM!

internal/api/ingest.go (1)

39-45: LGTM!

internal/api/router.go (1)

20-43: LGTM!

Also applies to: 196-198, 299-341

internal/api/structured_query_test.go (1)

32-43: LGTM!

Also applies to: 285-298

internal/discovery/discovery.go (1)

49-59: LGTM!

Also applies to: 217-255

Comment thread CHANGELOG.md Outdated
Comment thread deployments/compose/standalone.yaml Outdated
Comment thread deployments/Dockerfile
Comment thread internal/api/ingest_test.go Outdated
Comment thread internal/api/ingest.go Outdated
Comment thread internal/api/router_test.go
Comment thread internal/settings/validate.go
Comment thread internal/settings/watch.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/settings/watch.go (1)

80-86: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Reconcile settings after a watcher queue overflow.

fsnotify.ErrEventOverflow indicates that filesystem notifications were lost. When this error occurs, reset the debounce timer so its normal path calls s.TriggerReload("watch"). That reload reparses the complete settings directory.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b2f6f5c1-0bde-4b64-8caa-cd85c395c4cc

📥 Commits

Reviewing files that changed from the base of the PR and between 9ad2d00 and 5646dab.

📒 Files selected for processing (6)
  • CHANGELOG.md
  • deployments/compose/standalone.yaml
  • docs/src/content/docs/configuration.mdx
  • internal/api/router_test.go
  • internal/settings/watch.go
  • internal/settings/watch_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout. (10)
  • GitHub Check: E2E tests
  • GitHub Check: Integration tests
  • GitHub Check: Docs build
  • GitHub Check: Unit tests
  • GitHub Check: Coverage
  • GitHub Check: Validate snapshot build
  • GitHub Check: Lint
  • GitHub Check: Analyze (go)
  • GitHub Check: Analyze (javascript-typescript)
  • GitHub Check: Analyze (go)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*.go: - Go 1.26, strict formatting (gofumpt, enforced by CI)

  • No global state: Dependencies are passed explicitly (constructor injection).
  • Structured logging with log/slog (JSON handler)

Files:

  • internal/settings/watch.go
  • internal/api/router_test.go
  • internal/settings/watch_test.go
internal/*/**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

  • Package naming: Lowercase, single word (or abbreviated). internal/ enforces module privacy.

Files:

  • internal/settings/watch.go
  • internal/api/router_test.go
  • internal/settings/watch_test.go
**/*_test.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*_test.go: - Table-driven tests: Use tests := []struct{ name string; ... } with t.Run(tt.name, ...) for test cases.

  • Every new function should have corresponding test cases. Run make lint and make test before considering work complete.

Files:

  • internal/api/router_test.go
  • internal/settings/watch_test.go
docs/src/content/docs/**/*.mdx

📄 CodeRabbit inference engine (AGENTS.md)

docs/src/content/docs/**/*.mdx: - Opt a page into the Cloud CTA with cloudCta frontmatter, not by importing the component.

  • Never hand-write ® or in prose.
  • Never hand-write utm_* params or rel on a link to wavehouse.cloud or wave-rf.com. Use cloudLink() / relFor() from docs/src/config/outbound.ts.

Files:

  • docs/src/content/docs/configuration.mdx
**/*.md

📄 CodeRabbit inference engine (AGENTS.md)

Every code change should update the corresponding docs in the same PR. A code change without its doc update is incomplete.

Files:

  • CHANGELOG.md
🧠 Learnings (2)
📚 Learning: 2026-08-13T12:17:52.620Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 470
File: docs/src/content/docs/reverse-proxy.mdx:137-144
Timestamp: 2026-08-13T12:17:52.620Z
Learning: For Wave-RF/WaveHouse documentation, verify claims about implementation control flow against the authoritative implementation source (for example, internal/auth/auth.go) rather than relying solely on docs/** content. Documentation may lag behind or paraphrase behavior, so control-flow claims should be confirmed in source code.

Applied to files:

  • docs/src/content/docs/configuration.mdx
📚 Learning: 2026-06-26T12:23:22.696Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 346
File: internal/stream/subscriber_test.go:9-28
Timestamp: 2026-06-26T12:23:22.696Z
Learning: In this Go repository, prefer table-driven tests (e.g., `[]struct{...}` with `t.Run(...)`) only for tests that cover multiple scenarios/inputs and can be cleanly enumerated. Do not artificially rewrite a clear single-scenario sequential behavioral-flow test into a table-driven form just to fit the pattern; if there’s only one meaningful scenario, keep the test as a straightforward linear flow (as in `TestSubscriber_SendDeliversThenDropsWhenFull`).

Applied to files:

  • internal/settings/watch_test.go
🪛 LanguageTool
CHANGELOG.md

[style] ~13-~13: Since ownership is already implied, this phrasing may be redundant.
Context: ...et (bare docker run boots; bind-mount your own directory over it), and the dev `config...

(PRP_OWN)

🔇 Additional comments (6)
internal/api/router_test.go (1)

8-8: LGTM!

Also applies to: 160-160, 180-180, 198-198, 217-217, 256-256, 273-273, 291-291, 406-406

CHANGELOG.md (1)

13-13: LGTM!

docs/src/content/docs/configuration.mdx (1)

26-27: LGTM!

Also applies to: 42-43, 70-70, 116-116, 131-137, 188-236, 283-283, 298-304, 315-315, 328-331, 360-360, 373-376, 386-392

deployments/compose/standalone.yaml (1)

42-58: LGTM!

internal/settings/watch.go (1)

6-6: LGTM!

Also applies to: 17-17, 26-79

internal/settings/watch_test.go (1)

56-67: LGTM!

Also applies to: 70-92

Comment thread internal/api/router_test.go
Comment thread internal/settings/watch_test.go Outdated
@taitelee

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 690b4cec-07ea-43f0-86b2-faf02d9b7696

📥 Commits

Reviewing files that changed from the base of the PR and between 5646dab and 98fb9c1.

📒 Files selected for processing (1)
  • internal/settings/watch_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
🧰 Additional context used
📓 Path-based instructions (2)
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

Return errors, don't panic. Wrap with fmt.Errorf("context: %w", err).

Files:

  • internal/settings/watch_test.go
**/*_test.go

📄 CodeRabbit inference engine (AGENTS.md)

Every new function should have corresponding test cases.

Files:

  • internal/settings/watch_test.go
🔇 Additional comments (3)
internal/settings/watch_test.go (3)

5-5: LGTM!

Also applies to: 18-28, 32-34


36-43: LGTM!

Also applies to: 54-64


76-100: LGTM!

Comment thread internal/settings/watch_test.go Outdated
@taitelee

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 23, 2026
Comment on lines 20 to 42
WH_CH_ADDR: clickhouse:9000
# NATS lives at /app/data/nats, Pebble at /app/data/pebble — subdirs
# are convention, not config. The Dockerfile pre-creates /app/data
# owned by the nonroot user; bind-mount any persistent volume here.
WH_CH_HTTP_SCHEME: http
WH_DATA_DIR: /app/data
# No auth on/off switch: set WH_AUTH_JWT_SECRET to validate tokens;
# without it, every request resolves to the policy default_role.
# WH_AUTH_JWT_SECRET: "change-me"
#
# Optional non-JWT operator credential: a request presenting it
# (Authorization: Operator <key>, or the X-Operator-Key alias) gets
# full-access admin, works even if the policy is wiped (break-glass).
# Treat it as an admin secret.
# WH_AUTH_OPERATOR_KEY: "change-me-operator-key"
#
# Seed a policy on first boot. WaveHouse is fail-closed: with no policy,
# every request (even admin) is denied. dev-policy.yaml ships a permissive
# "public" trial policy (read/write the demo tables, NOT admin) so the
# quickstart works tokenless. Tune it before production — see that file.
WH_POLICY_FILE_PATH: /app/policy.yaml
WH_DEDUPE_ENABLED: "false"
# With dedupe on, reject rows missing the id field instead of publishing
# them un-deduped (default false → logged + counted, not rejected).
WH_DEDUPE_REQUIRE_ID: "false"
WH_SCHEMA_REFRESH_INTERVAL: "60"
WH_DLQ_ENABLED: "true"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these other config things not moved over too?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made a split on this based on ownership and lifecycle rather than what simply looks like configuration. Everything left in the environment variables controls resources that run for the entire process. The dedupe variable manages the Pebble store, DLQ variable creates the NATS stream, and the data directory and CH variables define where state lives. These need a full restart to change, so I kept them here The policy file path and pipes directory are the only exceptions. They are legacy seed paths that I'll remove in the next PR.

Comment thread deployments/Dockerfile.goreleaser Outdated
Comment thread docs/src/content/docs/configuration.mdx Outdated
| `dedupe.enabled` | `WH_DEDUPE_ENABLED` | `false` | Enable event deduplication. When enabled, the ingest handler checks for duplicates using the configured ID field. |
| `dedupe.id_field` | `WH_DEDUPE_ID_FIELD` | `event_id` | JSON field name in the ingest body used as the dedup key. |
| `dedupe.require_id` | `WH_DEDUPE_REQUIRE_ID` | `false` | With dedupe enabled, controls what happens to a row missing `id_field` (which can't be deduped, so idempotency wouldn't apply to it). Such a row is always logged at `WARN` and counted by `wavehouse_ingest_dedupe_missing_id_total`, in both modes. Default (`false`): it is then published un-deduped. Set `true` to reject it instead (`400` for a single insert; a per-record failure in a batch) — a server-side tripwire for producers that must guarantee the id. |
| `dedupe.enabled` | `WH_DEDUPE_ENABLED` | `false` | Enable event deduplication. Boot config because it owns the embedded Pebble store's lifecycle, which only a restart can change. |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like pebble and dedupe settings to be able to become hot-reloadable too

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on now.

Comment thread internal/settings/seed.go
Comment on lines +11 to +18
// seedFS holds the starter settings directory: every file present, every
// key set to its default. It is the ONE place defaults live — the binary
// has no compiled fallbacks — and it ships three ways: `wavehouse
// init-settings` writes it out, the container images bake it at
// /app/settings, and the compose quickstart serves from that copy.
//
//go:embed seed/*.json
var seedFS embed.FS

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this, why do it so many different ways? Like if we are aiming to embed it in the go binary, then why have the docker image builder do it a different way?

@taitelee taitelee Aug 24, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is only one source for the seed. It uses the files under internal/settings/seed. Two things use this folder. The binary embeds the files directly. This allows the init-settings command to write them anywhere without needing the source tree. The container images just copy the exact same directory. I could not run init-settings during the image build. The build platform creates the goreleaser image but the target platform compiles the binary. Running it during the build would require QEMU. I wrote the Dockerfile specifically to avoid that. The only real inconsistency was how the two Dockerfiles copied the files. I fixed that (comment somewhere above). They both use the exact same copy line now. I also updated the comment to explain this setup instead of saying it ships three ways (on next push).

Comment thread docs/src/content/docs/configuration.mdx Outdated
Comment thread docs/src/content/docs/configuration.mdx
Comment thread docs/src/content/docs/configuration.mdx Outdated
Comment thread internal/api/router.go
@github-actions github-actions Bot added the area/dedupe Deduplication (Pebble, ScyllaDB) label Aug 24, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
config.yaml (1)

52-53: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Remove the stale boot-level dedupe.enabled key.

The settings migration moves dedupe.enabled into config.json and removes WH_DEDUPE_ENABLED. This file still declares dedupe.enabled: false, so the development configuration does not follow the new contract. Remove this block and let the settings directory control deduplication.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 842876f6-f897-43f8-bac6-c712d5521fb9

📥 Commits

Reviewing files that changed from the base of the PR and between b40fc01 and 8f1bd64.

📒 Files selected for processing (36)
  • AGENTS.md
  • CHANGELOG.md
  • Makefile
  • README.md
  • cmd/wavehouse/main.go
  • cmd/wavehouse/validate_test.go
  • config.yaml
  • deployments/Dockerfile
  • deployments/Dockerfile.goreleaser
  • deployments/compose/standalone.yaml
  • docs/src/config/sidebar.ts
  • docs/src/content/docs/api.md
  • docs/src/content/docs/configuration.mdx
  • docs/src/content/docs/deployment.md
  • docs/src/content/docs/development.md
  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/index.mdx
  • docs/src/content/docs/reverse-proxy.mdx
  • docs/src/content/docs/settings-directory.mdx
  • internal/api/ingest.go
  • internal/api/ingest_test.go
  • internal/api/router_test.go
  • internal/api/settings_test.go
  • internal/config/config.go
  • internal/config/config_test.go
  • internal/dedupe/managed.go
  • internal/dedupe/managed_test.go
  • internal/settings/seed.go
  • internal/settings/seed/config.json
  • internal/settings/settings.go
  • internal/settings/store.go
  • internal/settings/store_test.go
  • internal/settings/validate.go
  • internal/settings/validate_test.go
  • tests/e2e/fixtures/config.yaml
  • tests/e2e/fixtures/settings/config.json
💤 Files with no reviewable changes (2)
  • internal/config/config_test.go
  • internal/config/config.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.mdx

📄 CodeRabbit inference engine (AGENTS.md)

  • In MDX, leave a blank line between a JSX tag and a code fence.

Files:

  • docs/src/content/docs/index.mdx
  • docs/src/content/docs/settings-directory.mdx
  • docs/src/content/docs/configuration.mdx
  • docs/src/content/docs/reverse-proxy.mdx
docs/src/content/**/*.mdx

📄 CodeRabbit inference engine (AGENTS.md)

docs/src/content/**/*.mdx: - Opt a page into the Cloud CTA with cloudCta frontmatter, not by importing the component.

  • Never hand-write ® or in prose.
  • Never hand-write utm_* params or rel on a link to wavehouse.cloud or wave-rf.com. Use cloudLink() / relFor() from docs/src/config/outbound.ts.
  • Default to top-down: flowchart TB/TD, and direction TB inside subgraphs — not LR/RL.

Files:

  • docs/src/content/docs/index.mdx
  • docs/src/content/docs/settings-directory.mdx
  • docs/src/content/docs/configuration.mdx
  • docs/src/content/docs/reverse-proxy.mdx
**/*.md

📄 CodeRabbit inference engine (AGENTS.md)

  • Never hard-wrap prose. One paragraph is one line. No wrapping at 72/80 columns, no "semantic linefeeds" splitting a paragraph at sentence boundaries.

Files:

  • README.md
  • CHANGELOG.md
  • AGENTS.md
  • docs/src/content/docs/development.md
  • docs/src/content/docs/deployment.md
  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/api.md
**/*.go

📄 CodeRabbit inference engine (AGENTS.md)

  • No global state: Dependencies are passed explicitly (constructor injection).

Files:

  • internal/settings/seed.go
  • internal/dedupe/managed_test.go
  • internal/api/ingest.go
  • cmd/wavehouse/main.go
  • internal/settings/settings.go
  • internal/settings/validate.go
  • cmd/wavehouse/validate_test.go
  • internal/api/settings_test.go
  • internal/settings/store_test.go
  • internal/dedupe/managed.go
  • internal/api/router_test.go
  • internal/settings/validate_test.go
  • internal/api/ingest_test.go
  • internal/settings/store.go
CHANGELOG.md

📄 CodeRabbit inference engine (AGENTS.md)

Every code change should update the corresponding docs in the same PR. A code change without its doc update is incomplete.

Files:

  • CHANGELOG.md
**/*_test.go

📄 CodeRabbit inference engine (AGENTS.md)

**/*_test.go: - Table-driven tests: Use tests := []struct{ name string; ... } with t.Run(tt.name, ...) for test cases.

  • Every new function should have corresponding test cases. Run make lint and make test before considering work complete.

Files:

  • internal/dedupe/managed_test.go
  • cmd/wavehouse/validate_test.go
  • internal/api/settings_test.go
  • internal/settings/store_test.go
  • internal/api/router_test.go
  • internal/settings/validate_test.go
  • internal/api/ingest_test.go
🧠 Learnings (2)
📚 Learning: 2026-08-13T12:17:52.620Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 470
File: docs/src/content/docs/reverse-proxy.mdx:137-144
Timestamp: 2026-08-13T12:17:52.620Z
Learning: For Wave-RF/WaveHouse documentation, verify claims about implementation control flow against the authoritative implementation source (for example, internal/auth/auth.go) rather than relying solely on docs/** content. Documentation may lag behind or paraphrase behavior, so control-flow claims should be confirmed in source code.

Applied to files:

  • docs/src/content/docs/settings-directory.mdx
  • docs/src/content/docs/configuration.mdx
📚 Learning: 2026-06-26T12:23:22.696Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 346
File: internal/stream/subscriber_test.go:9-28
Timestamp: 2026-06-26T12:23:22.696Z
Learning: In this Go repository, prefer table-driven tests (e.g., `[]struct{...}` with `t.Run(...)`) only for tests that cover multiple scenarios/inputs and can be cleanly enumerated. Do not artificially rewrite a clear single-scenario sequential behavioral-flow test into a table-driven form just to fit the pattern; if there’s only one meaningful scenario, keep the test as a straightforward linear flow (as in `TestSubscriber_SendDeliversThenDropsWhenFull`).

Applied to files:

  • internal/api/router_test.go
  • internal/settings/validate_test.go
🪛 LanguageTool
docs/src/content/docs/settings-directory.mdx

[style] ~14-~14: Consider an alternative for the overused word “exactly”.
Context: ...hey live, and what the server adopts is exactly what the files say. The container image...

(EXACTLY_PRECISELY)


[style] ~14-~14: Since ownership is already implied, this phrasing may be redundant.
Context: ...o a bare docker run boots; bind-mount your own directory over that path to edit. Chec...

(PRP_OWN)


[style] ~28-~28: Consider using the typographical ellipsis character here instead.
Context: ...ttings/reload** (admin-only) — returns {"adopted": bool, "findings": [...]}; 200when adopted,422` when reje...

(ELLIPSIS)

CHANGELOG.md

[style] ~13-~13: Since ownership is already implied, this phrasing may be redundant.
Context: ...t-reloadable half of configuration gets its own page; configuration.mdx is boot confi...

(PRP_OWN)


[style] ~13-~13: This sentence is over 40 words long. Consider splitting it up, as shorter sentences make the text easier to read.
Context: ...doption, matching wavehouse validate. The tenant tunables migrate out of boot config into the directory's config.json: dedupe.id_field / dedupe.require_id (now with the per-table overrides under dedupe.tables that #222 asked for, resolved per record through the table → global cascade in one atomic snapshot read, so a reload lands at a record boundary and never mixes documents within one record), query.default_max_rows (read per query), schema.refresh_interval (re-read after each tick, so a change applies from the next cycle), and the CORS allowlist (cors.allowed_origins, resolved per request; an empty list [] means allow-all, same as ["*"] — and now that the key is hand-written and required, validation warns on [] so the natural-looking spelling for "no origins" can't hot-reload into allow-all unannounced). The corresponding YAML/env keys are *...

(TOO_LONG_SENTENCE)


[style] ~13-~13: Since ownership is already implied, this phrasing may be redundant.
Context: ...et (bare docker run boots; bind-mount your own directory over it), and the dev `config...

(PRP_OWN)

docs/src/content/docs/development.md

[style] ~141-~141: Consider using the typographical ellipsis character here instead.
Context: ... air -c .air.toml ``` deps-up runs `docker compose ... up -d --wait clickhouse`, which blocks ...

(ELLIPSIS)


[style] ~231-~231: Since ownership is already implied, this phrasing may be redundant.
Context: ...ips with dedupe.enabled: false. Write your own settings directory and flip it there (t...

(PRP_OWN)

docs/src/content/docs/configuration.mdx

[style] ~178-~178: Since ownership is already implied, this phrasing may be redundant.
Context: ... validate / init-settings CLI — has its own page: [Settings Directory](/settings-di...

(PRP_OWN)

🔇 Additional comments (17)
AGENTS.md (1)

36-36: LGTM!

Also applies to: 57-65

CHANGELOG.md (1)

13-13: LGTM!

config.yaml (1)

8-9: LGTM!

Also applies to: 39-40, 67-68, 79-93

docs/src/content/docs/api.md (1)

597-597: LGTM!

Also applies to: 799-813

docs/src/config/sidebar.ts (1)

48-48: LGTM!

tests/e2e/fixtures/settings/config.json (1)

1-17: LGTM!

tests/e2e/fixtures/config.yaml (1)

8-19: LGTM!

Also applies to: 28-33

internal/settings/seed.go (1)

12-18: LGTM!

internal/settings/seed/config.json (1)

3-3: LGTM!

internal/settings/settings.go (1)

75-90: LGTM!

internal/settings/store.go (1)

32-36: LGTM!

Also applies to: 66-80, 117-138

internal/settings/validate.go (1)

321-323: LGTM!

Also applies to: 361-372

internal/settings/store_test.go (1)

77-77: LGTM!

Also applies to: 113-113, 133-133, 159-177

internal/settings/validate_test.go (1)

38-62: LGTM!

Also applies to: 264-275, 336-337

cmd/wavehouse/main.go (1)

177-188: LGTM!

Also applies to: 295-382, 487-488, 545-551

cmd/wavehouse/validate_test.go (1)

22-22: LGTM!

deployments/Dockerfile (1)

34-41: LGTM!

Comment thread docs/src/content/docs/configuration.mdx
Comment thread docs/src/content/docs/development.md Outdated
Comment thread internal/api/ingest_test.go Outdated
Comment thread internal/api/ingest.go
@taitelee

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 25, 2026
}
defer func() { _ = w.Close() }()
dir := filepath.Clean(s.dir)
if err := w.Add(dir); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only start listening here, but we read the files way earlier in settings.Open. Anything edited in between fires no event, so it just never gets picked up — no log, nothing. That gap has the whole ClickHouse dial in it.

The case that worries me is a ConfigMap update landing during a rolling restart. One burst, nothing touches the dir again, so those pods sit on stale settings until someone bounces them, and they all look healthy the entire time.

Can we just TriggerReload("watch") right after the Add? watch_test.go already retries the write in a loop to dodge this same race, so we kind of already know about it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. This actually fixes the race condition that would occur in main.go since the watcher used to start before the dedupe hook was registered. Moving the reload trigger to start after every AfterAdopt hook is in place.

Comment thread cmd/wavehouse/main.go Outdated
}
defer func() { _ = dedup.Close() }()
}
settingsStore.AfterAdopt(func() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this move above the boot apply? The read at 365 isn't under s.mu, so if a reload lands between there and here there's no hook registered yet and we end up with the settings saying dedupe is on while Managed is still off.

Different problem from the watch.go one I commented on — there the settings never make it into memory, here they do and we just don't act on them.

Window is tiny so I'm not holding the PR on it, but worth flagging that ed5c9ce made it quieter: that state used to be a wall of 500s, now it takes the new ErrDisabled path and silently publishes un-deduped instead. Apply is idempotent so hoisting this and having boot call the same closure should be fine, no?

@taitelee taitelee Aug 25, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. There is only one reconcileDedupe closure now. I registered it with AfterAdopt before the boot apply calls it. Either the hook sees a reload in the middle or the boot apply reads it. There is no race window either way. I moved the SIGHUP and watcher goroutines below this block for the same reason as the previous comment.

Comment thread internal/api/ingest.go Outdated
// read above and this call (the two transition at
// different instants). Publish un-deduped, as a record
// under the other setting would have been.
h.logger.WarnContext(ctx, "dedupe switched off mid-reload; publishing without idempotency", "event_id", eventID, "table", table)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get a counter next to this like the missing-id case does at 437? WARN exports at 100%, so a busy ingest crossing a reload is a pile of lines with nothing aggregating them.

The bigger reason though — if that main.go ordering race ever latches, this stops being transient and goes unbounded, every record forever, and a counter is the only way you'd ever notice. Probably drop the log line to Debug once the counter is carrying it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding wavehouse_ingest_dedupe_disabled_total{table} next to the missing-id counter and dropping the line to Debug

…t apply, count reload-window skips; boot-config docs omit tunables
@taitelee

Copy link
Copy Markdown
Member Author

The configuration file is only for boot settings now. I removed the intro line about tunables and the CORS sentence. I also deleted the Query, Schema Discovery, and Deduplication sections. The settings directory row is the only mention of settings left in the document. I moved all the details from those old sections into the settings directory file, includes the notes on result shaping, resource limits, refresh cycles, and the on demand endpoint.

@taitelee

Copy link
Copy Markdown
Member Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 25, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 2b26c918-4bd0-41e2-bbfe-52ee86eee647

📥 Commits

Reviewing files that changed from the base of the PR and between ed5c9ce and 50c63a9.

📒 Files selected for processing (15)
  • CHANGELOG.md
  • README.md
  • cmd/wavehouse/main.go
  • deployments/Dockerfile
  • deployments/Dockerfile.goreleaser
  • deployments/compose/standalone.yaml
  • docs/src/content/docs/configuration.mdx
  • docs/src/content/docs/deployment.md
  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/settings-directory.mdx
  • internal/api/ingest.go
  • internal/config/config.go
  • internal/settings/seed.go
  • internal/settings/watch.go
  • internal/settings/watch_test.go

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout. (1)
  • GitHub Check: Analyze (actions)
🧰 Additional context used
📓 Path-based instructions (8)
- **Column-level access control is a hard cap on every read path.**

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal/api/ingest.go
- **Opt a page into the Cloud CTA with `cloudCta` frontmatter**, not by importing the component.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • docs/src/content/docs/settings-directory.mdx
  • docs/src/content/docs/configuration.mdx
- **Table-driven tests**: Use `tests := []struct{ name string; ... }` with `t.Run(tt.name, ...)` for test cases.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal/settings/watch_test.go
- **Every new function should have corresponding test cases.** Run `make lint` and `make test` before considering work complete.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal/settings/watch.go
  • internal/settings/seed.go
  • cmd/wavehouse/main.go
  • internal/settings/watch_test.go
  • internal/config/config.go
  • internal/api/ingest.go
- **In MDX, leave a blank line between a JSX tag and a code fence.**

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • docs/src/content/docs/settings-directory.mdx
  • docs/src/content/docs/configuration.mdx
- **No global state**: Dependencies are passed explicitly (constructor injection).

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • internal/settings/watch.go
  • internal/settings/seed.go
  • cmd/wavehouse/main.go
  • internal/settings/watch_test.go
  • internal/config/config.go
  • internal/api/ingest.go
Every code change should update the corresponding docs in the same PR. A code change without its doc update is incomplete.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • deployments/Dockerfile
  • README.md
  • internal/settings/watch.go
  • CHANGELOG.md
  • docs/src/content/docs/getting-started.md
  • internal/settings/seed.go
  • docs/src/content/docs/settings-directory.mdx
  • deployments/compose/standalone.yaml
  • cmd/wavehouse/main.go
  • docs/src/content/docs/configuration.mdx
  • internal/settings/watch_test.go
  • docs/src/content/docs/deployment.md
  • internal/config/config.go
  • deployments/Dockerfile.goreleaser
  • internal/api/ingest.go
- **Never hard-wrap prose. One paragraph is one line.** No wrapping at 72/80 columns, no "semantic linefeeds" splitting a paragraph at sentence boundaries.

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • README.md
  • CHANGELOG.md
  • docs/src/content/docs/getting-started.md
  • docs/src/content/docs/deployment.md
🧠 Learnings (2)
📚 Learning: 2026-08-13T12:17:52.620Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 470
File: docs/src/content/docs/reverse-proxy.mdx:137-144
Timestamp: 2026-08-13T12:17:52.620Z
Learning: For Wave-RF/WaveHouse documentation, verify claims about implementation control flow against the authoritative implementation source (for example, internal/auth/auth.go) rather than relying solely on docs/** content. Documentation may lag behind or paraphrase behavior, so control-flow claims should be confirmed in source code.

Applied to files:

  • docs/src/content/docs/settings-directory.mdx
📚 Learning: 2026-06-26T12:23:22.696Z
Learnt from: EricAndrechek
Repo: Wave-RF/WaveHouse PR: 346
File: internal/stream/subscriber_test.go:9-28
Timestamp: 2026-06-26T12:23:22.696Z
Learning: In this Go repository, prefer table-driven tests (e.g., `[]struct{...}` with `t.Run(...)`) only for tests that cover multiple scenarios/inputs and can be cleanly enumerated. Do not artificially rewrite a clear single-scenario sequential behavioral-flow test into a table-driven form just to fit the pattern; if there’s only one meaningful scenario, keep the test as a straightforward linear flow (as in `TestSubscriber_SendDeliversThenDropsWhenFull`).

Applied to files:

  • internal/settings/watch_test.go
🪛 LanguageTool
CHANGELOG.md

[style] ~13-~13: Since ownership is already implied, this phrasing may be redundant.
Context: ...t-reloadable half of configuration gets its own page; configuration.mdx is boot confi...

(PRP_OWN)


[style] ~13-~13: This sentence is over 40 words long. Consider splitting it up, as shorter sentences make the text easier to read.
Context: ...doption, matching wavehouse validate. The tenant tunables migrate out of boot config into the directory's config.json: dedupe.id_field / dedupe.require_id (now with the per-table overrides under dedupe.tables that #222 asked for, resolved per record through the table → global cascade in one atomic snapshot read, so a reload lands at a record boundary and never mixes documents within one record), query.default_max_rows (read per query), schema.refresh_interval (re-read after each tick, so a change applies from the next cycle), and the CORS allowlist (cors.allowed_origins, resolved per request; an empty list [] means allow-all, same as ["*"] — and now that the key is hand-written and required, validation warns on [] so the natural-looking spelling for "no origins" can't hot-reload into allow-all unannounced). The corresponding YAML/env keys are *...

(TOO_LONG_SENTENCE)

docs/src/content/docs/settings-directory.mdx

[style] ~14-~14: Consider an alternative for the overused word “exactly”.
Context: ...hey live, and what the server adopts is exactly what the files say. The container image...

(EXACTLY_PRECISELY)

🔇 Additional comments (7)
CHANGELOG.md (1)

13-13: LGTM!

README.md (1)

135-135: LGTM!

docs/src/content/docs/configuration.mdx (1)

26-26: LGTM!

Also applies to: 42-42, 66-66, 106-106, 115-115, 162-166, 214-214, 228-231, 254-258

docs/src/content/docs/deployment.md (1)

19-20: LGTM!

Also applies to: 185-185, 367-367

docs/src/content/docs/getting-started.md (1)

23-24: LGTM!

docs/src/content/docs/settings-directory.mdx (1)

14-18: LGTM!

Also applies to: 44-45, 70-70

cmd/wavehouse/main.go (1)

356-360: 🗄️ Data Integrity & Integration

No change needed for this concern. When Managed.Apply(true) fails, CheckAndMark returns dedupe.ErrUnavailable, and internal/api/ingest.go aborts with HTTP 500 instead of publishing without idempotency.

Comment thread docs/src/content/docs/deployment.md Outdated
Comment thread internal/api/ingest.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/api HTTP handlers, routing, middleware area/dedupe Deduplication (Pebble, ScyllaDB) area/docs Documentation, site/, README area/infra CI, build, deploy, Docker, release area/query Structured query AST, SQL builder dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation go Pull requests that update go code

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

2 participants