Skip to content

feat(write,query,logs): msgpack/json writes, query --estimate, logs, import stats, signal-aware root (PR9) - #9

Merged
xe-nvdk merged 1 commit into
mainfrom
feat/pr9-msgpack-estimate-logs
Sep 7, 2026
Merged

xe-nvdk merged 1 commit into
mainfrom
feat/pr9-msgpack-estimate-logs

Conversation

@xe-nvdk

@xe-nvdk xe-nvdk commented Sep 7, 2026

Copy link
Copy Markdown
Member

Stacked on #8 (feat/pr8-delete-backup); merge that first.

Summary

  • arcli write --format lp|msgpack|json: pass-through MessagePack to /api/v1/write/msgpack (exact length for files, chunked for stdin, 1 GiB cap, gzip/zstd passed through, first-byte guard against text files) and a validating JSON→MessagePack converter (64 MiB) that rejects up front what the server would 400, 500, or silently drop: ragged/empty columns, mixed types, integers beyond int64, a time column mixing units, nested values, the unsupported compact f array. --precision is line-protocol only.
  • arcli query --estimate: /api/v1/query/estimate; success:false on HTTP 200 is surfaced as an error, -o json prints the raw document.
  • arcli logs: GET /api/v1/logs with --level (minimum), --since (1m–24h, rounded up to minutes), --limit (1–1000); table/json/csv. Help notes the per-process ring buffer and the load-balancer caveat; "logs": null is normalised to [] in JSON output.
  • arcli import stats: process-wide import counters.
  • arcli config create|update --token-stdin: reads the token from a pipe or file; a terminal is refused (the token would be echoed), the value must be printable ASCII, and the other flags are validated before stdin is drained.
  • Signal-aware root: SIGINT/SIGTERM cancel the command context; arcli exits 130/143 with a one-line reminder that anything the server already accepted continues there. A second signal kills the process. retention execute --wait, cq run --wait, and delete --wait treat an interrupt like a timeout for their hints.
  • Hand-rolled MessagePack encoder (no new dependency); go.mod is unchanged.

Arc-side observations (not changed here)

  • /api/v1/write/msgpack silently drops batch items or array elements that fail to decode and still answers 204; only a whole-document failure is a 400. The JSON path exists mostly to give users a validating front door.
  • The columnar shape carries no tag metadata (every column is a field); only the row shape keeps tags as tag columns.
  • /api/v1/query/estimate reports failures as HTTP 200 with success:false.

Test plan

  • gofmt -l . empty, go vet ./... clean, go test -race -count=1 ./... green
  • Unit: MessagePack spec byte vectors; columnar/row/batch/array conversion summaries; ~30 rejection cases incl. oversize document; first-byte allowlist; WriteMsgPack headers/400/oversize; estimate 200-false/400/504; logs params, null normalisation (struct and raw), validation, 403; import stats; --token-stdin pipe/CRLF/empty/both-flags/non-ASCII/typo-before-read/tty//dev/null
  • Live against arc serve (auth on): msgpack columnar + row + batch + gzip file + stdin; JSON columnar/row/batch; rows verified via arcli query after buffer flush; --format msgpack on a line-protocol file rejected with the first-byte message; query --estimate table and json; logs table/json/csv with --level/--since, empty window; import stats; --token-stdin create/update; SIGINT → 130 and SIGTERM → 143 mid-dial
  • Internal deep review (configuration matrix) + security review: no blockers or highs; all mediums applied (JSON cap and oversize error, TTY refusal, ASCII token check, stdin ordering, logs null in JSON, --timeout start after JSON conversion, SIGTERM exit code)

…import stats, signal-aware root (PR9)

- write --format lp|msgpack|json: pass-through MessagePack (1 GiB, gzip/zstd sniffed) and a
  validating JSON→MessagePack converter (64 MiB) that rejects what the server would 400,
  500, or silently drop
- query --estimate against /api/v1/query/estimate
- logs: GET /api/v1/logs with --level (minimum), --since (1m-24h), --limit; table/json/csv
- import stats: process-wide import counters
- config create/update --token-stdin (pipe or file; terminal refused; printable ASCII)
- root: SIGINT/SIGTERM cancel the command context; exit 130/143 with a server-continues hint
- retention/cq/delete: interrupt counts as "arcli gave up" for the wait hints
@xe-nvdk
xe-nvdk changed the base branch from feat/pr8-delete-backup to main September 7, 2026 22:19
@xe-nvdk
xe-nvdk merged commit 900ec16 into main Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant