feat(write,query,logs): msgpack/json writes, query --estimate, logs, import stats, signal-aware root (PR9) - #9
Merged
Conversation
…import stats, signal-aware root (PR9) - write --format lp|msgpack|json: pass-through MessagePack (1 GiB, gzip/zstd sniffed) and a validating JSON→MessagePack converter (64 MiB) that rejects what the server would 400, 500, or silently drop - query --estimate against /api/v1/query/estimate - logs: GET /api/v1/logs with --level (minimum), --since (1m-24h), --limit; table/json/csv - import stats: process-wide import counters - config create/update --token-stdin (pipe or file; terminal refused; printable ASCII) - root: SIGINT/SIGTERM cancel the command context; exit 130/143 with a server-continues hint - retention/cq/delete: interrupt counts as "arcli gave up" for the wait hints
Merged
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #8 (
feat/pr8-delete-backup); merge that first.Summary
arcli write --format lp|msgpack|json: pass-through MessagePack to/api/v1/write/msgpack(exact length for files, chunked for stdin, 1 GiB cap, gzip/zstd passed through, first-byte guard against text files) and a validating JSON→MessagePack converter (64 MiB) that rejects up front what the server would 400, 500, or silently drop: ragged/empty columns, mixed types, integers beyond int64, atimecolumn mixing units, nested values, the unsupported compactfarray.--precisionis line-protocol only.arcli query --estimate:/api/v1/query/estimate;success:falseon HTTP 200 is surfaced as an error,-o jsonprints the raw document.arcli logs:GET /api/v1/logswith--level(minimum),--since(1m–24h, rounded up to minutes),--limit(1–1000); table/json/csv. Help notes the per-process ring buffer and the load-balancer caveat;"logs": nullis normalised to[]in JSON output.arcli import stats: process-wide import counters.arcli config create|update --token-stdin: reads the token from a pipe or file; a terminal is refused (the token would be echoed), the value must be printable ASCII, and the other flags are validated before stdin is drained.retention execute --wait,cq run --wait, anddelete --waittreat an interrupt like a timeout for their hints.go.modis unchanged.Arc-side observations (not changed here)
/api/v1/write/msgpacksilently drops batch items or array elements that fail to decode and still answers 204; only a whole-document failure is a 400. The JSON path exists mostly to give users a validating front door./api/v1/query/estimatereports failures as HTTP 200 withsuccess:false.Test plan
gofmt -l .empty,go vet ./...clean,go test -race -count=1 ./...greenWriteMsgPackheaders/400/oversize; estimate 200-false/400/504; logs params, null normalisation (struct and raw), validation, 403; import stats;--token-stdinpipe/CRLF/empty/both-flags/non-ASCII/typo-before-read/tty//dev/nullarc serve(auth on): msgpack columnar + row + batch + gzip file + stdin; JSON columnar/row/batch; rows verified viaarcli queryafter buffer flush;--format msgpackon a line-protocol file rejected with the first-byte message;query --estimatetable and json;logstable/json/csv with--level/--since, empty window;import stats;--token-stdincreate/update; SIGINT → 130 and SIGTERM → 143 mid-dial--timeoutstart after JSON conversion, SIGTERM exit code)