feat(delete,backup): predicate delete, full backup and restore (PR8) - #8
Merged
Merged
Conversation
arcli delete --database --measurement --where over /api/v1/delete/:
the predicate is sent as "(w) IS TRUE" so Arc's rewrite (NOT (w),
which also drops NULL rows) matches its own preview; without --yes a
server dry run feeds the confirmation prompt; a zero-match preview is
verified with a COUNT query so a mistyped column is an error, not
"0 rows"; partial results (207 or a mid-run 5xx) are reported.
arcli backup {create,list,show,status,delete,restore} over
/api/v1/backup/*. The server publishes progress from a goroutine after
the 202, so create/restore snapshot the status first and wait for an
operation that is newer than it; --wait polls to completion and exits
non-zero on failure in every output mode. Restore is prompted, derives
its staged-metadata warning from the manifest, and refuses
--with-config for a backup that has none.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
arcli delete --database DB --measurement M --where SQL [--dry-run] [--yes]overPOST /api/v1/delete/(admin; the server refuses with 403 unlessdelete.enabled=true). Arc rewrites affected Parquet files without the matching rows and removes a file when every row matches.NOT (w)rows, which under three-valued logic also drops rows whose predicate is NULL, while its preview counts only TRUE rows. arcli sends(w) IS TRUEon both calls so preview and rewrite agree and NULL rows are kept (standard DELETE semantics). Verified on real data in the smoke. Worth an Arc fix: rewrite withWHERE NOT COALESCE((w), FALSE).--yes: non-interactive stdin refused first, then a server dry run (dry_run:true, confirm:true, so a preview above the confirmation threshold isn't refused) feeds a prompt with the server's row/file counts and its configured limits; full-table predicates get "ALL rows" wording.--dry-runonly reports;--yesskips the preflight scan and the prompt.SELECT count(*) … WHERE (w) IS TRUEthrough the query endpoint: an HTTP 400 → "the WHERE clause does not evaluate against DB.M: …"; any other failure → warning, zero result stands.success:false/failed_files→PartialDeleteErrorwith the decoded result; the command prints the failed files and exits 1. Preview-vs-real count mismatch → warning.WHEREstripped; structural checks (;,--,/*, unbalanced quotes/parens); the server's keyword/function denylist is authoritative and surfaced verbatim.--timeoutdefaults to 30m; on a client timeout the server keeps rewriting (message says so).arcli backup {create,list,show,status,delete,restore}over/api/v1/backup/*(admin;FeatureDisabledErrorwhenbackup.enabled=false).create/restoresnapshot the status before the POST and wait (bounded) for an operation of our kind that is newer than the snapshot before printing the id or entering--wait.--wait(default--wait-timeout2h, the server's budget) exits non-zero onfailedin table and JSON modes.backup deleteGETs first (the server answers 500 for an unknown id) and re-GETs on a 500 to report "no longer exists".backup restoreGETs the manifest first (the server 202s a missing id), derives "metadata staged / config restored" from the manifest ∩ flags, refuses--with-configwhen the backup has none, prompts with the databases, overwrite semantics, and quiescence advice, and prints the restart note whenever metadata/config were included.idlewhile waiting → "no longer tracked (server restarted?)".HTTPErrornow carries the raw body for structured errors).HTTPError(also benefitsarcli query).Test plan
gofmt -l .empty,go vet ./...,go test -race -count=1 ./...greenDeleteRowson 200/207/500-with-counts/403/400, backup id regex, 409 operation, list/show/status shapes incl. idle, restore body; command fakes for the delete preflight/prompt/real sequence (asserting exactly one dry-run request after "N"), zero-match disambiguation in all modes, partial failure, disabled server; backup fake modelling the delayed publish, second create after a completed one reporting the new id, restore prompt content,--with-configrefusal, data-only vs default restart note, JSON--waitfailure exits non-zeroarc serve(HEAD e5c3f5e,ARC_DELETE_ENABLED=true, local backup path) with rows incl. a NULL tag: preview 3 / deleted 3 / NULL row survived; non-TTY refused before preflight; structural and server denylist errors; unknown measurement zeros; unknown column → "does not evaluate" in dry-run,--yes, and interactive; genuine zero match → zeros; full-table JSON; backup create prints a new id even right after a completed backup;--waittable and JSON; list newest-first; show manifest; bad id client-side; delete missing → 404 before prompt; restore--data-only --waitwith no staged note; default restore prints the restart note;--with-configrefused; restore--waittracks the right idARC_BACKUP_ENABLED=false→ "backups are disabled"; default server (delete.enabled=false) → 403 verbatimReview notes
Internal: adversarial plan review (NULL-predicate server bug; backup-id publish race; zero-match ambiguity; restore flags computed from the request not the manifest), deep review (High: JSON
--waitexited 0 on failure; start detection reworked to compare against the snapshot's own operation after a clock-slack version was caught by a unit test), security review (checklist PASS; query error decoder scrubbed). Follow-ups noted in README: composed--before/--afterflags.