Repository navigation
feat(setup): clone local SQLite database when creating new worktree - #1466
Conversation
📝 WalkthroughWalkthroughAdds Changes
Sequence Diagram(s)sequenceDiagram
participant CLI as "User / CI"
participant Setup as "setup.sh"
participant FS as "Filesystem (workspace dirs)"
participant Neon as "Neon (branches)"
participant Electric as "Electric SQL"
CLI->>Setup: run setup
Setup->>FS: sanitize workspace name, check dest exists
alt dest absent
Setup->>FS: create dir, set secure permissions
Setup->>FS: copy source DB file
Setup->>FS: copy WAL/SHM if present
FS-->>Setup: copy success
else dest present
FS-->>Setup: skip copy
end
Setup->>Neon: step_setup_neon_branch (create/use branch)
Neon-->>Setup: branch info
Setup->>Electric: start Electric SQL
Electric-->>Setup: started
Setup->>CLI: write .env and finish
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.superset/setup.sh:
- Around line 196-200: Remove the unsupported flag from the neonctl invocation:
the branch creation call using "neonctl branches create" (which assigns output
to the neon_output variable) should not pass "--schema-only false"; edit the
code that builds the command (the if ! neon_output=$(neonctl branches create ...
) then block) to drop the "--schema-only false" argument and rely on the default
behavior that includes schema and data when creating a branch from the parent
(references: neonctl branches create, neon_output, WORKSPACE_NAME,
NEON_PROJECT_ID).
| if ! neon_output=$(neonctl branches create \ | ||
| --project-id "$NEON_PROJECT_ID" \ | ||
| --name "$WORKSPACE_NAME" \ | ||
| --schema-only false \ | ||
| --output json); then |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
neonctl branches create --schema-only flag documentation
💡 Result:
The --schema-only flag is documented under Neon CLI → branches create:
- Command:
neon branches create --schema-only(works the same with theneonctlalias) - What it does: creates a schema-only branch (schema copied, no data).
- Constraint: it “requires exactly one read-write compute.” [1]
- Context:
neonis the CLI command name, withneonctlas an alias. [2]
Sources:
[1] Neon Docs — Neon CLI commands — branches (create options, --schema-only) (neon.com)
[2] Neon Docs — Neon CLI (Overview) (shows neon usage and neonctl alias) (neon.com)
Citations:
🌐 Web query:
neonctl --schema-only false syntax examples
💡 Result:
As of the Neon docs update April 17, 2025, schema-only branches were not yet supported in neonctl (“Support for the Neon CLI will come in a future release”). [1] So neonctl ... --schema-only false is likely not a valid flag (at least per the published docs).
Boolean false syntax in neonctl (working pattern)
neonctl boolean options accept either:
--flag false--no-flag
Example from the CLI docs (analytics is true by default): [2]
neonctl projects list --no-analytics
neonctl projects list --analytics falseIf your goal is “schema-only = false” (i.e., include data)
Neon branches created via CLI are schema+data by default (so you typically don’t need to pass anything to mean “false”). Example: [3]
neonctl branches create --project-id=<your_project_id>If your goal is “schema-only = true” (schema-only branch)
Use the Neon API with init_source: "schema-only" (example from docs): [4]
curl --request POST \
--url https://console.neon.tech/api/v2/projects/<project_id>/branches \
--header 'accept: application/json' \
--header "authorization: Bearer $NEON_API_KEY" \
--header 'content-type: application/json' \
--data '{
"branch": {
"parent_id": "<parent_branch_id>",
"name": "my_schema_only_branch",
"init_source": "schema-only"
}
}'If you tell me which neonctl command you’re running (e.g., branches create, some dump/export command, etc.) and your neonctl --version, I can confirm whether --schema-only exists in that specific version and the exact accepted syntax.
🏁 Script executed:
cd .superset && grep -r "neonctl" . --include="*.json" --include="*.txt" --include="*.md" | head -20Repository: superset-sh/superset
Length of output: 46
Remove the --schema-only false flag—it is not a documented neonctl option and will likely cause the command to fail.
The --schema-only flag is not supported in neonctl as of the current Neon CLI documentation (April 2025). More importantly, the default behavior of neonctl branches create already includes both schema and data from the parent branch, making this flag unnecessary. To simplify, remove --schema-only false from line 199 and rely on the default branch creation behavior.
🤖 Prompt for AI Agents
In @.superset/setup.sh around lines 196 - 200, Remove the unsupported flag from
the neonctl invocation: the branch creation call using "neonctl branches create"
(which assigns output to the neon_output variable) should not pass
"--schema-only false"; edit the code that builds the command (the if !
neon_output=$(neonctl branches create ... ) then block) to drop the
"--schema-only false" argument and rely on the default behavior that includes
schema and data when creating a branch from the parent (references: neonctl
branches create, neon_output, WORKSPACE_NAME, NEON_PROJECT_ID).
🧹 Preview Cleanup CompleteThe following preview resources have been cleaned up:
Thank you for your contribution! 🎉 |
Copy ~/.superset/local.db to ~/.superset-<workspace>/local.db during workspace setup so new worktrees start with existing data (projects, settings, synced tables) instead of an empty database. Skips the clone if the destination already exists, the source database is missing, or the workspace is the default "superset" instance. Also copies WAL/SHM files for consistency.
b5da1f1 to
f289cc2
Compare
Kitenite
changed the title
The dev workspace (~/.superset-dev/) is the correct source for cloning the local SQLite database to new worktrees. Also skip the clone when running in the dev workspace itself.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In @.superset/setup.sh:
- Around line 189-202: Replace the raw cp of the SQLite file and the manual
WAL/SHM copy loop with a single sqlite3 backup invocation to produce a
consistent snapshot: call sqlite3 against the source DB with the .backup command
to create the destination DB, check the sqlite3 exit status and log/return an
error on failure, then set permissions on the resulting dest file; also add a
check in step_check_dependencies to ensure sqlite3 is available before
attempting the backup. Ensure you update the logic that currently references
copying WAL/SHM (the cp + for-suffix loop) and the chmod calls to instead run
only after a successful sqlite3 .backup.
| # Copy the database file (and WAL/SHM files if they exist) | ||
| if ! cp "$source_db" "$dest_db"; then | ||
| error "Failed to copy local database" | ||
| return 1 | ||
| fi | ||
| chmod 600 "$dest_db" | ||
|
|
||
| # Copy WAL and SHM files if present for consistency | ||
| for suffix in "-wal" "-shm"; do | ||
| if [ -f "${source_db}${suffix}" ]; then | ||
| cp "${source_db}${suffix}" "${dest_db}${suffix}" 2>/dev/null || true | ||
| chmod 600 "${dest_db}${suffix}" 2>/dev/null || true | ||
| fi | ||
| done |
There was a problem hiding this comment.
Prefer sqlite3 .backup over raw file copy to avoid a corrupt clone.
Copying a SQLite database and its WAL/SHM files with separate cp commands is not atomic. If the source database is open in WAL mode and another process checkpoints or writes between the copies, the destination can end up with an inconsistent (corrupt) database. The SQLite documentation explicitly warns against this.
Using the built-in backup API via the sqlite3 CLI produces a consistent snapshot in a single step and eliminates the need to handle WAL/SHM files separately.
Proposed fix using sqlite3 .backup
- # Copy the database file (and WAL/SHM files if they exist)
- if ! cp "$source_db" "$dest_db"; then
- error "Failed to copy local database"
- return 1
- fi
- chmod 600 "$dest_db"
-
- # Copy WAL and SHM files if present for consistency
- for suffix in "-wal" "-shm"; do
- if [ -f "${source_db}${suffix}" ]; then
- cp "${source_db}${suffix}" "${dest_db}${suffix}" 2>/dev/null || true
- chmod 600 "${dest_db}${suffix}" 2>/dev/null || true
- fi
- done
+ # Use sqlite3 backup API for a consistent snapshot (safe even if source is in WAL mode)
+ if ! sqlite3 "$source_db" ".backup '$dest_db'"; then
+ error "Failed to clone local database"
+ return 1
+ fi
+ chmod 600 "$dest_db"You may also want to add sqlite3 to the dependency check in step_check_dependencies (it's usually pre-installed on macOS/Linux but worth verifying):
+ if ! command -v sqlite3 &> /dev/null; then
+ missing+=("sqlite3 (usually pre-installed; Run: brew install sqlite)")
+ fi🤖 Prompt for AI Agents
In @.superset/setup.sh around lines 189 - 202, Replace the raw cp of the SQLite
file and the manual WAL/SHM copy loop with a single sqlite3 backup invocation to
produce a consistent snapshot: call sqlite3 against the source DB with the
.backup command to create the destination DB, check the sqlite3 exit status and
log/return an error on failure, then set permissions on the resulting dest file;
also add a check in step_check_dependencies to ensure sqlite3 is available
before attempting the backup. Ensure you update the logic that currently
references copying WAL/SHM (the cp + for-suffix loop) and the chmod calls to
instead run only after a successful sqlite3 .backup.
Summary
~/.superset/local.dbto~/.superset-<workspace>/local.dbSUPERSET_DIR_NAME(e.g.,~/.superset-my-branch/)Changes
.superset/setup.sh: Addedstep_clone_local_db()function (new Step 4) that:env.shared.tsgetWorkspaceName()logiclocal.db+ WAL/SHM files with secure permissions (700/600)Test Plan
~/.superset-<name>/local.dbis copied from~/.superset/local.dblocal.dbare not overwrittenSummary by CodeRabbit