fix(dappnode): bridge legacy state through schema v1

Author: hmzakhalid

agent/CRATES_ARCHITECTURE.md

@@ -249,6 +249,10 @@ identity pair as fresh. A partial identity, any additional unversioned key, any
 log without a marker, upgrades, and downgrades fail closed. This narrow exception is
 required because autowallet atomically creates the two bootstrap identities before the
 builder can stamp the schema; it does not let protocol state bypass compatibility checks.
+The DAppNode v0.2.3 package is the explicit bridge for the previously shipped v0.1.8
+state: its entrypoint atomically moves `/data/.enclave` to `/data/.interfold`, and the
+v0.2.3 release stamps schema version 1 before a later fail-closed binary is installed.
+If both namespace roots exist, the bridge refuses to choose between them.
 
 ## Actor and message topology
 

agent/flow-trace/00_INDEX.md

@@ -203,3 +203,4 @@ _Found during source-code cross-referencing of these trace documents._
 | 14  | **Post-startup network lag recovery**               | Resolved | `NetEventBuffer` still fails readiness closed if its broadcast input lags during startup buffering, because historical/live reconciliation would be incomplete. Once `SyncEnded` has moved it to `Running`, a `Lagged(n)` signal is warned and skipped, and the ingress receive loop continues from the oldest retained event instead of stopping the buffer and permanently deafening gossip consumers.                                                                                                                                                                                                                                                                                                                         |
 | 15  | **Local/network event redelivery collision**        | Resolved | EventStore now treats the same HLC timestamp plus stable event ID and equal payload as an idempotent duplicate even when transport context differs (`Local` versus `Net`). It retains the first stored context and still fails closed when different payloads claim the same timestamp, preserving collision detection without panicking on sync/resync redelivery.                                                                                                                                                                                                                                                                                                                                                          |
 | 16  | **Crash-torn event-log tail**                        | Resolved | Startup validates active-segment frames against the commitlog index, truncates only an unindexed CRC/length-invalid physical suffix, and restores complete CRC-valid/decodable records whose tail index entry was lost. Indexed corruption remains fatal. Runtime reads return correlated errors instead of panicking query handlers, and `node validate --repair` exposes the same narrowly bounded recovery explicitly.                                                                                                                                                                                                                                                   |
+| 17  | **DAppNode v0.1.8 schema upgrade**                   | Resolved | DAppNode v0.2.3 is shipped as the required compatibility bridge. Its entrypoint atomically renames the legacy `/data/.enclave` state root to `/data/.interfold` (and refuses ambiguous dual roots), preserves existing encrypted identities, and then relies on the v0.2.3 release's one-time schema-1 stamp. Later fail-closed binaries therefore see a proven marker instead of permanently rejecting the shipped v0.1.8 datastore.                                                                                                                                                                                        |

agent/flow-trace/06_DEACTIVATION_AND_COMPLETION.md

@@ -287,6 +287,11 @@ recovery as startup and refuses to remove indexed records. Runtime EventStore qu
 returned to the correlated caller rather than panicking the actor; committed corruption remains a
 startup/integrity failure.
 
+For DAppNode installations, package v0.2.3 is the mandatory bridge from the shipped v0.1.8 state.
+It atomically moves the legacy `.enclave` custom-config root to `.interfold`, preserves the encrypted
+operator/libp2p identity, and lets the v0.2.3 binary stamp schema version 1 before later binaries
+enforce the marker. An ambiguous volume containing both roots fails closed.
+
 ### Restart + Persist State Diagram
 
 ```mermaid

dappnode/README.md

@@ -155,25 +155,30 @@ block heights.
 
 ### Credentials file
 
-Create a local JSON file containing all three credential values:
+Create a local JSON file containing the password and operator key:
 
 ```json
 {
   "password": "a strong encryption password",
-  "private_key": "0x<64 hex characters>",
-  "network_private_key": "0x<64 hex characters>"
+  "private_key": "0x<64 hex characters>"
 }
 ```
 
 Upload it in the setup wizard as **Ciphernode Credentials JSON**. DAppNode copies it to
 `/run/secrets/secrets.json` before starting the container. The entrypoint validates a maximum size
 of 16 KiB, required fields, and key encodings, then runs the exact commands supported by the pinned
-Interfold v0.1.8 image. Any failed command aborts startup. Wallet and network keys are encrypted in
-`/data`; v0.1.8 stores its password key there as a mode-`0400` file. After successful persistence,
+Interfold v0.2.3 image. Any failed command aborts startup. The wallet command atomically derives and
+stores both the Ethereum and libp2p identities. Both keys are encrypted in `/data`; v0.2.3 stores
+its password key there as a mode-`0400` file. After successful persistence,
 the entrypoint removes the combined plaintext upload. Provisioning sends the secrets through the
 CLI's hidden TTY prompts over stdin; plaintext credentials are never placed in process arguments or
 container environment variables.
 
+Legacy three-field files containing `network_private_key` are accepted for upgrade compatibility,
+but v0.2.3 ignores that obsolete field on a fresh setup. When encrypted identity state already
+exists, a matching upload is removed without rewriting either identity; a password mismatch fails
+closed.
+
 The Ethereum key must correspond to `NODE_ADDRESS`. Keep an encrypted offline backup of this JSON;
 do not paste its contents into package environment variables, `EXTRA_OPTS`, logs, or support
 bundles. On an ordinary restart, persisted credential state is reused from `/data` and the upload is
@@ -208,9 +213,8 @@ At container startup, `entrypoint.sh`:
 - network name and RPC URL
 - contract addresses and deploy blocks
 
-4. Validates the uploaded credential file and programs password, network key, and wallet key. The
-   pinned v0.1.8 network-key command is `interfold net keypair set`; an isolated Expect helper feeds
-   its hidden prompts from stdin and failures stop startup.
+4. Validates the uploaded credential file and programs the password plus atomic wallet/network
+   identity. An isolated Expect helper feeds hidden prompts from stdin and failures stop startup.
 5. Builds CLI args, including verbosity and `--peer` flags from `PEERS`.
 6. Executes:
 
@@ -223,12 +227,22 @@ The state and databases live under `/data` inside the container, which is backed
 that volume includes the password key, DAppNode backups of `/data` must be encrypted and
 access-controlled even though the wallet and network keys inside the volume are encrypted.
 
+### Required v0.1.8 upgrade bridge
+
+DAppNode package v0.2.3 is the required bridge from the shipped v0.1.8 package to later binaries.
+On its first start it atomically renames the custom-config state root from `/data/.enclave` to
+`/data/.interfold`, refusing to proceed if both roots exist. The v0.2.3 binary then stamps schema
+version 1 using its release-era compatibility behavior. Later fail-closed binaries can therefore
+verify the marker instead of rejecting the old unversioned datastore. Do not skip this package when
+upgrading an existing v0.1.8 node, and keep a verified backup of `/data` until the bridge has started
+successfully.
+
 ## Health semantics
 
-Interfold v0.1.8 does not expose a local readiness endpoint. The package health check therefore uses
+Interfold v0.2.3 does not expose a local readiness endpoint. The package health check therefore uses
 the strongest non-invasive local signals available in that release: PID 1 must be the expected
 `interfold start` command using `/data/config.yaml`, the protected config/password files must exist,
-the v0.1.8 Sled/event-log directories must be initialized, and the configured QUIC UDP listener must
+the v0.2.3 Sled/event-log directories must be initialized, and the configured QUIC UDP listener must
 be bound. This detects the old false-positive case where an unrelated process matched `pgrep`, as
 well as missing credentials, uninitialized persistence, and a dead network listener.
 

dappnode/dappnode_package.json

@@ -1,7 +1,7 @@
 {
   "name": "interfold-ciphernode.public.dappnode.eth",
-  "version": "0.1.8",
-  "upstreamVersion": "0.1.8",
+  "version": "0.2.3",
+  "upstreamVersion": "0.2.3",
   "upstreamArg": "UPSTREAM_VERSION",
   "upstreamRepo": "theinterfold/interfold",
   "shortDescription": "Interfold Ciphernode - Run a node for confidential computing",

dappnode/docker-compose.yml

@@ -4,8 +4,8 @@ services:
     build:
       context: .
       args:
-        UPSTREAM_VERSION: 0.1.8
-    image: 'ciphernode.interfold-ciphernode.public.dappnode.eth:0.1.8'
+        UPSTREAM_VERSION: 0.2.3
+    image: 'ciphernode.interfold-ciphernode.public.dappnode.eth:0.2.3'
     restart: unless-stopped
     # Allow the node's 30-second durability barrier to finish before Docker sends SIGKILL.
     stop_grace_period: 45s

dappnode/entrypoint.sh

@@ -9,9 +9,11 @@ CONFIG_FILE="${CONFIG_FILE:-$CONFIG_DIR/config.yaml}"
 TEMPLATE_FILE="${TEMPLATE_FILE:-/opt/config.template.yaml}"
 SECRETS_FILE="${SECRETS_FILE:-/run/secrets/secrets.json}"
 CREDENTIAL_PROVISIONER="${CREDENTIAL_PROVISIONER:-/opt/provision-credentials.exp}"
-# Interfold v0.1.8 resolves a relative `key_file: key` beside a discovered
+LEGACY_STATE_DIR="${LEGACY_STATE_DIR:-$CONFIG_DIR/.enclave}"
+CURRENT_STATE_DIR="${CURRENT_STATE_DIR:-$CONFIG_DIR/.interfold}"
+# Interfold v0.2.3 resolves a relative `key_file: key` beside a discovered
 # /data/config.yaml to this path for the default node profile.
-PASSWORD_FILE="${PASSWORD_FILE:-$CONFIG_DIR/.enclave/config/_default/key}"
+PASSWORD_FILE="${PASSWORD_FILE:-$CURRENT_STATE_DIR/config/_default/key}"
 
 log() { printf '[%s] %s\n' "$(date '+%H:%M:%S')" "$1"; }
 fail() {
@@ -36,6 +38,28 @@ fi
 [ -r "$TEMPLATE_FILE" ] || fail "configuration template is not readable: $TEMPLATE_FILE"
 mkdir -p "$CONFIG_DIR"
 
+migrate_legacy_state() {
+    if [ -L "$LEGACY_STATE_DIR" ] || [ -L "$CURRENT_STATE_DIR" ]; then
+        fail "legacy/current state paths must not be symbolic links"
+    fi
+    if [ -e "$LEGACY_STATE_DIR" ] && [ ! -d "$LEGACY_STATE_DIR" ]; then
+        fail "legacy state path is not a directory: $LEGACY_STATE_DIR"
+    fi
+    if [ -e "$CURRENT_STATE_DIR" ] && [ ! -d "$CURRENT_STATE_DIR" ]; then
+        fail "current state path is not a directory: $CURRENT_STATE_DIR"
+    fi
+    if [ -d "$LEGACY_STATE_DIR" ] && [ -e "$CURRENT_STATE_DIR" ]; then
+        fail "both legacy and current state directories exist; refusing an ambiguous upgrade"
+    fi
+    if [ -d "$LEGACY_STATE_DIR" ]; then
+        log "Migrating the v0.1.8 state namespace to Interfold v0.2.3..."
+        mv -- "$LEGACY_STATE_DIR" "$CURRENT_STATE_DIR" \
+            || fail "could not migrate legacy state into $CURRENT_STATE_DIR"
+    fi
+}
+
+migrate_legacy_state
+
 # Set non-secret defaults.
 export NETWORK="${NETWORK:-sepolia}"
 export QUIC_PORT="${QUIC_PORT:-37173}"
@@ -64,12 +88,14 @@ validate_secret_file() {
 
     jq -e '
         type == "object" and
-        (keys | sort == ["network_private_key", "password", "private_key"]) and
+        ((keys | sort == ["password", "private_key"]) or
+            (keys | sort == ["network_private_key", "password", "private_key"])) and
         (.password | type == "string" and length > 0 and length <= 1024 and
             test("^[^\\r\\n\\u0000]+$") and . == gsub("^\\s+|\\s+$"; "")) and
         (.private_key | type == "string" and test("^0x[0-9a-fA-F]{64}$")) and
-        (.network_private_key | type == "string" and test("^0x[0-9a-fA-F]{64}$"))
-    ' "$SECRETS_FILE" >/dev/null || fail "credentials file must contain valid password, private_key, and network_private_key strings"
+        ((has("network_private_key") | not) or
+            (.network_private_key | type == "string" and test("^0x[0-9a-fA-F]{64}$")))
+    ' "$SECRETS_FILE" >/dev/null || fail "credentials file must contain valid password and private_key strings"
 }
 
 validate_persisted_password_file() {
@@ -82,25 +108,24 @@ validate_persisted_password_file() {
 configure_credentials() {
     validate_secret_file
 
-    [ -r "$CREDENTIAL_PROVISIONER" ] || fail "credential provisioner is not readable: $CREDENTIAL_PROVISIONER"
-
-    local provisioning_mode=new
-
     if [ -e "$PASSWORD_FILE" ]; then
         validate_persisted_password_file
         jq -er '.password' "$SECRETS_FILE" | tr -d '\n' | cmp -s - "$PASSWORD_FILE" \
             || fail "uploaded password does not match the persisted credential key"
-        provisioning_mode=existing
         log "Using the matching persisted encryption password."
+        rm -f "$SECRETS_FILE"
+        log "Existing encrypted wallet/network identity was preserved."
+        return
     fi
 
+    [ -r "$CREDENTIAL_PROVISIONER" ] || fail "credential provisioner is not readable: $CREDENTIAL_PROVISIONER"
     log "Provisioning encrypted credentials through hidden stdin prompts..."
-    jq -jr '[.password, .network_private_key, .private_key][] | @base64 + "\n"' "$SECRETS_FILE" \
-        | expect "$CREDENTIAL_PROVISIONER" "$CONFIG_FILE" "$provisioning_mode" \
+    jq -jr '[.password, .private_key][] | @base64 + "\n"' "$SECRETS_FILE" \
+        | expect "$CREDENTIAL_PROVISIONER" "$CONFIG_FILE" \
         || fail "one or more credential commands failed"
 
     # DAppNode copies fileUpload content into this container before startup.
-    # Wallet/network keys are encrypted in /data and v0.1.8 stores the password
+    # Wallet/network keys are encrypted in /data and v0.2.3 stores the password
     # key there with mode 0400. Remove the combined plaintext upload.
     rm -f "$SECRETS_FILE"
     log "Credential setup completed."

dappnode/healthcheck.sh

@@ -1,5 +1,5 @@
 #!/bin/sh
-# Best available local health signal for the v0.1.8 image. That release has no
+# Best available local health signal for the v0.2.3 image. That release has no
 # readiness HTTP/control endpoint, so require the expected PID 1, its exact
 # start/config arguments, protected local credential/config files, and the QUIC
 # listener. This is stronger than PID-only liveness but is not a chain-sync or
@@ -8,9 +8,9 @@ set -eu
 
 PROC_ROOT="${PROC_ROOT:-/proc}"
 CONFIG_FILE="${CONFIG_FILE:-/data/config.yaml}"
-PASSWORD_FILE="${PASSWORD_FILE:-/data/.enclave/config/_default/key}"
-DB_PATH="${DB_PATH:-/data/.enclave/data/_default/db}"
-EVENT_LOG_PATH="${EVENT_LOG_PATH:-/data/.enclave/data/_default/log.0}"
+PASSWORD_FILE="${PASSWORD_FILE:-/data/.interfold/config/_default/key}"
+DB_PATH="${DB_PATH:-/data/.interfold/data/_default/db}"
+EVENT_LOG_PATH="${EVENT_LOG_PATH:-/data/.interfold/data/_default/log.0}"
 QUIC_PORT="${QUIC_PORT:-37173}"
 SS_BIN="${SS_BIN:-ss}"
 STAT_BIN="${STAT_BIN:-stat}"

dappnode/provision-credentials.exp

@@ -1,6 +1,6 @@
 #!/usr/bin/expect -f
 # Feed credentials to the hidden interactive prompts exposed by Interfold
-# v0.1.8. Secrets arrive as base64-encoded lines on stdin; neither the
+# v0.2.3. Secrets arrive as base64-encoded lines on stdin; neither the
 # plaintext nor the encoded value is placed in argv or the environment.
 
 set timeout 30
@@ -55,32 +55,20 @@ proc send_at_prompt {pattern secret description} {
     }
 }
 
-if {$argc != 2} {
-    fail "expected CONFIG_FILE and provisioning mode"
+if {$argc != 1} {
+    fail "expected CONFIG_FILE"
 }
 set config_file [lindex $argv 0]
-set mode [lindex $argv 1]
-if {$mode ne "new" && $mode ne "existing"} {
-    fail "provisioning mode must be new or existing"
-}
 
 set password [read_secret "password"]
-set network_private_key [read_secret "network private key"]
 set private_key [read_secret "wallet private key"]
 
-if {$mode eq "new"} {
-    spawn -noecho interfold password set --config $config_file
-    send_at_prompt {Please enter a new password} $password "new password"
-    send_at_prompt {Please confirm your password} $password "password confirmation"
-    finish_child "password command"
-}
+spawn -noecho interfold password set --config $config_file
+send_at_prompt {Please enter a new password} $password "new password"
+send_at_prompt {Please confirm your password} $password "password confirmation"
+finish_child "password command"
 unset password
 
-spawn -noecho interfold net keypair set --config $config_file
-send_at_prompt {Enter your network private key} $network_private_key "network private key"
-finish_child "network key command"
-unset network_private_key
-
 spawn -noecho interfold wallet set --config $config_file
 send_at_prompt {Enter your Ethereum private key} $private_key "wallet private key"
 finish_child "wallet command"

dappnode/setup-wizard.yml

@@ -135,8 +135,7 @@ fields:
       ```json
       {
         "password": "a strong encryption password",
-        "private_key": "0x<64 hex characters>",
-        "network_private_key": "0x<64 hex characters>"
+        "private_key": "0x<64 hex characters>"
       }
       ```