Extend Tlog persistentStorage to persist encryption state by sfc-gh-ahusain · Pull Request #8344 · apple/foundationdb

sfc-gh-ahusain · 2022-09-27T20:53:26Z

Description

Major changes proposed are:
1. Extend TLog persistentStorage to persist encryption state
2. Encryption state persisted is derived from corresponding
db-config and relevant SERVER_KNOBS. In near future, knobs
shall be removed.
3. On TLog startup, the persisted encryption state is compared
against cluster configuration, if mismatch, the TLog is killed
and not allowed to rejoin the cluster.

Testing

devRunCorrectness - 100K

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

The PR has a description, explaining both the problem and the solution.
The description mentions which forms of testing were done and the testing seems reasonable.
Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

fdb-windows-ci · 2022-09-27T21:44:21Z

Doxense CI Report for Windows 10

Commit ID: 5c7df6a
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-09-27T23:44:52Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 5c7df6a
Duration 2:51:18
Result: ❌ FAILED
Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-28T00:56:30Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 5c7df6a
Duration 4:02:55
Result: ❌ FAILED
Error: Build has timed out.
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-28T16:35:15Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: f2eab3a
Duration 0:05:23
Result: ❌ FAILED
Error: Error while executing command: aws eks --region us-west-2 update-kubeconfig --name foundationdb-dev --role-arn ${EKS_ROLE}. Reason: exit status 254
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-28T16:35:39Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: f2eab3a
Duration 0:05:47
Result: ❌ FAILED
Error: Error while executing command: aws eks --region us-west-2 update-kubeconfig --name foundationdb-dev --role-arn ${EKS_ROLE}. Reason: exit status 254
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-28T16:36:20Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: f2eab3a
Duration 0:06:31
Result: ❌ FAILED
Error: Error while executing command: aws eks --region us-west-2 update-kubeconfig --name foundationdb-chaos-0 --role-arn ${EKS_ROLE}. Reason: exit status 255
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

fdb-windows-ci · 2022-09-28T17:15:34Z

Doxense CI Report for Windows 10

Commit ID: f2eab3a
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-09-28T17:21:16Z

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Commit ID: f2eab3a
Duration 0:51:24
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T05:20:29Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 341672e
Duration 0:04:44
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T05:20:40Z

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Commit ID: 341672e
Duration 0:04:52
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T05:20:51Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 341672e
Duration 0:05:06
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T05:20:53Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 341672e
Duration 0:05:05
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T05:42:55Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 52bc0fa
Duration 0:16:56
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

fdb-windows-ci · 2022-09-29T05:45:16Z

Doxense CI Report for Windows 10

Commit ID: 341672e
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-09-29T06:14:57Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 52bc0fa
Duration 0:49:03
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-09-29T06:17:19Z

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Commit ID: 52bc0fa
Duration 0:51:20
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

fdb-windows-ci · 2022-09-29T06:23:50Z

Doxense CI Report for Windows 10

Commit ID: 52bc0fa
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-11-02T22:50:03Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: f3a2013
Duration 1:18:56
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-02T23:30:25Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: f3a2013
Duration 1:59:17
Result: ❌ FAILED
Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T06:23:49Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 3da1211
Duration 0:16:18
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T06:27:40Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 3da1211
Duration 0:20:07
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

fdb-windows-ci · 2022-11-03T06:57:07Z

Doxense CI Report for Windows 10

Commit ID: 3da1211
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-11-03T07:07:28Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 3da1211
Duration 1:00:01
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T08:05:09Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 3da1211
Duration 1:57:36
Result: ❌ FAILED
Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T14:21:01Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: cbbf1f0
Duration 0:16:42
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T14:41:56Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: cbbf1f0
Duration 0:37:36
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T14:43:27Z

Result of foundationdb-pr-macos-m1 on macOS BigSur 11.5.2

Commit ID: cbbf1f0
Duration 0:39:08
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T14:44:46Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: cbbf1f0
Duration 0:40:27
Result: ❌ FAILED
Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T14:48:14Z

Result of foundationdb-pr-macos on macOS Monterey 12.x

Commit ID: cbbf1f0
Duration 0:43:52
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

fdb-windows-ci · 2022-11-03T15:00:06Z

Doxense CI Report for Windows 10

Commit ID: cbbf1f0
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-11-03T16:03:02Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: cbbf1f0
Duration 1:58:40
Result: ❌ FAILED
Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

Description Major changes proposed are: 1. Extend TLog persistentStorage to persist encryption state 2. Encryption state persisted is derived from correspoding db-config and relevant SERVER_KNOBS. In near future, knobs shall be removed. 3. On TLog startup, the persisted encryption state is compared against cluster configuration, if mismatch, the TLog is killed and not allowed to rejoin the cluster. Testing devRunCorrectness - 100K

Description diff-1: Store encryptionAtRestMode state in Coordinators Major changes proposed are: 1. Extend TLog persistentStorage to persist encryption state 2. Encryption state persisted is derived from correspoding db-config and relevant SERVER_KNOBS. In near future, knobs shall be removed. 3. On TLog startup, the persisted encryption state is compared against cluster configuration, if mismatch, the TLog is killed and not allowed to rejoin the cluster. Testing devRunCorrectness - 100K

Description diff-2: Extend ClusterController endpoints to allow query cluster's encryptionAtRest status Update Tlog recovery to ensure on-disk encryption status matches with cluster's cstate persisted encryptionAtRest diff-1: Store encryptionAtRestMode state in Coordinators Major changes proposed are: 1. Extend TLog persistentStorage to persist encryption state 2. Encryption state persisted is derived from correspoding db-config and relevant SERVER_KNOBS. In near future, knobs shall be removed. 3. On TLog startup, the persisted encryption state is compared against cluster configuration, if mismatch, the TLog is killed and not allowed to rejoin the cluster. Testing devRunCorrectness - 100K

Description diff-3: Address review comment. diff-2: Extend ClusterController endpoints to allow query cluster's encryptionAtRest status Update Tlog recovery to ensure on-disk encryption status matches with cluster's cstate persisted encryptionAtRest diff-1: Store encryptionAtRestMode state in Coordinators Major changes proposed are: 1. Extend TLog persistentStorage to persist encryption state 2. Encryption state persisted is derived from correspoding db-config and relevant SERVER_KNOBS. In near future, knobs shall be removed. 3. On TLog startup, the persisted encryption state is compared against cluster configuration, if mismatch, the TLog is killed and not allowed to rejoin the cluster. Testing devRunCorrectness - 100K

fdb-windows-ci · 2022-11-03T16:35:20Z

Doxense CI Report for Windows 10

Commit ID: 8913bde
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2022-11-03T16:36:21Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 8913bde
Duration 0:16:41
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T16:58:16Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 8913bde
Duration 0:38:33
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T17:01:56Z

Result of foundationdb-pr-macos-m1 on macOS BigSur 11.5.2

Commit ID: 8913bde
Duration 0:42:13
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T17:03:59Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 8913bde
Duration 0:44:17
Result: ✅ SUCCEEDED
Error: N/A
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

foundationdb-ci · 2022-11-03T18:18:32Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 8913bde
Duration 1:58:51
Result: ❌ FAILED
Error: Error while executing command: if $(grep -q -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log); then echo "TESTS FAILED SEE THESE LOGS:"; echo ; grep -l -- "--- FAIL:" ${CODEBUILD_SRC_DIR}/fdb-kubernetes-tests/logs/*.log; exit 1; fi. Reason: exit status 1
Build Logs (available for 30 days)
Build Artifact (available for 30 days)

sfc-gh-ahusain requested a review from sfc-gh-etschannen September 27, 2022 20:53

sfc-gh-ahusain commented Sep 27, 2022

View reviewed changes

Comment thread fdbserver/TLogServer.actor.cpp Outdated

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from 5c7df6a to f2eab3a Compare September 28, 2022 16:29

sfc-gh-ahusain marked this pull request as ready for review September 28, 2022 16:30

sfc-gh-ahusain requested review from sfc-gh-nwijetunga and sfc-gh-yiwu September 28, 2022 16:30

sfc-gh-nwijetunga reviewed Sep 28, 2022

View reviewed changes

Comment thread fdbserver/TLogServer.actor.cpp Outdated

sfc-gh-nwijetunga reviewed Sep 28, 2022

View reviewed changes

Comment thread fdbserver/TLogServer.actor.cpp

jzhou77 reviewed Sep 29, 2022

View reviewed changes

Comment thread fdbserver/TLogServer.actor.cpp Outdated

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from f2eab3a to 341672e Compare September 29, 2022 05:15

sfc-gh-ahusain requested review from jzhou77 and sfc-gh-nwijetunga September 29, 2022 05:22

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from 341672e to 52bc0fa Compare September 29, 2022 05:25

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from f3a2013 to 3da1211 Compare November 3, 2022 06:07

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from 3da1211 to cbbf1f0 Compare November 3, 2022 14:04

sfc-gh-ahusain added 4 commits November 3, 2022 09:15

sfc-gh-ahusain force-pushed the ahusain-tlog-persist branch from cbbf1f0 to 8913bde Compare November 3, 2022 16:16

sfc-gh-ahusain requested a review from sfc-gh-etschannen November 3, 2022 17:05

sfc-gh-etschannen approved these changes Nov 3, 2022

View reviewed changes

sfc-gh-ahusain merged commit a7d1236 into apple:main Nov 3, 2022

Uh oh!

Conversation

sfc-gh-ahusain commented Sep 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code-Reviewer Section

For Release-Branches

Uh oh!

fdb-windows-ci commented Sep 27, 2022

Doxense CI Report for Windows 10

Uh oh!

Uh oh!

foundationdb-ci commented Sep 27, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 28, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 28, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 28, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 28, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

fdb-windows-ci commented Sep 28, 2022

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Sep 28, 2022

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Uh oh!

Uh oh!

Uh oh!

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

fdb-windows-ci commented Sep 29, 2022

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Sep 29, 2022

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

Uh oh!

fdb-windows-ci commented Sep 29, 2022

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Nov 2, 2022

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Nov 3, 2022

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Uh oh!

foundationdb-ci commented Nov 3, 2022

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

fdb-windows-ci commented Nov 3, 2022

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Nov 3, 2022

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

sfc-gh-ahusain commented Sep 27, 2022 •

edited

Loading