Support TLog encryption in commit proxy

[sfc-gh-yiwu]

This PR add support for TLog encryption through commit proxy. The encryption is done on per-mutation basis. As CP writes mutations to TLog, it inserts encryption header alongside encrypted mutations. Storage server (and other consumers of TLog such as storage cache and backup worker) decrypts the mutations as they peek TLog.

Code-Reviewer Section

The general guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

• The PR has a description, explaining both the problem and the solution.
• The description mentions which forms of testing were done and the testing seems reasonable.
• Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

• This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
• There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 995d708
• Result: FAILED
• Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[sfc-gh-yiwu]

Send PR out for preview. There are still much work to be done.

• Simulation still failing for key_not_found error, which probably due to incorrect cipher key fetching logic
• Currently in commit proxy, cipher keys are being fetch per mutation. Should probably batch cipher key for each commit batch, make it concurrent with resolver requests. Similar change also needed for storage server side logic.
• BackupWorker and RestoreLoaders currently throw exception if they encounter encrypted mutation, but it will fail simulations. Need to fix.
• Need to add new tests

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 995d708
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: ed0ae74
• Result: FAILED
• Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for macOS BigSur 11.5.2

• CodeBuild project: foundationdb-pr-macos
• Commit ID: ed0ae74
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: ed0ae74
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 7e6d7a7
• Result: FAILED
• Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 7e6d7a7
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: c89278e
• Result: FAILED
• Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for macOS BigSur 11.5.2

• CodeBuild project: foundationdb-pr-macos
• Commit ID: c89278e
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: c89278e
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for macOS BigSur 11.5.2

• CodeBuild project: foundationdb-pr-macos
• Commit ID: 3a2f697
• Result: FAILED
• Error: Error while executing command: ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ${HOME}/.ssh_key ec2-user@${MAC_EC2_HOST} /usr/local/bin/bash --login -c ./build_pr_macos.sh. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 40b8f3f
• Result: FAILED
• Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 3a2f697
• Result: FAILED
• Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 3a2f697
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[sfc-gh-yiwu]

Linux CI failing on the following test but I couldn't reproduce locally. Rebase and see.

bin/fdbserver -r simulation -f tests/fast/SwizzledRollbackSideband.toml -s 334565662 -b on  --crash

[sfc-gh-xwang]

Linux CI failing on the following test but I couldn't reproduce locally. Rebase and see.

bin/fdbserver -r simulation -f tests/fast/SwizzledRollbackSideband.toml -s 334565662 -b on  --crash

You need to use gcc to reproduce CI failure.

[sfc-gh-yiwu]

You need to use gcc to reproduce CI failure.

@sfc-gh-xwang Thanks a lot!

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 7038ba7
• Result: FAILED
• Error: Error while executing command: if python3 -m joshua.joshua list --stopped | grep ${ENSEMBLE_ID} | grep -q 'pass=10[0-9][0-9][0-9]'; then echo PASS; else echo FAIL && exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 7038ba7
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for Linux CentOS 7

• CodeBuild project: foundationdb-pr
• Commit ID: 4bfc795
• Result: FAILED
• Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 4bfc795
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

AWS CodeBuild CI Report for macOS BigSur 11.5.2

• CodeBuild project: foundationdb-pr-macos
• Commit ID: 4bfc795
• Result: SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[sfc-gh-yiwu]

@sfc-gh-ahusain updated.

[foundationdb-ci]

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

• Commit ID: 8c163fe
• Duration 0:06:59
• Result: ❌ FAILED
• Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr on Linux CentOS 7

• Commit ID: 8c163fe
• Duration 0:07:02
• Result: ❌ FAILED
• Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 8c163fe
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[sfc-gh-ahusain]

LGTM. Thanks for doing this

[foundationdb-ci]

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

• Commit ID: 8c163fe
• Duration 0:39:47
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr on Linux CentOS 7

• Commit ID: 4f4fb5e
• Duration 0:07:05
• Result: ❌ FAILED
• Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
• Build Logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

• Commit ID: 4f4fb5e
• Duration 0:07:09
• Result: ❌ FAILED
• Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
• Build Logs (available for 30 days)

[fdb-windows-ci]

Doxense CI Report for Windows 10

• Commit ID: 4f4fb5e
• Result: ✔️ SUCCEEDED
• Build Logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr-macos on macOS BigSur 11.5.2

• Commit ID: 4f4fb5e
• Duration 0:38:55
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[sfc-gh-yiwu]

Build is failing due to #7442. Will rebase after that PR is merged.

[sfc-gh-yiwu]

Build failure is due to #7442. Will wait for that PR and rebase.

[foundationdb-ci]

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

• Commit ID: aca12ac
• Duration 0:41:45
• Result: ❌ FAILED
• Error: Error while executing command: tar -czf "${CODEBUILD_SRC_DIR}/packaging/docker/website/${FDB_VERSION}/fdb_${FDB_VERSION}.tar.gz" --directory "${CODEBUILD_SRC_DIR}/packaging/docker/website/${FDB_VERSION} .. Reason: exit status 2
• Build Logs (available for 30 days)

[foundationdb-ci]

Result of foundationdb-pr on Linux CentOS 7

• Commit ID: aca12ac
• Duration 1:05:59
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Logs (available for 30 days)

[sfc-gh-ahusain]

LGTM