Use KeyspaceSnapshotFile to filter range files by jzhou77 · Pull Request #9511 · apple/foundationdb

jzhou77 · 2023-02-28T02:07:08Z

Replace this text with your description here...

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

The PR has a description, explaining both the problem and the solution.
The description mentions which forms of testing were done and the testing seems reasonable.
Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

halfprice · 2023-02-28T02:16:15Z

+				if (params->matchFilters(keyRange)) {
+					files.push_back(rangeFile);
+				}
+			}


Break from the loop when there is no error?

No. If there are two snapshots within the version range, we should gather all files for both. The filtering based on version is at line 858.

How will the range files be used? Say there are multiple range files for the same key within the interested beginVersion and endVersion?

It's up to the user to decide that. This tool just output all matching (version, key, value) tuples.

foundationdb-ci · 2023-02-28T02:35:46Z

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Commit ID: 5737512
Duration 0:28:28
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T02:40:02Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 5737512
Duration 0:32:42
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T02:43:50Z

Result of foundationdb-pr-macos on macOS Monterey 12.x

Commit ID: 5737512
Duration 0:36:30
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T03:09:58Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 5737512
Duration 1:02:41
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T03:29:28Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 5737512
Duration 1:22:07
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

fdb-windows-ci · 2023-02-28T03:41:10Z

Doxense CI Report for Windows 10

Commit ID: 5737512
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

halfprice · 2023-02-28T04:01:25Z

+				if (params->matchFilters(keyRange)) {
+					files.push_back(rangeFile);
+				}
+			}


How will the range files be used? Say there are multiple range files for the same key within the interested beginVersion and endVersion?

Set max length as well to avoid TraceEventOverflow.

foundationdb-ci · 2023-02-28T18:24:17Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: c12e0f1
Duration 0:24:39
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T18:28:25Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 267c9c2
Duration 1:03:09
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T18:29:12Z

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Commit ID: c12e0f1
Duration 0:29:33
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T18:36:16Z

Result of foundationdb-pr-macos on macOS Monterey 12.x

Commit ID: c12e0f1
Duration 0:36:39
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

fdb-windows-ci · 2023-02-28T18:36:36Z

Doxense CI Report for Windows 10

Commit ID: c12e0f1
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2023-02-28T18:56:15Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: c12e0f1
Duration 0:56:38
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T19:31:51Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 267c9c2
Duration 2:06:35
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T20:25:45Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 267c9c2
Duration 3:00:34
Result: ❌ FAILED
Error: Build has timed out.
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-02-28T21:00:10Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: c12e0f1
Duration 3:00:32
Result: ❌ FAILED
Error: Build has timed out.
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-02-28T21:45:57Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 23702f1
Duration 0:03:38
Result: ❌ FAILED
Error: AccessDenied: Access Denied status code: 403, request id: BV2056ZCTYFES05B, host id: hGh+fu/4R/D906UEQpX5BM9IJaLjp7prtzxuWIwkSLe+EtgLS57URa3eS/KGyIJOGVCOi/r5vGE=
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T21:46:00Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 23702f1
Duration 0:03:41
Result: ❌ FAILED
Error: AccessDenied: Access Denied status code: 403, request id: 3P43412ZDAD8Z1P8, host id: a+mI+Ea49H3cXGk4pehBArok0fpaCfQbpZVLNy+0TS7PRBlfCPtzFJx7UXHh/7kYUZH5kcXI7lw=
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-02-28T21:46:00Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 23702f1
Duration 0:03:41
Result: ❌ FAILED
Error: AccessDenied: Access Denied status code: 403, request id: 3P409CZ3D9XSKNHQ, host id: OEWBeNOEUk0UR/ilORZvIENQsUXKwPaGioVWGnsI0BAue1SZDRo/lI2IMb6RdhfRiNnvxq6o6GM=
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T22:24:02Z

Result of foundationdb-pr-macos on macOS Monterey 12.x

Commit ID: f89610e
Duration 0:00:13
Result: ❌ FAILED
Error: Unexpected EC2 error: InvalidGroup.NotFound
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T22:24:06Z

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Commit ID: f89610e
Duration 0:00:16
Result: ❌ FAILED
Error: Unexpected EC2 error: InvalidGroup.NotFound
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T23:17:33Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: f89610e
Duration 0:53:40
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-28T23:45:10Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: f89610e
Duration 1:21:18
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-01T00:26:16Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: f89610e
Duration 2:02:24
Result: ❌ FAILED
Error: Error while executing command: ninja -v -C build_output -j ${NPROC} all packages strip_targets. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

fdb-windows-ci · 2023-03-01T04:51:19Z

Doxense CI Report for Windows 10

Commit ID: f89610e
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

Use KeyspaceSnapshotFile to filter range files

5737512

jzhou77 requested review from halfprice and vishesh February 28, 2023 02:07

halfprice reviewed Feb 28, 2023

View reviewed changes

halfprice previously approved these changes Feb 28, 2023

View reviewed changes

jzhou77 dismissed halfprice’s stale review via 267c9c2 February 28, 2023 17:25

jzhou77 requested a review from halfprice February 28, 2023 17:46

halfprice previously approved these changes Feb 28, 2023

View reviewed changes

Change mutation and KV logging to SevInfo

c12e0f1

Set max length as well to avoid TraceEventOverflow.

jzhou77 dismissed halfprice’s stale review via c12e0f1 February 28, 2023 17:59

jzhou77 force-pushed the restore branch from 267c9c2 to c12e0f1 Compare February 28, 2023 17:59

Output in HEX format for easy regex matching

23702f1

Merge branch 'release-7.1' into restore

f89610e

jzhou77 requested a review from halfprice March 1, 2023 17:10

halfprice approved these changes Mar 1, 2023

View reviewed changes

jzhou77 merged commit bf9c642 into apple:release-7.1 Mar 1, 2023

Uh oh!

Conversation

jzhou77 commented Feb 28, 2023

Code-Reviewer Section

For Release-Branches

Uh oh!

halfprice Feb 28, 2023

Choose a reason for hiding this comment

Uh oh!

jzhou77 Feb 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

halfprice Feb 28, 2023

Choose a reason for hiding this comment

Uh oh!

jzhou77 Feb 28, 2023

Choose a reason for hiding this comment

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-macos on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

fdb-windows-ci commented Feb 28, 2023

Doxense CI Report for Windows 10

Uh oh!

halfprice Feb 28, 2023

Choose a reason for hiding this comment

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-macos on macOS Monterey 12.x

Uh oh!

fdb-windows-ci commented Feb 28, 2023

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 28, 2023

Result of foundationdb-pr-macos on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 28, 2023

jzhou77 Feb 28, 2023 •

edited

Loading