HDDS-7741. [Snapshot] Delete keys in snapshot scope from deleteTable during createSnapshot to accommodate snapshot garbage collection#4280
Merged
smengcl merged 11 commits intoapache:masterfrom Mar 16, 2023
Conversation
…during createSnapshot to accommodate snapshot garbage collection
Change-Id: Ibbf5f8c3ed29f4d235af4c75d40c351b55a3cc98
GeorgeJahad
reviewed
Feb 22, 2023
...nager/src/main/java/org/apache/hadoop/ozone/om/request/snapshot/OMSnapshotCreateRequest.java
Outdated
Show resolved
Hide resolved
Change-Id: If40fe3b6397c0f53c5970bd17a0b3ecbadacf2d8
Change-Id: Ide13aee59fc84d785aba5bc1ff735529c48149e3
Change-Id: I478aaff0d112f8f14381dd04aac994afc446e70a
Change-Id: Ida129a5821dedfc5883868ec1597bad014317662
...ontainer-service/src/main/java/org/apache/hadoop/ozone/container/metadata/DatanodeTable.java
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmMetadataManagerImpl.java
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Show resolved
Hide resolved
...ger/src/main/java/org/apache/hadoop/ozone/om/response/snapshot/OMSnapshotCreateResponse.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java
Show resolved
Hide resolved
Change-Id: I8f8dab1473b3e7c65acd2e04ef0ed13367d15082
Conflicts: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OmSnapshotManager.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/snapshot/OMSnapshotCreateResponse.java Change-Id: I7c1bcce86d7a98cd83aa3b63be446ab94a501102
prashantpogde
approved these changes
Mar 10, 2023
Contributor
prashantpogde
left a comment
prashantpogde
left a comment
There was a problem hiding this comment.
thanks @smengcl for making these changes. LGTM
…moved when proper refactor HDDS-5905 (HDDS-6483) is in. Change-Id: I74e5cb7a8a147a78f61dcfc6325fe6ec1f96b517
Contributor
Author
|
Thanks @prashantpogde for the review. I've added a note that the lock is a stopgap solution before the proper fix (HDDS-5905, HDDS-6483) is in. As I talked to @kerneltime last week that it is not ideal to have a lock on a RocksDB table in the first place. |
Contributor
Author
|
Thanks @prashantpogde for the review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
During snapshot creation, use deleteRange to remove the keys in the snapshot scope (bucket) from deletedTable.
deleteRangeinTableclass to expose RocksDBdeleteRangeAPI.deleteRangedeletedTableOMSnapshotCreateRequest#validateAndUpdateCacheWhat is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7741
How was this patch tested?