"getValue() method is redundant, use fromMetaWalEntry instead" - #3997
Conversation
keith-turner
left a comment
There was a problem hiding this comment.
For the test changes, thinking those should not change. If those test are failing let me know and I would be happy to take a look and see what is happening.
ctubbsii
left a comment
There was a problem hiding this comment.
At a glance, these changes look good to me, but I'm not sure I understand why the test is failing. Or rather, I'm not sure how the test was passing before.
One thing I saw was that the test can be simplified by using a new SimpleImmutableEntry(key, entry.getValue()); rather than creating an anonymous subclass of Entry on lines 60-75.
Another issue is that the LogEntry.validateFilePath method (where this is failing) seems to have an out-of-date concept of the format of the file path. I don't think it's correct.
Given the out of date format that is being validated in validateFilePath method, I see why it's failing (because the test's file name is "foo" instead of a UUID), but I don't understand why it wasn't already failing before.
This will need to be tracked down and understood, I think, in order to make progress here.
|
@keith-turner helped me understand the test failures (or rather, why the current test is passing in the main branch when it probably shouldn't), and I opened up #4035 to help with that. Once that is merged, I'll help resolve the conflicts with this PR, so we can merge this in after that. |
|
These changes look good, as-is, and can be merged now to drop storing the "filePath" in the Value and reading it from the column qualifier instead. However, there's some follow-on actions that you can add to this PR, or can create a separate PR for, at your discretion. Here's some things I saw while looking at this code, where there's room for improvement:
Some of these are certainly more complicated than others, and would require a separate PR. But some of these are small enough to include with this. |
|
I removed the switchFile method and updated the switchVolumes method to directly create a new LogEntry instance with the switched file path. Now, the LogEntry class itself is responsible for creating instances based on the file path. Also, I added the functionality to add LogEntry information to a mutation by calling logEntry.addToMutation(mutation). |
* Remove getValue method (inline the only use needed and update test) * Rename variable in fromMetaWalEntry method to be less confusing and style the method by removing blank lines * Revert and simplify changes to VolumeUtil * Use LogEntry.addToMutation in one more place
This issue caused #4065 and needs to be reverted and redone. The explanation is the misunderstanding of the LogEntry schema at #4058 (comment) This reverts commit 6cfa1b2.
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata table in MetaDataTableScanner. It previously implemented a manual parsing of a very old and outdated schema that just happened to still work with the current format. This fixes apache#4065 and will be compatible with the changes in PR apache#3997.
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata table in MetaDataTableScanner. It previously implemented a manual parsing of a very old and outdated schema that just happened to still work with the current format. This fixes apache#4065 and will be compatible with the changes in PR apache#3997.
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata table in MetaDataTableScanner. It previously implemented a manual parsing of a very old and outdated schema that just happened to still work with the current format. This fixes #4065 and will be compatible with the changes in PR #3997.
Remove the `LogEntry.getValue()` method and use the column qualifier for the file path, without the redundant storage in the Value * Remove `getValue()` method and use empty `new Value()` * Create addToMutation method to remove boilerplate for storing the LogEntry in the metadata * Strictly validate the metadata log entries' column qualifier This re-applies the reverted change for this ticket, 0cdf6dd, but without the renaming of filePath to logReference, since with further analysis of the schema, it was discovered that these are, in fact, file paths, and not just generic references to log IDs. This re-application also removes the metadata constraint that expected the value to be non-empty, which was overlooked on the first attempt. --------- Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
Remove the `LogEntry.getValue()` method and use the column qualifier for the file path, without the redundant storage in the Value * Remove `getValue()` method and use empty `new Value()` * Create addToMutation method to remove boilerplate for storing the LogEntry in the metadata * Strictly validate the metadata log entries' column qualifier This re-applies the reverted change for apache#3997, 0cdf6dd, but without the renaming of filePath to logReference, since with further analysis of the schema, it was discovered that these are, in fact, file paths, and not just generic references to log IDs. This re-application also removes the metadata constraint that expected the value to be non-empty, which was overlooked on the first attempt. --------- Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
getValue() method is redundant, the same value can be obtained using the fromMetaWalEntry method.