Skip to content

"getValue() method is redundant, use fromMetaWalEntry instead" - #3997

Merged
ctubbsii merged 12 commits into
apache:mainfrom
rsingh433:RemoveGetValue3943
Dec 12, 2023
Merged

"getValue() method is redundant, use fromMetaWalEntry instead"#3997
ctubbsii merged 12 commits into
apache:mainfrom
rsingh433:RemoveGetValue3943

Conversation

@rsingh433

Copy link
Copy Markdown
Contributor

getValue() method is redundant, the same value can be obtained using the fromMetaWalEntry method.

@keith-turner keith-turner left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the test changes, thinking those should not change. If those test are failing let me know and I would be happy to take a look and see what is happening.

Comment thread test/src/main/java/org/apache/accumulo/test/manager/MergeStateIT.java Outdated
@ctubbsii ctubbsii linked an issue Dec 6, 2023 that may be closed by this pull request

@ctubbsii ctubbsii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At a glance, these changes look good to me, but I'm not sure I understand why the test is failing. Or rather, I'm not sure how the test was passing before.

One thing I saw was that the test can be simplified by using a new SimpleImmutableEntry(key, entry.getValue()); rather than creating an anonymous subclass of Entry on lines 60-75.

Another issue is that the LogEntry.validateFilePath method (where this is failing) seems to have an out-of-date concept of the format of the file path. I don't think it's correct.

Given the out of date format that is being validated in validateFilePath method, I see why it's failing (because the test's file name is "foo" instead of a UUID), but I don't understand why it wasn't already failing before.

This will need to be tracked down and understood, I think, in order to make progress here.

@ctubbsii

ctubbsii commented Dec 7, 2023

Copy link
Copy Markdown
Member

@keith-turner helped me understand the test failures (or rather, why the current test is passing in the main branch when it probably shouldn't), and I opened up #4035 to help with that. Once that is merged, I'll help resolve the conflicts with this PR, so we can merge this in after that.

@ctubbsii

ctubbsii commented Dec 7, 2023

Copy link
Copy Markdown
Member

These changes look good, as-is, and can be merged now to drop storing the "filePath" in the Value and reading it from the column qualifier instead. However, there's some follow-on actions that you can add to this PR, or can create a separate PR for, at your discretion. Here's some things I saw while looking at this code, where there's room for improvement:

  1. Pull in the mutation modification work into the LogEntry class. So, instead of calling code calling in to LogEntry to get the column qualifier, value, etc. to update a mutation, you just pass a mutation into the LogEntry class like: logEntry.addToMutation(mutation);, and the mutation updating is done internal to that method. That way, the column qualifier, value, etc. doesn't need to be used outside the class (except maybe for unit testing) and they can be made private, or even simplified or removed.
  2. Get rid of switchFile method, since that's outdated, and simplify how VolumeUtil is using it
  3. Rename "filePath", "filename", and "getFilePath()", etc. to something that reflects what is actually stored there. From what I can tell, what's stored there is a "-/host:port/UUID" (with "-/" ignored). So, maybe a better name is "logReference", since the host and UUID together could be called that. Or something like "serverAndUUID", but that's not really very friendly. If there's a better name, I'm open to that as well.
  4. Make the validate method more rigorous and expect exactly 3 items ("-/host:port/UUID" -> {"-", "host:port", "UUID"}. Right now validates either 2 or 3... and it's not clear why. Probably so it could validate the old value, which did not contain the "-/" portion, as well as the column qualifier, which did. But it doesn't need to validate the 2-item version, since that shouldn't exist
  5. Add upgrade code to drop the "-/" portion entirely, and further simplify all the code that tries to handle that, so it's only "host:port/UUID" instead in the column qualifier.

Some of these are certainly more complicated than others, and would require a separate PR. But some of these are small enough to include with this.

@rsingh433

Copy link
Copy Markdown
Contributor Author

I removed the switchFile method and updated the switchVolumes method to directly create a new LogEntry instance with the switched file path. Now, the LogEntry class itself is responsible for creating instances based on the file path. Also, I added the functionality to add LogEntry information to a mutation by calling logEntry.addToMutation(mutation).

Comment thread core/src/main/java/org/apache/accumulo/core/tabletserver/log/LogEntry.java Outdated
* Remove getValue method (inline the only use needed and update test)
* Rename variable in fromMetaWalEntry method to be less confusing and
  style the method by removing blank lines
* Revert and simplify changes to VolumeUtil
* Use LogEntry.addToMutation in one more place
@ctubbsii
ctubbsii merged commit 6cfa1b2 into apache:main Dec 12, 2023
asfgit pushed a commit that referenced this pull request Dec 13, 2023
This issue caused #4065 and needs to be reverted and redone. The
explanation is the misunderstanding of the LogEntry schema at
#4058 (comment)

This reverts commit 6cfa1b2.
ctubbsii added a commit to ctubbsii/accumulo that referenced this pull request Dec 14, 2023
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata
table in MetaDataTableScanner. It previously implemented a manual
parsing of a very old and outdated schema that just happened to still
work with the current format.

This fixes apache#4065 and will be compatible with the changes in PR apache#3997.
ctubbsii added a commit to ctubbsii/accumulo that referenced this pull request Dec 14, 2023
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata
table in MetaDataTableScanner. It previously implemented a manual
parsing of a very old and outdated schema that just happened to still
work with the current format.

This fixes apache#4065 and will be compatible with the changes in PR apache#3997.
ctubbsii added a commit that referenced this pull request Dec 14, 2023
Use LogEntry.fromMetaWalEntry to parse log entries from the metadata
table in MetaDataTableScanner. It previously implemented a manual
parsing of a very old and outdated schema that just happened to still
work with the current format.

This fixes #4065 and will be compatible with the changes in PR #3997.
ctubbsii added a commit to ctubbsii/accumulo that referenced this pull request Dec 14, 2023
Remove the `LogEntry.getValue()` method and use the column qualifier
for the file path, without the redundant storage in the Value

* Remove `getValue()` method and use empty `new Value()`
* Create addToMutation method to remove boilerplate for storing
  the LogEntry in the metadata
* Strictly validate the metadata log entries' column qualifier

This re-applies the reverted change for this ticket,
0cdf6dd, but without the renaming of
filePath to logReference, since with further analysis of the schema, it
was discovered that these are, in fact, file paths, and not just generic
references to log IDs.

This re-application also removes the metadata constraint that expected
the value to be non-empty, which was overlooked on the first attempt.

---------

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
ctubbsii added a commit to ctubbsii/accumulo that referenced this pull request Dec 14, 2023
Remove the `LogEntry.getValue()` method and use the column qualifier
for the file path, without the redundant storage in the Value

* Remove `getValue()` method and use empty `new Value()`
* Create addToMutation method to remove boilerplate for storing
  the LogEntry in the metadata
* Strictly validate the metadata log entries' column qualifier

This re-applies the reverted change for apache#3997,
0cdf6dd, but without the renaming of
filePath to logReference, since with further analysis of the schema, it
was discovered that these are, in fact, file paths, and not just generic
references to log IDs.

This re-application also removes the metadata constraint that expected
the value to be non-empty, which was overlooked on the first attempt.

---------

Co-authored-by: Christopher Tubbs <ctubbsii@apache.org>
@rsingh433
rsingh433 deleted the RemoveGetValue3943 branch December 14, 2023 17:04
@ctubbsii ctubbsii added this to the 3.1.0 milestone Jul 12, 2024
@ctubbsii ctubbsii modified the milestones: 3.1.0, 4.0.0 Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate removing getValue() from LogEntry class

4 participants