Use LogEntry concrete type in more places - #4085
Conversation
* Track the actual column qualifier read from the metadata table, so when it is deleted, it can delete the actual entry, rather than try to delete the existing entry with the new format (requires no longer strictly requiring that log entries start with a `-/`). This fixes apache#4061 for the main branch for 3.1; a similar issue may still exist for 2.1, but will require a different fix. * Remove use of `Collection<Collection<String>>` and use `Collection<LogEntry>` where possible, since the former was for a previous WAL storage format that we do not support, and the latter preserves the strongly-typed LogEntry in more places
|
|
||
| public boolean recoverLogs(KeyExtent extent, Collection<Collection<String>> walogs) | ||
| throws IOException { | ||
| public boolean recoverLogs(KeyExtent extent, Collection<LogEntry> walogs) throws IOException { |
There was a problem hiding this comment.
Protip: this diff looks bigger than it is in GitHub. Best to ignore whitespace / indentation while comparing for a more friendly view.
| return fromPath(parts[1]); | ||
| Text qualifier = entry.getKey().getColumnQualifier(); | ||
| String[] parts = qualifier.toString().split("/", 2); | ||
| Preconditions.checkArgument(parts.length == 2, "Malformed write-ahead log %s", qualifier); |
There was a problem hiding this comment.
Can you verify the contents of parts[0] here? Not sure if that is possible when considering possible old code.
There was a problem hiding this comment.
As far as I can tell, the parts[0] could contain - (current code) or tserver:port (previous version). I can't be sure about prior to that, but I believe there was an upgrade step that removed relative paths from WAL entries that would have rewritten anything that would be older than these two most recent versions.
My concern about adding strict validation on the ignored part is that we could cause an unnecessary recovery failure. Accommodating variance in the part that doesn't matter by not validating its content avoids that.
I just have a lot of uncertainty about the history of the various formats of these columns. I would prefer not to strictly validate the ignored part, and we just make sure we go through upgrade code for any future metadata schema changes.
3 participants
-/). This fixes Ensure log entries in the metadata table are up-to-date on upgrade (or remove strict validation) #4061 for the main branch for 3.1; a similar issue may still exist for 2.1, but will require a different fix.Collection<Collection<String>>and useCollection<LogEntry>where possible, since the former was for a previous WAL storage format that we do not support, and the latter preserves the strongly-typed LogEntry in more places