Skip to content

parseMessageMetadata error when broker entry metadata enable #10967

Closed

Description

@aloyszhang

Describe the bug
If broker entry metadata enabled, we may get the following exception in many cases :

Received error from server: Failed to get batch size for entry java.lang.IllegalArgumentException: Invalid unknonwn tag type: 6

this problem also reported in :
#10950
#9255
#10924

this is caused by #9046,

Before #9046 code is

public static MessageMetadata parseMessageMetadata(ByteBuf buffer) {
        try {
            // first, skip raw metadata if exist
            skipBrokerEntryMetadataIfExist(buffer);
            // initially reader-index may point to start_of_checksum : increment reader-index to start_of_metadata
            // to parse metadata
            skipChecksumIfPresent(buffer);
            int metadataSize = (int) buffer.readUnsignedInt();

            int writerIndex = buffer.writerIndex();
            buffer.writerIndex(buffer.readerIndex() + metadataSize);
            ByteBufCodedInputStream stream = ByteBufCodedInputStream.get(buffer);
            MessageMetadata.Builder messageMetadataBuilder = MessageMetadata.newBuilder();
            MessageMetadata res = messageMetadataBuilder.mergeFrom(stream, null).build();
            buffer.writerIndex(writerIndex);
            messageMetadataBuilder.recycle();
            stream.recycle();
            return res;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

After #9046,

 public static MessageMetadata parseMessageMetadata(ByteBuf buffer) {
        MessageMetadata md = LOCAL_MESSAGE_METADATA.get();
        parseMessageMetadata(buffer, md);
        return md;
    }

    public static void parseMessageMetadata(ByteBuf buffer, MessageMetadata msgMetadata) {
        // initially reader-index may point to start_of_checksum : increment reader-index to start_of_metadata
        // to parse metadata
        skipChecksumIfPresent(buffer);
        int metadataSize = (int) buffer.readUnsignedInt();

        msgMetadata.parseFrom(buffer, metadataSize);
    }

this PR removed code

           skipBrokerEntryMetadataIfExist(buffer);

So, this may cause exceptions when parseMessageMetadata

Activity

  1. added a commit that references this issue on Jun 18, 2021
    0774b5f
  2. added a commit that references this issue on Jun 23, 2021
    d6ee19b
  3. added a commit that references this issue on Jun 25, 2021
    9c3e904
  4. added a commit that references this issue on Sep 6, 2021
    10e74d9
  5. added a commit that references this issue on Mar 18, 2022
    bff84c6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugThe PR fixed a bug or issue reported a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions