Skip to content

ENCODINGS_LOOKUP mixes string and bytes keys in the same dict #44

Description

@sumanjeet0012

ENCODINGS_LOOKUP is a single dict that uses both str keys (encoding names like "base16") and bytes keys (prefix codes like b"f"). This is confusing, type-unsafe, and could cause subtle bugs if a name happens to match a prefix byte.

Problem

In multibase/multibase.py:

ENCODINGS_LOOKUP = {}
for codec in ENCODINGS:
    ENCODINGS_LOOKUP[codec.encoding] = codec  # str key: "base16"
    ENCODINGS_LOOKUP[codec.code] = codec      # bytes key: b"f"

The resulting dict has mixed key types:

ENCODINGS_LOOKUP = {
    "identity": Encoding(...),     # str
    b"\x00": Encoding(...),        # bytes
    "base2": Encoding(...),        # str
    b"0": Encoding(...),           # bytes
    "base16": Encoding(...),       # str
    b"f": Encoding(...),           # bytes
    # ...
}

Problems:

  1. Type confusion — Functions like is_encoding_supported() accept str but the dict also has bytes keys, so is_encoding_supported(b"f") returns True even though the docstring says it takes a str
  2. Potential collision — If an encoding name were a single character that matches another encoding's prefix byte, there would be a collision (currently safe but fragile)
  3. get_codec() relies on this — It checks data[:4] for emoji, then data[:1] for single-byte prefixes, both against the same mixed dict

Proposed Solution

Split into two separate dicts:

_ENCODINGS_BY_NAME: dict[str, Encoding] = {}
_ENCODINGS_BY_CODE: dict[bytes, Encoding] = {}

for codec in ENCODINGS:
    _ENCODINGS_BY_NAME[codec.encoding] = codec
    _ENCODINGS_BY_CODE[codec.code] = codec

Update all functions to use the appropriate dict:

def is_encoding_supported(encoding: str) -> bool:
    return encoding in _ENCODINGS_BY_NAME

def get_codec(data):
    data = ensure_bytes(data, "utf8")
    # Check emoji first (4-byte prefix)
    if len(data) >= 4 and data[:4] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:4]]
    # Check single-byte prefix
    if data[:1] in _ENCODINGS_BY_CODE:
        return _ENCODINGS_BY_CODE[data[:1]]
    raise InvalidMultibaseStringError(...)

Keep ENCODINGS_LOOKUP as a backward-compatible alias if needed:

# Backward compatibility
ENCODINGS_LOOKUP = {**_ENCODINGS_BY_NAME, **_ENCODINGS_BY_CODE}

Related

  • File: multibase/multibase.py

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions