Fix SSRF bypass for IPv4-mapped IPv6 in HTML-to-Markdown expansion

[adulau]

Motivation

• Prevent SSRF bypass where IPv4-mapped IPv6 literals (e.g. ::ffff:127.0.0.1) were not normalized and therefore escaped existing IPv4 blocklist checks.
• Harden URL validation to reject HTTP(S) URLs that lack a hostname as a fail-closed behavior before DNS or literal IP checks.

Description

• Add a _normalize_ip_address helper that converts IPv4-mapped IPv6 addresses to their underlying IPv4 address and return a proper ipaddress object.
• Update _is_ip_blocked to use _normalize_ip_address so mapped IPv4-in-IPv6 literals are checked against BLOCKED_RANGES.
• Make is_safe_url reject URLs without a hostname by checking parsed.hostname in addition to scheme validation.
• Add a regression test file tests/test_html_to_markdown.py covering mapped IPv6 literals, public mapped literals, DNS responses returning mapped addresses, and hostless URLs.
• Modified file: misp_modules/modules/expansion/html_to_markdown.py; added file: tests/test_html_to_markdown.py.

Testing

• Compiled the modified files with python -m compileall -q misp_modules/modules/expansion/html_to_markdown.py tests/test_html_to_markdown.py and then ran python -m unittest tests.test_html_to_markdown.
• Test run: 4 tests executed and all passed (OK).

---

Codex Task

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3bae4108a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Handle IPv4Address before reading ipv4_mapped

For any plain IPv4 literal, or for a hostname whose DNS answer includes an A record, ipaddress.ip_address() returns an IPv4Address, which does not have an ipv4_mapped attribute. This line therefore raises AttributeError before the blocklist check can return, so URLs like http://93.184.216.34/ and ordinary hostnames resolving to IPv4 addresses can crash the expansion module instead of being allowed or blocked. Guard this with getattr(ip, "ipv4_mapped", None) or an isinstance(ip, ipaddress.IPv6Address) check.

Useful? React with 👍 / 👎.