Skip to content

Fix Modbus handler DoS from unbounded, byte-at-a-time oversized reads (#619) - #623

Open
Krisztian766 wants to merge 1 commit into
mushorg:mainfrom
Krisztian766:fix-619-modbus-oversized-length-dos
Open

Fix Modbus handler DoS from unbounded, byte-at-a-time oversized reads (#619)#623
Krisztian766 wants to merge 1 commit into
mushorg:mainfrom
Krisztian766:fix-619-modbus-oversized-length-dos

Conversation

@Krisztian766

Copy link
Copy Markdown

Fixes #619.

Root cause

_, _, length = struct.unpack(">HHH", request[:6])
while len(request) < (length + 6):
    try:
        new_byte = sock.recv(1)
        request += new_byte
    except Exception:
        break

length comes straight from the client with no upper bound (up to 0xFFFF),
and the loop reads it back one byte at a time, appending onto request
each time. This is two compounding problems, not one:

  1. CPU cost, not just syscall count. request += new_byte copies the
    entire accumulated buffer every iteration (bytes are immutable), so a
    64KB frame is O(n²) byte copies, not O(n). Measured in isolation with
    plain blocking sockets: ~1.2 seconds of CPU time for a single 0xFFFF-byte
    frame delivered in one segment, matching the issue's reproduction steps
    exactly.
  2. An actual infinite loop, not just a slow one, if the peer half-closes
    after sending only the header.
    sock.recv(1) on a socket whose peer
    has shut down its write side returns b'' immediately on every call
    (not an exception) - so request never grows, the loop condition never
    changes, and this spins at 100% CPU forever. I confirmed this manually:
    the connection never closed within a 15-second wall-clock timeout, and
    it doesn't recover on its own - not even gevent.Timeout inside the same
    process can rescue it, since the tight CPU-bound loop never yields to
    gevent's hub, so no other callback in the process ever runs again,
    including the timer backing that Timeout.

Since Conpot runs every protocol (HTTP, S7, Modbus, ...) as greenlets on one
shared event loop, both variants stall or fully freeze every other emulated
service for as long as the malicious connection is being processed - matching
the issue's description and its noted use as a timing-based fingerprinting
vector.

Fix

  • Reject any declared length over 254 immediately, before attempting to read
    any body at all. 254 isn't arbitrary: a conforming Modbus/TCP frame is at
    most a 1-byte unit id + up to 253 bytes of PDU, the size limit inherited
    from serial Modbus (256-byte ADU minus 1-byte address minus 2-byte CRC).
    No legitimate client needs more.
  • Replace the byte-at-a-time loop with a single bounded sock.recv(remaining)
    for whatever's left (now capped at 254), and treat an empty read as
    end-of-stream (break) instead of looping on it.

Testing

  • Added test_oversized_length_is_rejected_without_reading_body, which
    reproduces the half-close hang. Deliberately not a wall-clock timing
    assertion, since the underlying bug is a genuine infinite loop rather than
    a slow path - see the test's docstring for why even gevent.Timeout can't
    safely bound it against the pre-fix code (confirmed manually: hangs the
    whole process rather than failing cleanly, so I didn't want a test that
    can hang CI indefinitely on a future regression - the docstring explains
    the manual reproduction instead).
  • All 7 tests in conpot/tests/test_modbus_server.py pass with the fix
    (python -m pytest conpot/tests/test_modbus_server.py -v).
  • Manually reproduced the original issue two more ways before writing the
    fix: (a) plain blocking sockets outside gevent, isolating the O(n²)
    byte-copy cost at ~1.2s for a 64KB frame; (b) a full gevent StreamServer
    with a second "victim" connection sampled continuously during the attack,
    showing a 739.9ms max latency spike (vs <1ms baseline) pre-fix and 39.1ms
    post-fix.
  • black --check passes on both changed files.

Note (unrelated to this fix)

conpot/tests/test_logger_taxii.py fails to even import on Python 3.13
(libtaxii imports the removed stdlib cgi module), and running the full
suite with a fresh env needs setuptools<81 pinned explicitly since
pkg_resources was dropped from newer setuptools before the <84 upper
bound in pyproject.toml catches it. Neither is related to this change -
flagging in case it's useful, happy to open separate issues if wanted.

…mushorg#619)

A client could declare an MBAP length up to 0xFFFF and the handler would
read the body one byte at a time via sock.recv(1) in a loop with no upper
bound. Two compounding problems made this a real DoS, not just a slow path:

- With the full declared body delivered in one segment, the byte-at-a-time
  reads plus repeated `request += new_byte` (an O(n) bytes copy per
  iteration, so O(n^2) overall) measurably pins the CPU for over a second
  on a 64KB frame.
- If the peer instead half-closes after sending only the header, recv(1)
  returns b'' immediately on every call instead of raising - `request`
  never grows and the loop never terminates. Confirmed manually: this
  hangs the process indefinitely (15s+ wall clock, unrecoverable even via
  gevent.Timeout, since the tight loop never yields to gevent's hub).

Since Conpot runs every protocol as greenlets on one shared event loop,
either case stalls or fully freezes every other emulated service for as
long as the malicious connection is being processed.

Fix: reject any declared length over 254 bytes immediately (a conforming
Modbus/TCP frame is at most 1-byte unit id + 253-byte PDU, the limit
inherited from serial Modbus - RFC/spec max, not an arbitrary number), and
replace the byte-at-a-time read with a single bounded sock.recv(remaining)
for whatever's left within that cap.

Added a regression test that reproduces the half-close hang deterministically
(not timing-based - the underlying bug is a genuine infinite loop) and
verified: all 7 tests in test_modbus_server.py pass with the fix, and the
new test reliably hangs the process before it (manually confirmed, not
included as a second automated test to avoid a permanently-hanging CI job
if this ever regresses again).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Modbus handler stalls every protocol on an oversized request body

1 participant