Skip to content

chore(deps): bump github.com/projectdiscovery/utils@v0.11.6 - #7766

Merged
dwisiswant0 merged 2 commits into
projectdiscovery:devfrom
MRX-72:fix-7749-zstd-decoder-leak
Sep 25, 2026
Merged

dwisiswant0 merged 2 commits into
projectdiscovery:devfrom
MRX-72:fix-7749-zstd-decoder-leak

Conversation

@MRX-72

@MRX-72 MRX-72 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Closes #7749.

Root cause is in the pinned github.com/projectdiscovery/utils dependency, http/normalization.go, exactly as the issue describes: zstd responses are decompressed through a *zstd.Decoder wrapped in io.NopCloser, so the decoder's background worker goroutines never get released. When a response's decompressed size exceeds MaxBodyRead, LimitReader stops before EOF, the workers block permanently, and each such response leaks goroutines plus their window buffers until the process gets OOM-killed. gzip/deflate/brotli don't spawn goroutines, so this is zstd-only.

The fix already landed upstream — github.com/projectdiscovery/utils@27864e5 (utils PR #783, merged 2026-09-21) forces synchronous decode via zstd.WithDecoderConcurrency(1). But no tagged utils release contains that commit yet (latest tag is v0.11.5, which doesn't include it), and dev is pinned to v0.11.4, which has the bug. This PR bumps go.mod to a pseudo-version at that commit — the interim step the issue itself calls for — so nuclei picks up the fix now instead of waiting on a release. Happy to switch this to a tagged version once one ships.

Verification: ran the deterministic reproducer from the issue report (NewResponseChain + Fill against a >cap zstd body, 50 iterations, with a gzip control) against the bumped dependency:

zstd 4MB (> cap, stops early)    goroutines 1 -> 1 (delta=0)
gzip 4MB (> cap) [control]       goroutines 1 -> 1 (delta=0)
zstd 256KB (< cap, drained)      goroutines 1 -> 1 (delta=0)

The issue's own numbers on the buggy version showed delta=+57 for the same scenario. go build ./..., go vet ./pkg/protocols/http/..., and go test ./pkg/protocols/http/httpclientpool/... all pass.

Summary by CodeRabbit

  • Chores
    • Completed routine maintenance to keep the application’s underlying components up to date.
    • No user-facing features or behavior changed in this update.
    • The public interface, visible appearance, and available functionality remain unchanged, so existing workflows continue to work as before.

@coderabbitai

coderabbitai Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 601d911f-8922-4465-b4c3-91f6f0744186

📥 Commits

Reviewing files that changed from the base of the PR and between 7c7d6d2 and ddc104f.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (1)
  • go.mod

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.


Walkthrough

The go.mod file changes dependency declarations: hdm/jarm-go is declared directly, projectdiscovery/utils is updated, and projectdiscovery/jarm-go is added indirectly.

Changes

Module dependency declarations

Layer / File(s) Summary
Update module dependency declarations
go.mod
Declares github.com/hdm/jarm-go v0.0.8 directly and removes its indirect declaration. Updates github.com/projectdiscovery/utils to v0.11.6-0.20260921202125-27864e59be94 and adds github.com/projectdiscovery/jarm-go v0.0.0-20260910160638-430e2bae86f6 indirectly.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Bug fix

Suggested reviewers: mzack9999

Merge Risk: ⚪ Minimal · up to ddc10

This dependency-only change brings in the reported zstd fix, with matching checksum updates. No actionable merge risk is apparent from the reviewed changes.

Architecture Summary

Architecture risk: 🔵 Low · up to ddc10

The change affects 1 system.

Changed systems: go.mod

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — go.mod (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in go.mod: Updated github.com/projectdiscovery/utils from v0.11.4 to v0.11.6.
  • observed — Modified behavior in go.mod: Added github.com/google/go-github/v92 v92.0.0 as an indirect dependency.
  • observed — Modified behavior in go.mod: Added github.com/projectdiscovery/jarm-go v0.0.0-20260910160638-430e2bae86f6 as an indirect dependency.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the dependency update to github.com/projectdiscovery/utils v0.11.6, which matches the main change.
Linked Issues check ✅ Passed The PR updates github.com/projectdiscovery/utils to include upstream commit 27864e5. The PR summary states that this commit uses zstd.WithDecoderConcurrency(1), which prevents persistent zstd wo…
Out of Scope Changes check ✅ Passed The whole-PR summary shows dependency declaration changes only. The github.com/projectdiscovery/utils update directly supports issue [#7749]. The added indirect dependency declarations are part of t…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

A rabbit checks the module list,
New versions hop into the mix.
One JARM entry takes its place,
Another joins the indirect space.
The burrow rests; the diff is small.

Comment @coderabbitai help to get the list of available commands.

@neo-by-projectdiscovery-dev

neo-by-projectdiscovery-dev Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Neo - PR Security Review

This incremental delta promotes utils from its pseudo-version to the now-tagged v0.11.6 release and records the corresponding go.sum checksums — no nuclei source code was modified and no exploitable vulnerabilities were introduced.

What Neo reviewed

go.mod, go.sum

Comment @pdneo help for available commands. · Open in Neo

@MRX-72
MRX-72 force-pushed the fix-7749-zstd-decoder-leak branch from ea3e658 to 6910eb7 Compare September 24, 2026 17:19
@MRX-72
MRX-72 changed the base branch from main to dev September 24, 2026 17:19
http.NewResponseChain() decompresses Content-Encoding: zstd responses
through utils's wrapDecodeReader(), which builds a *zstd.Decoder and
wraps it in io.NopCloser. The decoder is never closed, and klauspost's
zstd decoder keeps background worker goroutines alive until Close() is
called. When a response's decompressed size exceeds MaxBodyRead, the
LimitReader stops before EOF, the decoder's workers block permanently,
and the goroutines plus their window buffers leak for the life of the
process (projectdiscovery#7749). gzip/deflate/brotli don't spawn
goroutines, so only zstd is affected.

The fix (github.com/projectdiscovery/utils@27864e5, PR projectdiscovery#783) already
landed on utils's main branch, but no tagged release contains it yet
(latest is v0.11.5), so go.mod still points at v0.11.4. Bump to a
pseudo-version pinned at that commit as the interim step the issue
calls for, to be replaced by a tagged version once utils cuts one.

Verified with the deterministic reproducer from the issue report
(NewResponseChain + Fill against a >cap zstd body, 50 iterations):
goroutine count returns to baseline (delta=0) instead of leaking ~1
goroutine per response.
@MRX-72
MRX-72 force-pushed the fix-7749-zstd-decoder-leak branch from 6910eb7 to 0b404b6 Compare September 24, 2026 18:13
@dwisiswant0 dwisiswant0 changed the title Bump projectdiscovery/utils to pick up the zstd decoder leak fix chore(deps): bump github.com/projectdiscovery/utils@v0.11.6 Sep 25, 2026
Signed-off-by: Dwi Siswanto <git@dw1.io>
@dwisiswant0
dwisiswant0 force-pushed the fix-7749-zstd-decoder-leak branch from 7c7d6d2 to ddc104f Compare September 25, 2026 11:40
@dwisiswant0
dwisiswant0 merged commit c36316d into projectdiscovery:dev Sep 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] zstd response decoder is never closed: goroutine/memory leak on large Content-Encoding: zstd responses

2 participants