Repository navigation
Memory usage is tens of times higher than stdlib when used as Coraza WAF regex engine replacement #158
Description
Description
When using coregex to replace the default regexp engine in corazawaf/coraza, memory consumption is tens of times higher than the standard library under the same workload (loading OWASP CRS ruleset + processing requests).
Steps to Reproduce
- Register
coregexas Coraza'srxoperator via plugin system:
import (
"fmt"
"github.com/corazawaf/coraza/v3/experimental/plugins"
"github.com/corazawaf/coraza/v3/experimental/plugins/plugintypes"
"github.com/coregx/coregex"
"sync"
)
var doer = makeDoer(new(sync.Map))
func RegisterRX() {
plugins.RegisterOperator("rx", newRX)
}
type RegexpInf interface {
FindStringSubmatch(s string) []string
MatchString(s string) bool
}
type rx struct {
re RegexpInf
}
var _ plugintypes.Operator = (*rx)(nil)
func newRX(options plugintypes.OperatorOptions) (plugintypes.Operator, error) {
data := fmt.Sprintf("(?sm)%s", options.Arguments)
re, err := Do(data, func() (interface{}, error) {
return coregex.Compile(data)
})
if err != nil {
return nil, err
}
return &rx{re: re.(RegexpInf)}, nil
}
func (o *rx) Evaluate(tx plugintypes.TransactionState, value string) bool {
if tx.Capturing() {
match := o.re.FindStringSubmatch(value)
if len(match) == 0 {
return false
}
for i, c := range match {
if i == 9 {
return true
}
tx.CaptureField(i, c)
}
return true
}
return o.re.MatchString(value)
}
func Do(key string, fn func() (interface{}, error)) (interface{}, error) {
value, err, _ := doer(key, fn)
return value, err
}
func makeDoer(cache *sync.Map) func(string, func() (interface{}, error)) (interface{}, error, bool) {
return func(key string, fn func() (interface{}, error)) (interface{}, error, bool) {
value, found := cache.Load(key)
if found {
return value, nil, true
}
data, err := fn()
if err == nil {
cache.Store(key, data)
}
return data, err, false
}
}2.Load OWASP CRS ruleset with Coraza:
config := coraza.NewWAFConfig().
WithDirectives(`
Include @coraza.conf-recommended
Include @crs-setup.conf.example
Include @owasp_crs/*.conf
SecRequestBodyLimit 131072000
SecRuleEngine On
`).
WithRootFS(coreruleset.FS)
waf, _ := coraza.NewWAF(config)Reacted by Andrey Kolkov
Activity
kolkov commentedon Jun 15, 2026
Thank you for the detailed report and apologies for the delayed response — we've been heavily focused on the gogpu and born-ml ecosystems recently. This is a real and important issue, and we've now completed a deep investigation.
Root Cause
Each
coregex.Compile()eagerly allocates multiple internal engines (up to 7 PikeVM instances, 5 NFA copies, and several DFA caches) regardless of whether a search ever runs. For 900 CRS patterns, this adds up to ~134 MB vs stdlib's ~8.3 MB — a 16x overhead.The core problem is architectural: Rust regex (our reference) cleanly separates compiled regex (immutable, shared) from search state (mutable, per-thread, lazy). We currently allocate everything at compile time.
Measured Breakdown
(?i)literal cross-product expansionFix Plan
We're implementing Rust's memory architecture in a single refactor (not phased):
Target: ~20-30 KB/pattern (~2-3x stdlib), down from current 152 KB/pattern (16x).
Tracking in the next patch release. Will update here when the fix is available.
Merge pull request #160 from coregx/feature/lazy-memory-architecture
kolkov commentedon Jun 15, 2026
Fix released in v0.12.22.
What changed:
Expected improvement: ~5x reduction in memory per compiled pattern for WAF workloads.
Please try it and let us know if the memory usage is acceptable now:
If the overhead is still too high, we have a second phase ready (PikeVM program/cache full split, lazy NFA variants) that can push it further.