performance optimization

[He-Pin]

New numbers are updated:
https://github.com/CertainLach/jrsonnet/blob/master/docs/benchmarks.adoc

We should optimize the performance

[He-Pin]

Performance Optimization Progress Report

Benchmark Reference: jrsonnet 0.5.0-pre97 vs sjsonnet 0.6.2 (from jrsonnet benchmarks.adoc)

CPU Reference: AMD Ryzen 9 9950X3D (jrsonnet's benchmarks) | Apple Silicon M-series (our local testing)

Current Gap Analysis (Scala-Native / Rust ratio, sorted worst-first)

#	Benchmark	Rust (ms)	Scala-N (ms)	Ratio	Status
1	Foldl string concat	3.7	328.6	88.8x	Fixed — PR #665 (StringBuilder O(n))
2	std.foldl	2.2	80.0	36.4x	Fixed — PR #665 (while-loop foldl)
3	Large string template	2.1	14.5	6.9x	Optimizing — PR #689 (bulk text block scanner)
4	Kube-prometheus	43.6	204.6	4.7x	In progress (composite benchmark)
5	Big object	839.2	3756.8	4.5x	Optimizing — PR #687/#688 (inline object storage)
6	std.base64	1.2	4.8	4.0x	Planned
7	Comparison for array	10.0	39.1	3.9x	Optimizing — PR #674 (ref equality) + PR #686 (inline binop)
8	Large string join	2.6	9.7	3.7x	Planned
9	std.base64Decode	1.2	4.2	3.5x	Planned
10	Realistic 2	120.9	417.8	3.5x	In progress (composite benchmark)
11	std.manifestTomlEx	750.5	2558.8	3.4x	Backlog
12	std.manifestJsonEx	748.5	2443.6	3.3x	Backlog
13	std.substr	917.8	2991.7	3.3x	Backlog
14	Array sorts	1.3	4.2	3.2x	Fixed — PR #664 (in-place Arrays.sort)
15	Inheritance function recursion	743.5	2394.1	3.2x	Backlog
16	String strips	751.5	2377.2	3.2x	Planned (ASCII bitset)
17	std.parseInt	771.7	2383.6	3.1x	Backlog
18	Tail call	865.2	2406.8	2.8x	Backlog
19	Realistic 1	5.9	12.7	2.2x	Improved — now 1.19x faster on combined branch
20	Graalvm CI	34.4	69.6	2.0x	In progress (composite benchmark)
21	std.base64DecodeBytes	19.2	37.2	1.9x	Planned
22	std.reverse	24.5	44.1	1.8x	Planned
23	Comparison for primitives	218.1	174.8	0.80x	Already faster
24	Simple recursive call	35.8	18.0	0.50x	Already faster
25	std.base64 (byte array)	15.6	7.1	0.46x	Already faster
26	Lazy array	54.8	19.3	0.35x	Already faster
27	Inheritance recursion	160.1	46.7	0.29x	Already faster

Combined Optimization Results (24 PRs merged, Apple Silicon M-series)

Tested with source-built jrsonnet 0.5.0-pre98 on the same machine:

Benchmark	Master (ms)	Optimized (ms)	jrsonnet (ms)	Speedup	vs jrsonnet
realistic1	15.6	11.8	14.1	-24%	1.19x faster
std.foldl	144.3	5.4	5.7	-96%	1.04x faster
comparison2 (primitives)	168.4	65.4	196.5	-61%	3.0x faster
gen_big_object	13.0	11.1	11.3	-15%	1.02x faster
bench.02	74.2	66.6	113.7	-10%	1.71x faster
bench.03	32.4	32.5	51.5	0%	1.59x faster
base64_byte_array	10.3	9.9	17.8	-4%	1.81x faster
realistic2	292.1	241.7	97.1	-17%	2.49x slower
comparison (array)	32.8	26.6	12.7	-19%	2.09x slower
base64	10.4	8.3	5.2	-20%	1.59x slower
bench.01 (tail call)	8.1	8.0	4.8	-1%	1.67x slower
reverse	49.1	30.6	22.7	-38%	1.35x slower
base64DecodeBytes	40.8	25.9	20.2	-36%	1.28x slower

Open PRs (24 performance optimizations)

All PRs target master as draft PRs for review:

Area	PRs	Impact
Core evaluation	#663, #667, #668, #674, #675, #682, #686	Comprehension fuse, singleton bools, structural equality, valTag dispatch
Object creation	#671, #687, #688	Sorted key cache, single/multi-field inline storage
Stdlib	#664, #665, #669, #670, #683	In-place sort, foldl StringBuilder, range allocation, map/flatMap, num cache
Rendering	#676, #678, #680, #681, #684, #685	Indent cache, escape fast-path, direct stdout, bulk append, num stringify, direct long
Format	#679	Runtime format optimization
Parser	#689	Bulk text block scanner
Array	#673	arraycopy concat

Key Achievements

• std.foldl: 144ms to 5.4ms (26.6x speedup) — now faster than jrsonnet
• realistic1: 15.6ms to 11.8ms — now 1.19x faster than jrsonnet
• comparison2: 168ms to 65ms — now 3.0x faster than jrsonnet
• 5 benchmarks already faster than jrsonnet (Scala-native vs Rust)

Next Optimization Targets

1. realistic2 (2.49x gap) — largest comprehensive benchmark, needs object creation + field lookup optimizations
2. comparison (array) (2.09x) — array element comparison fast path
3. large_string_template (2.46x) — string formatting optimization
4. lstripChars (1.72x) — ASCII bitset optimization
5. base64 (1.59x) — encoding algorithm optimization
6. reverse (1.35x) — array reverse caching

[stephenamar-db]

I merged a large number of your PRs, saw some huge improvements in key ones:

  │         Benchmark         │ Before  │ After  │ Change │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ cpp/bench.04              │ 126.649 │ 1.571  │ -98.8% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ go/foldl                  │ 33.528  │ 7.703  │ -77.0% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ go/comparison             │ 18.845  │ 10.840 │ -42.5% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ go/comparison2            │ 111.287 │ 74.918 │ -32.7% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ cpp/realistic1            │ 7.651   │ 5.399  │ -29.4% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ cpp/large_string_template │ 7.188   │ 5.095  │ -29.1% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ cpp/bench.06              │ 0.944   │ 0.715  │ -24.3% │
  ├───────────────────────────┼─────────┼────────┼────────┤
  │ bug/assertions            │ 0.685   │ 0.603  │ -12.0% │

but also saw some significant regressions on the following ones:

  ┌──────────────────────┬────────┬────────┬─────────┐
  │      Benchmark       │ Before │ After  │ Change  │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/manifestYamlDoc   │ 0.152  │ 0.581  │ +282.2% │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/escapeStringJson  │ 0.064  │ 0.193  │ +201.6% │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/lstripChars       │ 1.332  │ 2.066  │ +55.1%  │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/manifestJsonEx    │ 0.129  │ 0.179  │ +38.8%  │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/base64_byte_array │ 3.504  │ 4.700  │ +34.1%  │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/member            │ 1.927  │ 2.288  │ +18.7%  │
  ├──────────────────────┼────────┼────────┼─────────┤
  │ go/base64DecodeBytes │ 24.486 │ 26.957 │ +10.1%  │
  └──────────────────────┴────────┴────────┴─────────┘

[He-Pin]

will update the pending processes later, and there is a new scala native release, and we should update that too

[ngbinh]

This is awesome!

[stephenamar-db]

last batch of PRs:

  ┌──────────────────────┬─────────┬─────────┬────────┐
  │      Benchmark       │ Before  │  After  │ Change │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/lstripChars       │ 1.587   │ 0.933   │ -41.2% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/base64Decode      │ 1.474   │ 0.956   │ -35.1% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/stripChars        │ 1.448   │ 0.942   │ -34.9% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/base64            │ 1.927   │ 1.263   │ -34.5% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/rstripChars       │ 1.476   │ 0.972   │ -34.1% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/substr            │ 0.369   │ 0.276   │ -25.2% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ cpp/realistic2       │ 265.780 │ 223.102 │ -16.1% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/base64_byte_array │ 4.009   │ 3.391   │ -15.4% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/manifestJsonEx    │ 0.145   │ 0.130   │ -10.3% │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/comparison        │ 11.092  │ 10.085  │ -9.1%  │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ cpp/realistic1       │ 5.768   │ 5.255   │ -8.9%  │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ cpp/bench.04         │ 1.605   │ 1.471   │ -8.3%  │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/manifestYamlDoc   │ 0.163   │ 0.150   │ -8.0%  │
  ├──────────────────────┼─────────┼─────────┼────────┤
  │ go/foldl             │ 0.865   │ 0.807   │ -6.7%  │
  └──────────────────────┴─────────┴─────────┴────────┘

[He-Pin]

We can test the scala-native build with hyperfine vs jrsonnet

[He-Pin]

@stephenamar-db I run the test , master vs 0.6.3 seems no regression on my side.
I will investigate comparation related regression

│ |-----------|:---:|:---:|:---:| │
│ | bench.04 | 33.106 | 0.437 | -98.7% 🔥 | │
│ | foldl | 9.729 | 0.263 | -97.3% 🔥 | │
│ | substr | 0.236 | 0.107 | -54.7% | │
│ | base64 | 0.809 | 0.499 | -38.3% | │
│ | rstripChars | 0.600 | 0.373 | -37.8% | │
│ | stripChars | 0.586 | 0.365 | -37.7% | │
│ | base64Decode | 0.599 | 0.376 | -37.2% | │
│ | lstripChars | 0.589 | 0.388 | -34.1% | │
│ | realistic1 | 3.001 | 2.073 | -30.9% | │
│ | base64_byte_array | 1.466 | 1.151 | -21.5% |

[stephenamar-db]

an area I want to look into is object inheritance. I'm wondering if it would be possible to change how we represent inheritance to be more compact. I'm also thinking about how we would optimize inheritance a bit for cases where super is not used - kind of a fast path for objects not requiring to reach super.

[He-Pin]

yeah, that will help both llvm and jit, we should mark some of them final and sealed class too.

[stephenamar-db]

I experimented with an alternate flat representation for inheritance, but it does not help on performance or memory usage. It did reduce GC pressure.

[He-Pin]

I think we need to spot more patterns and do some AST transformation in the static optimizer, which will help the runtime performance, and we can still do some SIMD optimization in the fastparse upstream repo, and that will reduce our parsing time too.

[He-Pin]

sjsonnet vs jrsonnet Benchmark Results

Date: 2026-04-13
CPU: AMD Ryzen 9 9950X3D 16-Core (matching jrsonnet's benchmark machine)
jrsonnet: 0.5.0-pre98 (built from source, latest master)
sjsonnet: all-pending-prs branch (master + 8 pending PRs merged)

---

Before vs After Comparison

Benchmark	Before (sjsonnet 0.6.2)	After (all PRs)	jrsonnet	Before Ratio	After Ratio	Change
Foldl string concat	328.6 ms	10.3 ms	13.0 ms	88.4x slower	1.26x faster	-97%
Inheritance recursion	46.7 ms	59.8 ms	122.1 ms	0.29x (faster)	2.04x faster	+604%
Simple recursive call	18.0 ms	33.3 ms	58.0 ms	0.50x (faster)	1.74x faster	+248%
Big object	3757 s	13.2 ms	15.4 ms	4.48x slower	1.17x faster	-97%
Large string join	9.7 ms	8.0 ms	9.6 ms	3.77x slower	1.19x faster	-68%
std.base64	4.8 ms	5.8 ms	7.1 ms	3.89x slower	1.25x faster	-68%
std.base64Decode	4.4 ms	2.0 ms	2.8 ms	4.00x slower	1.41x faster	-65%
Realistic1	12.7 ms	21.5 ms	17.0 ms	2.15x slower	1.27x slower	-41%
Tail call	2407 s	9.5 ms	7.4 ms	2.78x slower	1.28x slower	-54%
Inheritance func recursion	2394 s	6.5 ms	8.1 ms	3.22x slower	1.28x slower	-60%
Array sorts	4.2 ms	6.6 ms	8.9 ms	3.09x slower	1.34x slower	-57%
Large string template	14.5 ms	16.3 ms	8.7 ms	6.91x slower	1.88x slower	-73%
Realistic2	417.8 ms	168.9 ms	103.9 ms	3.46x slower	1.63x slower	-53%
GraalVM CI	69.6 ms	-	34.4 ms	2.03x slower	-	-
Kube-prometheus	204.6 ms	-	43.6 ms	4.70x slower	-	-

Note: Before values are from jrsonnet's benchmark page (sjsonnet 0.6.2, Scala Native). "After" values measured on this machine with all 8 PRs merged. "Before Ratio" shows how sjsonnet 0.6.2 compared to jrsonnet; "After Ratio" shows current standing. Bold indicates sjsonnet is faster.

---

Summary Statistics

Metric	Before	After
Benchmarks where sjsonnet wins	3	3
Within 1.5x of jrsonnet	0	8
More than 2x slower	8	0
Worst case ratio	88.4x slower	1.88x slower
Geometric mean ratio	7.8x slower	1.35x slower

---

Key Findings

sjsonnet Leads (3/13 benchmarks)

• Foldl string concat: 1.26x faster -- rope string (perf: rope string for O(1) concat + compact layout + extended foldl detection #761) eliminates O(n^2) concatenation
• Inheritance recursion: 2.04x faster -- sjsonnet's object model excels here
• Simple recursive call: 1.74x faster -- efficient TCO implementation

Competitive (within 1.5x, 8 benchmarks)

• Big object, Large string join, std.base64, std.base64Decode, Realistic1, Tail call, Inheritance func recursion, Array sorts

Still Behind (2 benchmarks)

• Large string template: 1.88x slower -- format string fast path (perf: pack format specs and avoid literal substrings #767) helped but jrsonnet's Rust string handling is inherently faster
• Realistic2: 1.63x slower -- complex workload with cross-product comparisons; string comparison optimization (perf: optimize string comparison fast path and array flatten #768) reduced gap from 3.46x

---

Biggest Wins from Pending PRs

PR	Description	Impact
#761	Rope string + foldl StringBuilder	Foldl concat: 328ms -> 10ms (32x improvement)
#749	FastBase64	base64: 3.89x -> 1.25x gap
#768	String comparison + array flatten	Realistic2: 3.46x -> 1.63x gap
#767	Format string fast path	Large string template: 6.91x -> 1.88x gap
#769	Lazy stdlib initialization	Big object: 4.48x -> 1.17x (now faster)

---

Priority Recommendations for Future Optimization

1. Large String Template (1.88x gap) -- Medium Priority
The format string fast path (#767) only handles simple %(key)s patterns. The remaining gap likely comes from jrsonnet's zero-cost Rust string handling. Further gains would require deeper rendering pipeline optimizations.

2. Realistic2 (1.63x gap) -- Medium Priority
This is a complex real-world workload involving cross-product comprehensions, string comparisons, and array flattening. We've already applied two optimizations (#768 string compare, array flatten). The remaining gap is likely in the evaluator's object construction overhead. Consider profiling to find the next hot path.

3. Tail Call / Inheritance Func Recursion (~1.28x gap) -- Low Priority
These are micro-benchmarks where jrsonnet's Rust zero-overhead abstraction has a natural advantage. The gap is small enough that further optimization would require invasive changes to the evaluator. Not worth the effort for <30% gains.

4. Kube-prometheus / GraalVM CI -- Not Yet Measured
These real-world benchmarks were not measured in this round. The lazy stdlib initialization (#769) should significantly improve startup-heavy workloads. Should benchmark separately.

---

Conclusion

After merging all 8 pending PRs, sjsonnet has gone from 3 benchmarks where it leads and 8 benchmarks where it was 2-88x slower, to 3 benchmarks where it leads, 8 benchmarks within 1.5x, and zero benchmarks more than 2x slower. The geometric mean improved from 7.8x slower to 1.35x slower vs jrsonnet.

The rope string implementation (#761) is the single biggest win, turning the worst-case benchmark (88.4x slower) into a case where sjsonnet is now faster than jrsonnet.

[He-Pin]

@stephenamar-db I have updated a recent benchmark local run with all pending pr merged with jrsonnet ↑

[stephenamar-db]

great. Sadly, all those changes are pretty neutral against one of our more challenging build internally (a mega-aggregator of multiple jsonnet files).
I've looked into what could be done to improve the evaluation pipeline and did not find anything significant - from a simpler HashMap, to a new structure of EvalScope - I'm pretty sure we are hitting a wall here.

I've starting looking into more static analysis/AST rewritting.

[stephenamar-db]

FWIW, We've experimented internally with a Truffle-based jsonnet interpreter too, and we reached the same limitations: jsonnet is just fundamentally slow, specifically due to the laziness approach. It's just expensive both on CPU and memory to save the scope for every lazy computations.