Repository navigation
get rid of String.intern #332
Description
InternCache is used in jackson for json property names only and solves the problem of extra memory footprint.
In the same time it uses String.intern - that is not an issue rather than misusage of it. The purpose of using string.intern is jvm specific: usually it maintains string literal pool (and covers internal jvm cases).
There are several known drawbacks of using intern:
- String pool Hashtable is non-resizable table. That means: we suffer of hash code collisions when table will have items more than its size (suffer as much as many items in string.intern pool)
- The other drawback is involve GC into that: string intern pool is subject for minor collection too. So, app suffers of using string.intern implicitly via GC phases.
There is no use cases of check the equality of strings using == and therefore no any reasons of using String.intern - the biggest profit is to have string deduplication (already achieved).
patch
===================================================================
--- src/main/java/com/fasterxml/jackson/core/util/InternCache.java (revision 489becbfb28a41980f0d5147d6069b30fa3b5864)
+++ src/main/java/com/fasterxml/jackson/core/util/InternCache.java (revision )
@@ -58,7 +58,7 @@
}
}
}
- result = input.intern();
+ result = input;
put(result, result);
return result;
}Test case
The general idea behind syntenic json is used in test (a flat json structure with 10000 properties with prefix someQName) is provide lots of different property names to trigger usage of string.intern within InternCache - that is close to real apps use cases where amount of unique property names is on a scale from hundreds to thousands.
{
"someQName0": 0,
"someQName1": 1,
....
"someQName9999": 9999
}JMH test
Let's measure performance of a single json parse with interting and w/o it: PerfInternCache.java
Benchmark Mode Cnt Score Error Units
-PerfInternCache.intern avgt 5 4098.696 ± 164.484 us/op
+PerfInternCache.noIntern avgt 5 2320.159 ± 204.301 us/opGC
Another test is to measure how intern is affecting implicitly via GC: handle 10k the same jsons as we use in previous test (the use case is very very close to real one in real apps like web service / microsevices):
InternCache GC timings java test
Run it with -verbose:gc -XX:+PrintGCDetails -Xloggc:gc.log
and after that get the total GC pause time
$ grep "GC" gc.log | grep "Times: " | awk '{S+=$8}END{print S}'
-intern 0.1907254 +- 0.00469 sec
+w/o intern 0.07665 +- 0.00498 secConclusion
Using intern harms application performance as explicitly via more expencive InternCache.intern and implicitly via GC. In the same time we keep memory footprint on the same low possible level.
Activity
Fix FasterXML#332
IRus commentedon Nov 13, 2016
@vladimirdolzhenko what about other property sizes? Like 10, 100, 1000, 100_000?
Fix FasterXML#332
vladimirdolzhenko commentedon Nov 13, 2016
@IRus
GC pauses in ms
IRus commentedon Nov 13, 2016
What about
PerfInternCache?vladimirdolzhenko commentedon Nov 13, 2016
@IRus
PerfInternCache in us/op:
cowtowncoder commentedon Nov 14, 2016
There is
JsonFactory.Feature.INTERN_FIELD_NAMESto use if you prefer no intern()ing.]vladimirdolzhenko commentedon Nov 14, 2016
@cowtowncoder I know about that flag - the problem that it is enabled by default and that kind of optimisation harms app more neither having it switched off.
What's the point to have such kind of degradation flag rather to fix it and make apps (bearing in mind the spread around of json) using jackon feel a bit better ?
vladimirdolzhenko commentedon Nov 14, 2016
@cowtowncoder
disabled flag INTERN_FIELD_NAMES PerfInternCache2.java (the numbers from another box - no any reasons to compare absolute values with prev one)
I would really consider to disable that flag by default.
vlsi commentedon Nov 14, 2016
@vladimirdolzhenko , would you please add
-prof gc(aka.addProfiler(GCProfiler.class)) outputs? (I mainly meangc.alloc.rate.norm)vladimirdolzhenko commentedon Nov 14, 2016
@vlsi that's all for Enabled/Disable flag only
vlsi commentedon Nov 14, 2016
That does look strange.
intern=true => allocates 10258886 bytes/iteration ~ 9.8 MiB/iteration
intern=false => allocates 8698649/iteration ~ 8.3 MiB/iteration
It sounds like "string intern makes memory allocation higher", that is counter-intuitive, at least at the first sight
IRus commentedon Nov 14, 2016
This benchmark was very short in terms of time, there are results by @isopov:
https://gist.github.com/isopov/b215f473deb4ef40118c00f054dc204c
cowtowncoder commentedon Nov 14, 2016
@vladimirdolzhenko I don't agree with our assessment of its harmfulness, based on invalid (as general statement) assumptions on usage. Performance to test is for databinding; and databind does in fact benefit from equality checks. You would have known this if you had asked; or perhaps dug around.
As to what is typical number of unique names, I don't know; but nor is there anything to suggest specific number.
I think it is very arrogant of you to make a demand of removing a feature that has served users well for, what, 7 years now. If you do not want the feature, turn it off. I keep mine on.
15 remaining items
cowtowncoder commentedon Dec 14, 2016
@amanhigh This is very likely due to sub-optimal usage of
ObjectMapper, and not due to interning. This method should not become hot spot unless domain of property names is unbounded.So while you can disable interning (and even canonicalization), this has so far always been due to either not reusing
ObjectMapper, or, in some cases, not closingJsonParserinstances (not reading until end-of-input OR closing).But if you have synthetic names, on the other hand, you do want to disable both (disabling canonicalization is actually sufficient).
amanhigh commentedon Jan 6, 2017
@cowtowncoder We used single instance of Object Mapper with readValue operating on byte stream. We did get a significant improment in 99th Percentile after disabling intern caching.
We haved Java Models for all classes and don't deserialize as map hence property names are bounded. I am not sure if same applies to Keys of hashMap as well as some properties are HashMaps with varying keys.
cowtowncoder commentedon Jan 6, 2017
@amanhigh I do not doubt that you saw a significant improvement, but everything I know about interning and canonicalization suggests that the problem is because Symbol table is not being reused.
If it was, interning would only ever occur once per unique String, and this really should not cause performance problems after steady state.
Now... there are a few reasons why underlying, per-JsonFactory symbol table might not be reused.
Not using same
ObjectMapper(or, if low-level operation,JsonFactory) is a common cause but not the only one. Another common cause is thatJsonParserinstance is not closed, either implicitly (by reading all content and encountering end-of-input) or explicitly (viaclose()).But there is another possibility: if number of distinct symbols (property names) is really large (in order of tens of thousands), there is size limit (something like 40,000 or so), symbol table is not reused.
This is a protective measure to prevent OOME and certain types of DoS attacks.
I suppose that in latter case you could get into trouble because
intern()calls would effectively not be cached. If this is the case, I would be interested in figuring out better ways to handle it.My main concern here is still the fact that I believe there are many use cases where intern()ing is beneficial, and that change to better support other use cases would have detrimental effect on these.
I am also open to being proven wrong on benefits: for example, it is possible that newest JVMs may have different kinds of trade-offs. I hope to find a way to validate my knowledge here.
The reason I may seem slow is simply that there are many important things to work on -- I think this is one too -- but juggling through many things makes it harder for me to get deep understanding.
So even if it may not seem like it, I really appreciate your help @amanhigh (and everyone else too).
cowtowncoder commentedon Jan 6, 2017
@amanhigh One more thing I just realized: your mention of
Maps. This might be the problem wrt unbounded nature -- what matters at low level really is JSON keys.Would there be a way for you to locally modify
jackson-corejar, adding minor diagnostics inByteQuadsCanonicalizer? I know that's a lot to ask and wish there were hooks in place but those do not exist at this moment. But what would be interesting would be to know if table is flushed, and/or what happens when table is being closed.I suspect that you could well be hitting maximum size -- if so, symptoms would make sense.
And this could guide efforts appropriately, addressing specific usage. And maybe make it easier to repro the issue with synthetic data?
vladimirdolzhenko commentedon Jan 6, 2017
@cowtowncoder I tried to persuade you that you don't understand String.intern properly and misuse it - with examples, benchmarks and so on - and I see that you would not take any of them into account
relgames commentedon Jan 10, 2017
Could you tell those use cases please? Maybe publish your benchmarks as well if you get some time.
cowtowncoder commentedon Jan 11, 2017
@relgames testing I did was by modifying existing tests like:
https://github.com/FasterXML/jackson-benchmarks
https://github.com/cowtowncoder/java-json-performance-benchmarks
by forcible disabling interning and comparing results to ones I get when interning is enabled.
Similarly I have earlier tested
jvm-serializersusage. All of these have relatively small vocabulary.While I would not mind sharing results (or someone else running those), I do not see enough value for me to spend more time on general question; and specifically not in being convinced to remove ability to intern() keys. Users are free to change the setting as they see fit; same goes for frameworks using Jackson.
IRus commentedon May 17, 2017
Hi there, FYI https://shipilev.net/jvm-anatomy-park/10-string-intern/
relgames commentedon May 20, 2017
"In almost every project we were taking care of, removing String.intern() from the hotpaths, or optionally replacing it with a handrolled deduplicator, was the very profitable performance optimization. Do not use String.intern() without thinking very hard about it, okay?"
Summary from JDK developer.
cowtowncoder commentedon May 20, 2017
@relgames Yes? I have thought about it more than people commenting here. Article itself is fine and properly discusses challenges. It may indeed be the case that for some usage turning off intern()ing makes sense -- if you end up having hundreds of thousands of symbols. I do not think this applies to majority of JSON usage.
Given that interning can easily be turned off by those who want to do that:
and since there is clear performance improvement by test I have (
jackson-benchmarks), there's little to discuss here. I do not see any point in removing the feature.At most it might make sense to figure if default settings should differ between usage via databinding (where intern()ing can help), or by direct setup as streaming parser (at which level there is likely no benefit) for Jackson 3.x. I will create a new issue for such discussion.
cowtowncoder commentedon Nov 28, 2017
For what it's worth, there's follow up issue: #378 -- and I have some good news to share there.
cowtowncoder commentedon Aug 29, 2018
And for TL;DNR observers, Jackson 3.0 DOES NOT use
String.intern()by default,and the implementation is such that it typically performs at same level with or without intern()ing, including usage through databinding.