Skip to content

Prevent memory leak when exception is thrown in adl_serializer::to_json - #3901

Merged
nlohmann merged 7 commits into
nlohmann:developfrom
barcode:fix_mem_leak_in_adl_serializer
Mar 8, 2023
Merged

Prevent memory leak when exception is thrown in adl_serializer::to_json#3901
nlohmann merged 7 commits into
nlohmann:developfrom
barcode:fix_mem_leak_in_adl_serializer

Conversation

@barcode

@barcode barcode commented Dec 27, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

(problem described in issue #3881)

There is a memory leak when throwing inside an adl serializers to_json method.

  • Add unit test
  • Implement fix
  • Fix Ci

Pull request checklist

Read the Contribution Guidelines for detailed information.

  • Changes are described in the pull request, or an existing issue is referenced.
  • The test suite compiles and runs without error.
  • Code coverage is 100%. Test cases can be added by editing the test suite.
  • The source code is amalgamated; that is, after making changes to the sources in the include/nlohmann directory, run make amalgamate to create the single-header files single_include/nlohmann/json.hpp and single_include/nlohmann/json_fwd.hpp. The whole process is described here.

@barcode
barcode requested a review from nlohmann as a code owner December 27, 2022 21:23
@barcode barcode mentioned this pull request Dec 27, 2022
Closed
1 of 2 tasks
@github-actions

github-actions Bot commented Dec 27, 2022

Copy link
Copy Markdown

🔴 Amalgamation check failed! 🔴

The source code has not been amalgamated.

@coveralls

coveralls commented Dec 27, 2022 •
edited
Loading

Copy link
Copy Markdown

Coverage Status

Coverage: 100.0%. Remained the same when pulling 79bae8a on barcode:fix_mem_leak_in_adl_serializer into b230614 on nlohmann:develop.

@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch 4 times, most recently from f8cf474 to db58287 Compare December 28, 2022 07:55
@barcode

barcode commented Dec 28, 2022

Copy link
Copy Markdown
Contributor Author

The test detects the memleak: https://github.com/nlohmann/json/actions/runs/3792554026/jobs/6448842970#step:5:316

@github-actions github-actions Bot added L and removed M labels Dec 28, 2022
@barcode

barcode commented Dec 28, 2022

Copy link
Copy Markdown
Contributor Author

I went with the solution using a member var, since it requires less changes to the existing code and the resulting code is simpler to understand.

@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch 2 times, most recently from 4018619 to df39768 Compare December 29, 2022 07:02
@gregmarr

gregmarr commented Dec 29, 2022

Copy link
Copy Markdown
Contributor

Since this keeps most of the handling in the json class rather than the new data class, such as the copy constructor and move constructor, should the assert invariant function stay there as well? Also, should the copy and move constructors of this class be defaulted? Seems like they should be deleted.

@barcode

barcode commented Dec 30, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author 
should the assert invariant function stay there as well?

in that case the assert invariant method is not called if an exception in the ctor occurred. Then we do not need to adapt that method to deal with broken objects. Since the dtor of an element is called after the objects dtor finishes, the call order would be OK as well (assert before destroy).

Also, should the copy and move constructors of this class be defaulted? Seems like they should be deleted.

we need at least the move ctor

json/include/nlohmann/json.hpp

Line 1196 in df39768

m_data(std::move(other.m_data))

The others we could delete.

@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch from 580c36e to 0e2f01a Compare January 3, 2023 07:08
gregmarr
gregmarr reviewed Jan 3, 2023
View reviewed changes
Comment thread include/nlohmann/json.hpp Outdated
gregmarr
gregmarr reviewed Jan 3, 2023
View reviewed changes
Comment thread include/nlohmann/json.hpp
@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch from 0e2f01a to cd0a103 Compare January 3, 2023 22:35
gregmarr
gregmarr reviewed Jan 3, 2023
View reviewed changes
Comment thread tests/src/unit-regression2.cpp Outdated
#endif

#ifdef JSON_HAS_CPP_20
#ifdef JSON_HAS_CPP_20 && __has_include(<span>)

@gregmarr gregmarr Jan 3, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The proper form for this is #if defined(JSON_HAS_CPP_20) && ...

@barcode barcode Jan 4, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should sleep instead of writing code while tired...

@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch 2 times, most recently from 0ed7496 to e9e1543 Compare January 4, 2023 06:36
@barcode

barcode commented Jan 4, 2023

Copy link
Copy Markdown
Contributor Author

I looked a bit at the ci_test_compilers_clang (10) issue.
This PR did not change the affected test and it worked in earlier builds.

It seems the compiler and changed to a newer version.

old:

Compiler: clang version 10.0.0; Target: x86_64-unknown-linux-gnu; Thread model: posix; InstalledDir: /usr/local/bin

new:

Compiler: Debian clang version 10.0.1-++20210313014605+ef32c611aa21-1~exp1~20210313125208.190; Target: x86_64-pc-linux-gnu; Thread model: posix; InstalledDir: /usr/bin

Since this issue has nothing to do with this PR. it probably should be fixed in a separate PR.

barcode added 4 commits January 31, 2023 19:29
barcode
Add unit test for leaking memory when throwing inside of an adl seria…
aadeb72 
…lizer

(problem described in issue nlohmann#3881)
barcode
Fix leaking memory when throwing inside of an adl serializer
5a1a575
barcode
Fix ci issues
96f75d2 
* Add checks for uninitialized m_value (this can happen due to exceptions in the ctor)
* Add missing noexcept
* Fix clang tidy errors
barcode
Move assert_invariant to surrounding basic_json class
3613fa6
@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch from e9e1543 to 1210c3e Compare January 31, 2023 18:30
@barcode
barcode force-pushed the fix_mem_leak_in_adl_serializer branch from 8e99a23 to 79bae8a Compare February 1, 2023 17:24
@barcode

barcode commented Feb 2, 2023

Copy link
Copy Markdown
Contributor Author

Work on this PR is done and it only needs a review.

nlohmann
nlohmann approved these changes Feb 7, 2023
View reviewed changes

@nlohmann nlohmann left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@nlohmann nlohmann linked an issue Feb 7, 2023 that may be closed by this pull request
Memory leak when exception is thrown in adl_serializer::to_json #3881
Closed
1 of 2 tasks
@nlohmann nlohmann added this to the Release 3.11.3 milestone Feb 7, 2023
@nlohmann
nlohmann merged commit bbe337c into nlohmann:develop Mar 8, 2023
@nlohmann

nlohmann commented Mar 8, 2023

Copy link
Copy Markdown
Owner

Thanks!

@raphael-grimm
raphael-grimm deleted the fix_mem_leak_in_adl_serializer branch August 4, 2023 07:21
@nlohmann nlohmann mentioned this pull request Nov 28, 2023
Merged
@gmyers18 gmyers18 mentioned this pull request Sep 5, 2024
Merged
3 of 4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nlohmann nlohmann nlohmann approved these changes
+1 more reviewer
@gregmarr gregmarr gregmarr left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

L tests

Projects

None yet

Milestone

Release 3.11.3

Development

Successfully merging this pull request may close these issues.

Memory leak when exception is thrown in adl_serializer::to_json

4 participants

@barcode @coveralls @gregmarr @nlohmann