File Unzip using miniz library#1098
Conversation
|
Do we know how well this handles really large files? Say you try to unzip a 10GB file. Does it store the whole thing in memory? (10GB memory usage) Are there any progress indicators while it's decompressing a large file? Or does it just hang until it returns? |
Hmm. This will require looking deeper into the C API. The C++ wrapper can't do what you're asking. I'll look into it. |
|
This can potentially get really intrusive depending on how the library is currently written. We'd also need to design an API to expose the progress. What I envision is that in the table showing all the resources, when the user click Most likely is that we inject a callback into the unzip routine that will be periodically called so we can update the UI with the % completed. This may need to go hand-in-hand with splitting up a large file so we don't hold the entire thing in memory. IOW, a lot of investigation to be done. |
|
I tested it on a 16GB file and it does work. It prints out progress as it unzips. It's not currently done on a background thread. But I feel that can be added later. One thing that should be tested is if large files work with Linux. I read that some Linux compilers require the following definition: #define _LARGEFILE64_SOURCE 1. And I want to see if it indeed works. |
jw098
changed the title
|
|
||
| // Construct your output path (e.g., target_dir + file_stat.m_filename) | ||
| std::string out_path = std::string(target_dir) + "/" + file_stat.m_filename; | ||
| fs::path const parent_dir{fs::path(out_path).parent_path()}; |
There was a problem hiding this comment.
Clang query was finally useful ! You're creating a path from string and that's forbidden
There was a problem hiding this comment.
Use PokemonAutomation::Filesystem::Path. For the reason why, you can read https://discord.com/channels/695809740428673034/1462210406616531259/1462567541825339635
3 participants
Note: this does not extract tar files, only zip files.