Unable to process a large volume of files.

[chaegumi]

My scenario involves over 13,000 HTML files with a total size of 1.15 GB; under these conditions, it resulted in an immediate error.

 13:01:25 [build] Waiting for integration "@playform/compress", hook "astro:build:done"...
 
<--- Last few GCs --->

 [3823096:0x773c000] 16006907 ms: Mark-Compact 2006.2 (2092.0) -> 2006.0 (2092.3) MB, pooled: 0 MB, 208.81 / 0.00 ms  (average mu = 0.355, current mu = 0.146) allocation failure; scavenge might not succeed
 [3823096:0x773c000] 16007231 ms: Mark-Compact 2008.0 (2094.3) -> 2008.0 (2109.8) MB, pooled: 1 MB, 320.49 / 0.00 ms  (average mu = 0.188, current mu = 0.008) allocation failure; scavenge might not succeed
 
 
 <--- JS stacktrace --->
 
 FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
 ----- Native stack trace -----
 
  1: 0xe36196 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
  2: 0x123f4a0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
  3: 0x123f777 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
  4: 0x146d1a5  [node]
  5: 0x1486a19 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
  6: 0x145b0e8 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  7: 0x145c015 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
  8: 0x143440e v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node]
  9: 0x1423504 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Tagged<v8::internal::Map>, v8::internal::AllocationAlignment) [node]
 10: 0x142540f v8::internal::FactoryBase<v8::internal::Factory>::NewRawTwoByteString(int, v8::internal::AllocationType) [node]
 11: 0x143fdea v8::internal::Factory::NewStringFromUtf8(v8::base::Vector<char const>, v8::internal::AllocationType) [node]
 12: 0x125b9bd v8::String::NewFromUtf8(v8::Isolate*, char const*, v8::NewStringType, int) [node]
 13: 0x10ed429  [node]
 14: 0xf6304d  [node]
 15: 0x7fc9bfe0f745 
 ELIFECYCLE  Command failed.
 /root/.local/share/pnpm/pnpm  run build： signal: aborted (core dumped)

[NikolaRHristov]

This:

node --max-old-space-size=16384 $(which npx) astro build

comes to mind or try setting:

export NODE_OPTIONS="--max-old-space-size=16384"

[chaegumi]

For small projects, it's fine, but for large projects, your approach is not advisable. I'll switch to using middleware