Zod•7mo ago

k9 - Is it expected that Zod would cause such a...

Is it expected that Zod would cause such a spike in build memory consumption? This is an interesting one. I've been able to deploy to Heroku for years. Now Heroku won't build my application because the Typescript compiler runs out of memory. The only meaningful change I can think of is introducing Zod. I deploy once every few days, so it might not be the case. Heroku configuration seems to use the same environment variables for compilation and deployment, (Wow do I have security questions about that one.) The important thing is that our runtime app caps memory at 350: NODE_OPTIONS=--max-old-space-size=350. But now the compilation phase dies with this error:

       > tsc --build src/tsconfig.json
       
<--- Last few GCs --->
[975:0x6dd3ae0]    21417 ms: Scavenge 346.6 (356.0) -> 345.6 (356.5) MB, 11.6 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
[975:0x6dd3ae0]    21426 ms: Scavenge 347.0 (356.5) -> 345.6 (357.0) MB, 2.2 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
[975:0x6dd3ae0]    21684 ms: Scavenge (reduce) 347.8 (357.3) -> 345.9 (354.3) MB, 3.5 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

       > tsc --build src/tsconfig.json
       
<--- Last few GCs --->
[975:0x6dd3ae0]    21417 ms: Scavenge 346.6 (356.0) -> 345.6 (356.5) MB, 11.6 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
[975:0x6dd3ae0]    21426 ms: Scavenge 347.0 (356.5) -> 345.6 (357.0) MB, 2.2 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
[975:0x6dd3ae0]    21684 ms: Scavenge (reduce) 347.8 (357.3) -> 345.9 (354.3) MB, 3.5 / 0.0 ms  (average mu = 0.339, current mu = 0.265) allocation failure; 
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

I compiled locally setting the environment variable a number of different values and found that roughly --max-old-size=400 would do the trick. But should Zod actually consume that much memory?

6 Replies

k9•7mo ago

I wonder if this is just an incremental increase, but I still can't believe the same environment variables are used during the build and runtime steps.

Unknown User•7mo ago

Message Not Public

k9•7mo ago

it makes me wonder whether I should abandon Zod if that's going to be an ongoing issue. Thank you That's going to be my approach for now (too bad)

Unknown User•7mo ago

Message Not Public

k9•7mo ago

Thanks, @gtflip

Scott Trinh•7mo ago

Type inference can be expensive for the typescript compiler and 5.4 fixed a type bug at the cost of much higher instantiations so it could be something like that.

Gaming

Programming

k9 - Is it expected that Zod would cause such a...