Hey there, it would be really useful for
Hey there, it would be really useful for the puppeter and playwright crawlers to be more considerate of their memory state while running. Ive had lots of occasions where crawlee ends up in a fatal OOM state where the autoscale pool tries to scale up, even when there isnt enough memory for the browser since the node process itself isnt consuming much RAM. eg, in a container with 4GB of memory, if node takes up 200mb and the browser takes up the remaining 3700mb, autoscaled pool would still try to scale up since node isnt consuming the ram
1 Reply
deep-jade•6mo ago
If you could create issues for Crawlee, that would be great. thanks