CA
foreign-sapphire
Faster task startup?
Any options for faster startup of an actor task? Specifically the
ACTOR: Pulling Docker image from repository. and the ACTOR: Starting Docker container. steps.3 Replies
foreign-sapphireOP•3y ago
In other words, have the task payload ready to run, on API call, with as little delay as possible. I suppose I could keep it running 24/7 but that would be a waste.
Latest crawlee under 128Mb with cheerio crawler should be done in 5sec (average) per request, even if its only single request in run or task, so just use light container and find a way to access web data wo browser and you will have solution which works comparable to regular API
There might be lags if you use residential proxies, can be solved by fast retrying i.e. set timeout to 10sec per request or check if you can use datacenter proxies instead residential
continuing-cyan•3y ago
Use the most base image. Apify team is working on improving the start times, they got slower lately.