CA
robust-apricot
I am loading Amazon page but its actually giving me 503 error
2023-04-19T11:52:09.550Z something went wrong TimeoutError: Waiting for selector
#productTitle failed: Waiting failed: 80000ms exceeded
2023-04-19T11:52:09.552Z at Timeout.<anonymous> (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/common/WaitTask.js:64:32)
2023-04-19T11:52:09.553Z at listOnTimeout (node:internal/timers:559:17)
2023-04-19T11:52:09.554Z at processTimers (node:internal/timers:502:7)
2023-04-19T11:52:09.555Z INFO [Status message]: Actor finished successfully (exit code 0)5 Replies
conscious-sapphire•3y ago
Hi @umairisrar, can you please provide more information on the problem (code, url, ...)?
robust-apricotOP•3y ago
Hello @vojtechmaslan Thanks for rapid response
The issue is related to amazon sending 503 error after loading amazon pages from same ip/account more frequently.
Do you think we need to add some sort of Proxy as currently we arent using any proxy in it
conscious-sapphire•3y ago
Proxy rotation should definitely help with that. Amazon usually blocks the IP after a couple of hundred requests. You can check more info here: https://crawlee.dev/docs/guides/proxy-management
Proxy Management | Crawlee
Using proxies to get around those annoying IP-blocks
robust-apricotOP•3y ago
Thank you. I will check this and let you know the feedback
@umairisrar just advanced to level 1! Thanks for your contributions! 🎉