CA
Crawlee & Apify•2y ago
like-gold

Can't mark request as failed

Hello, I am trying to mark a request as failed using crawlee + playwright. I tried multiple things, from throwing exceptions to use request.pushErrorMessage() method. The session.retire() is working for marking the session as bad, but not for mark the request as failed: 
await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;
await session.retire();
await request.pushErrorMessage(new Error("Captcha detected"));
return;
With the previous code, I get Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed. When I expect it to fail and retry Throwing directly an exception also does not work new Error("Captcha detected") There is a similar issue solved using Apify actor, but in this case I do not have access to it as I am using vanila Crawlee https://discord.com/channels/801163717915574323/1105228219013140480/1115869864268152852
4 Replies
national-gold
national-gold•2y ago
You could do something like request.noRetry = true; and then throw after it. If you just throw - request will be retried (3 times by default). This will show request as failed one in the status message
like-gold
like-goldOP•2y ago
Even throwing an exception I still see Terminal status message: Finished! Total 1 requests: 1 succeeded, 0 failed. The only time where It is retrying and I see failed requests is when I have a timeout or the website returns and 403 forbidden. But in my case it is returning an 200 OK but with a captcha and I want to mark this request as failed and retry it.
MEE6
MEE6•2y ago
@HelThunk just advanced to level 1! Thanks for your contributions! 🎉
national-gold
national-gold•2y ago
just throwing an error inside requestHandler will lead to further retry. Retrying 3 times will mark requests as failed and it will be reflected in the status message. I just tried it to double-checked

Did you find this page helpful?