Error: PlaywrightCrawler:SessionPool:Session "Cookie not in this host's domain"

I am using PlaywrightCrawler with Firefox. When accessing wellfound.com and see this error:

DEBUG PlaywrightCrawler:SessionPool:Session: Could not set cookies. {"errorMessages":["Cookie not in this host's domain. Cookie:prod.website-files.com Request:wellfound.com"]}

DEBUG PlaywrightCrawler:SessionPool:Session: Could not set cookies. {"errorMessages":["Cookie not in this host's domain. Cookie:prod.website-files.com Request:wellfound.com"]}

It might be that this cookie is something important: I'm navigating to another page on this site and get HTTP 403 and captcha... How to fix this error? Have these settings in code:

        
    useSessionPool: true,
    persistCookiesPerSession: true,
    sessionPoolOptions: {
         maxPoolSize: 300,
         sessionOptions:{
             maxAgeSecs: 70,
             maxUsageCount: 2,
         },
     },
     
    launchContext: {
        ...
        launchOptions: {
            bypassCSP: true,
            acceptDownloads: true,

        
    useSessionPool: true,
    persistCookiesPerSession: true,
    sessionPoolOptions: {
         maxPoolSize: 300,
         sessionOptions:{
             maxAgeSecs: 70,
             maxUsageCount: 2,
         },
     },
     
    launchContext: {
        ...
        launchOptions: {
            bypassCSP: true,
            acceptDownloads: true,

2 Replies

Hall•8mo ago

View post on community site

This post has been pushed to the community knowledgebase. Any replies in this thread will be synced to the community site.

Apify Community

Pepa J•7mo ago

Hi @new_in_town As I understand the error it happens when the request is sent. It makes sense do not set cookie related to different subdomain than the request is - as this is also the browser default behavior.

Gaming

Programming

Error: PlaywrightCrawler:SessionPool:Session "Cookie not in this host's domain"

Did you find this page helpful?