CA
Crawlee & Apify•2y ago
xenial-black

how can I mount the `createPlaywrightRouter()` properly?

I don't understand why crawlee is throwing an error about adding a router for the label undefined. here's my code: 
// routes.ts
export const createRouter = (args: { input: z.infer<typeof InputSchema> }) => {
    const router = createPlaywrightRouter();

    // ----- CHECK SESSION -----
    router.addDefaultHandler(async ({ session, page }) => {
    // ...
    });
    return router;
};
// routes.ts
export const createRouter = (args: { input: z.infer<typeof InputSchema> }) => {
    const router = createPlaywrightRouter();

    // ----- CHECK SESSION -----
    router.addDefaultHandler(async ({ session, page }) => {
    // ...
    });
    return router;
};
Which I'm importing into the main.ts file: 
import { createRouter } from "./routes";

const requestHandler = createRouter({ input: inputValidation.data });
const crawler = new PlaywrightCrawler({
    requestHandler,
    },
});
import { createRouter } from "./routes";

const requestHandler = createRouter({ input: inputValidation.data });
const crawler = new PlaywrightCrawler({
    requestHandler,
    },
});
TypeScript is happy with it, so I'm not sure why I get this error:
5 Replies
xenial-black
xenial-blackOP•2y ago
here are the logs

message.txt

xenial-black
xenial-blackOP•2y ago
as an info: I do need an input value within the routeHandler which is why I have the createRouter function, which then return the router. I now tried the following but this also wouldn't work: 
const crawler = new PlaywrightCrawler({
    proxyConfiguration,
    useSessionPool: true,
    persistCookiesPerSession: true,
    requestHandler: async ({ page }) => {
        await page.waitForSelector(...)
        // ...do stuff 

        // persist to the data store
        await Dataset.pushData({
            query: args.input.query,
            companies: companyInfoArray,
        });
    },
    sessionPoolOptions: {...}
});
const crawler = new PlaywrightCrawler({
    proxyConfiguration,
    useSessionPool: true,
    persistCookiesPerSession: true,
    requestHandler: async ({ page }) => {
        await page.waitForSelector(...)
        // ...do stuff 

        // persist to the data store
        await Dataset.pushData({
            query: args.input.query,
            companies: companyInfoArray,
        });
    },
    sessionPoolOptions: {...}
});
But still getting the same error. 🤔 I've also tried to use the createPlaywrightRouter() factory together with addDefaultHandler, but to no avail: 
const router = createPlaywrightRouter();
router.addDefaultHandler(async () => {
  console.log("hello");
};
const crawler = new PlaywrightCrawler({
    proxyConfiguration,
    useSessionPool: true,
    persistCookiesPerSession: true,
    requestHandler: async () => {
        console.log("do stuff");
    },
    // ...
});  
const router = createPlaywrightRouter();
router.addDefaultHandler(async () => {
  console.log("hello");
};
const crawler = new PlaywrightCrawler({
    proxyConfiguration,
    useSessionPool: true,
    persistCookiesPerSession: true,
    requestHandler: async () => {
        console.log("do stuff");
    },
    // ...
});
MEE6
MEE6•2y ago
@p6l.richard just advanced to level 2! Thanks for your contributions! 🎉
equal-aqua
equal-aqua•2y ago
Here is the proper way to implement router. Please, try it out. 
// routes.ts
import { createPlaywrightRouter } from '@crawlee/playwright';


export const pwRouter = createPlaywrightRouter();


pwRouter.addDefaultHandler(async (context: PlaywrightCrawlingContext) => {
    // do your things
});

// main.ts

import { pwRouter } from './routes.js';


const pwCrawler = new PlaywrightCrawler({
    proxyConfiguration,
    requestHandler: pwRouter,
    // ....
});
// routes.ts
import { createPlaywrightRouter } from '@crawlee/playwright';


export const pwRouter = createPlaywrightRouter();


pwRouter.addDefaultHandler(async (context: PlaywrightCrawlingContext) => {
    // do your things
});

// main.ts

import { pwRouter } from './routes.js';


const pwCrawler = new PlaywrightCrawler({
    proxyConfiguration,
    requestHandler: pwRouter,
    // ....
});
xenial-black
xenial-blackOP•2y ago
Thanks, Oleg. Ultimately, it was a bundle issue -- I'm now simply bundling everything into a single file and the routes are properly included.

Did you find this page helpful?