CA
Crawlee & Apify•5mo ago
sensitive-blue

Splitting the handlers into multiple files and testing

Hello so Ideally i would like to have a file for website im scraping (so ome will contain more than one handler per py file). Im thinking of what the best pattern for that is. I was just going from the docs and have router = Router[BeautifulSoupCrawlingContext]() as a global var in my routes.py but i would need to either pass that router around as a singleton into the different handler files or i would import the files into the one routes.py and then register the handers there which sounds better but then I have something like webpage_handler.py which has my handler_one(context) and handler_two(context) then i register them in routes with. Whitch is fine but doesn't look too pretty. 
@router.handler("my_label")
async def handler(context: BeautifulSoupCrawlingContext) -> None:
    handler_one(context)
@router.handler("another_label")
async def handler_another_name(context: BeautifulSoupCrawlingContext) -> None:
    handler_two(context)
@router.handler("my_label")
async def handler(context: BeautifulSoupCrawlingContext) -> None:
    handler_one(context)
@router.handler("another_label")
async def handler_another_name(context: BeautifulSoupCrawlingContext) -> None:
    handler_two(context)
to be honest not super sure wondering if someone already has a nice pattern that works.
2 Replies
Hall
Hall•5mo ago
Someone will reply to you shortly. In the meantime, this might help:
conscious-sapphire
conscious-sapphire•5mo ago
Hi. You could use decorators in this way: router.handler('label')(handler_one)

Did you find this page helpful?