CA
Crawlee & Apify•4w ago
stormy-gold

Customising logging

Is there a recommended way to customise logging? I want to be able to log which specific crawler and which handler a log is coming from. I have tried to override the logger in the crawler using 
import defaultLog, { Log } from '@apify/log';
...
const crawler = new BasicCrawler({
  requestHandler: router,
  log: defaultLog.child({ prefix: 'MyCrawler' })
})
import defaultLog, { Log } from '@apify/log';
...
const crawler = new BasicCrawler({
  requestHandler: router,
  log: defaultLog.child({ prefix: 'MyCrawler' })
})
but then I get the following type error: 
Type 'import("scrapers/node_modules/@apify/log/esm/index", { with: { "resolution-mode": "import" } }).Log' is not assignable to type 'import("scrapers/node_modules/@apify/log/cjs/index").Log'.
  Types have separate declarations of a private property 'options'.ts(2322)
Type 'import("scrapers/node_modules/@apify/log/esm/index", { with: { "resolution-mode": "import" } }).Log' is not assignable to type 'import("scrapers/node_modules/@apify/log/cjs/index").Log'.
  Types have separate declarations of a private property 'options'.ts(2322)
Thanks!
4 Replies
Hall
Hall•4w ago
Someone will reply to you shortly. In the meantime, this might help: -# This post was marked as solved by Matous. View answer.
genetic-orange
genetic-orange•4w ago
I think it should work if you get the log from 'apify' instead of '@apify/log'. 
import { log } from 'apify';
import { BasicCrawler } from 'crawlee';

const crawler = new BasicCrawler({
    log: log.child({ prefix: 'MyCrawler' }),
    requestHandler: ({ log: myLog, request }) => {
        myLog.info(`Check this out! ${request.url}`);
    },
});
import { log } from 'apify';
import { BasicCrawler } from 'crawlee';

const crawler = new BasicCrawler({
    log: log.child({ prefix: 'MyCrawler' }),
    requestHandler: ({ log: myLog, request }) => {
        myLog.info(`Check this out! ${request.url}`);
    },
});
This works fine for me. I think there is something broken with the import path. I'll pass this to the engineering team but let me know if this works for you first.
stormy-gold
stormy-goldOP•4w ago
@luigi.ruocco This works for me! Had to of course install the apify package separately though which isn't ideal as I only need crawlee/logging but did the trick
correct-apricot
correct-apricot•4w ago
@je no need for the apify package: 
import { log, LogLevel } from 'crawlee'
log.setOptions({
    logger: customLogger,
    level: LogLevel.DEBUG,
})
import { log, LogLevel } from 'crawlee'
log.setOptions({
    logger: customLogger,
    level: LogLevel.DEBUG,
})

Did you find this page helpful?