Failed to parse URL from [object Object]

This is the request that I'm trying to add:
let popReportRequest = new Request({
url: 'https://www.beckett.com/grading/pop-report/',
method: 'POST',
payload: payload_data,
headers: postHeaders
})
await crawler.addRequests([popReportRequest])
let popReportRequest = new Request({
url: 'https://www.beckett.com/grading/pop-report/',
method: 'POST',
payload: payload_data,
headers: postHeaders
})
await crawler.addRequests([popReportRequest])
This is the error message :
node:internal/deps/undici/undici:7004
throw new TypeError("Failed to parse URL from " + input, { cause: err });
^

TypeError: Failed to parse URL from [object Object]
at new Request (node:internal/deps/undici/undici:7004:19)
at file:///run/media/neonomade/work/collectibles/beckett_pup/src/main.js:30:29
at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
[cause]: TypeError [ERR_INVALID_URL]: Invalid URL
at new NodeError (node:internal/errors:399:5)
at URL.onParseError (node:internal/url:566:9)
at new URL (node:internal/url:646:5)
at new Request (node:internal/deps/undici/undici:7002:25)
at file:///run/media/neonomade/work/collectibles/beckett_pup/src/main.js:30:29
at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
input: '[object Object]',
code: 'ERR_INVALID_URL'
}
}

Node.js v18.15.0
node:internal/deps/undici/undici:7004
throw new TypeError("Failed to parse URL from " + input, { cause: err });
^

TypeError: Failed to parse URL from [object Object]
at new Request (node:internal/deps/undici/undici:7004:19)
at file:///run/media/neonomade/work/collectibles/beckett_pup/src/main.js:30:29
at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
[cause]: TypeError [ERR_INVALID_URL]: Invalid URL
at new NodeError (node:internal/errors:399:5)
at URL.onParseError (node:internal/url:566:9)
at new URL (node:internal/url:646:5)
at new Request (node:internal/deps/undici/undici:7002:25)
at file:///run/media/neonomade/work/collectibles/beckett_pup/src/main.js:30:29
at ModuleJob.run (node:internal/modules/esm/module_job:194:25) {
input: '[object Object]',
code: 'ERR_INVALID_URL'
}
}

Node.js v18.15.0
I don't understand why the URL can't be parsed...
3 Replies
NeoNomade
NeoNomadeOP3y ago
@HonzaS sorry for tagging you, but if you can have a look at this it would be great
dependent-tan
dependent-tan3y ago
this should work, no need to for the new Request
let popReportRequest = {
url: 'https://www.beckett.com/grading/pop-report/',
method: 'POST',
payload: payload_data,
headers: postHeaders
};
await crawler.addRequests([popReportRequest]);
let popReportRequest = {
url: 'https://www.beckett.com/grading/pop-report/',
method: 'POST',
payload: payload_data,
headers: postHeaders
};
await crawler.addRequests([popReportRequest]);
also notice that request queue is deduplicating on url by default, so another post request with the same url will not be added unless you specify another uniqueKey or set useExtendedUniqueKey to true
NeoNomade
NeoNomadeOP3y ago
oh yes I forgot about this I will enable useExtendedUniqueKey

Did you find this page helpful?