CA
Crawlee & Apify•13mo ago
flat-fuchsia

Error: Dataset with id: [ACTOR_DEFAULT_DATASET_ID] does not exist.

Hello everyone, I'm trying to use apify actor:push-data inside an actor. The command is inside a dockerized shell script [1] and as simple as this: 
curl http://httpbin.org/ip | apify actor:push-data
curl http://httpbin.org/ip | apify actor:push-data
But the run fails with: 
2024-04-18T13:34:28.642Z     at DatasetClient.throwOnNonExisting (/usr/local/lib/node_modules/apify-cli/node_modules/@crawlee/memory-storage/resource-clients/common/base-client.js:15:15)
2024-04-18T13:34:28.643Z     at DatasetClient.pushItems (/usr/local/lib/node_modules/apify-cli/node_modules/@crawlee/memory-storage/resource-clients/dataset.js:158:18)
2024-04-18T13:34:28.644Z     at async PushDataCommand.run (/usr/local/lib/node_modules/apify-cli/src/commands/actor/push-data.js:25:9)
2024-04-18T13:34:28.644Z     at async PushDataCommand._run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/command.js:43:20)
2024-04-18T13:34:28.645Z     at async Config.runCommand (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/config/lib/config.js:173:24)
2024-04-18T13:34:28.646Z     at async Main.run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/main.js:28:9)
2024-04-18T13:34:28.647Z     at async Main._run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/command.js:43:20)
2024-04-18T13:34:28.648Z     at async /usr/local/lib/node_modules/apify-cli/src/bin/run:7:9
2024-04-18T13:34:28.642Z     at DatasetClient.throwOnNonExisting (/usr/local/lib/node_modules/apify-cli/node_modules/@crawlee/memory-storage/resource-clients/common/base-client.js:15:15)
2024-04-18T13:34:28.643Z     at DatasetClient.pushItems (/usr/local/lib/node_modules/apify-cli/node_modules/@crawlee/memory-storage/resource-clients/dataset.js:158:18)
2024-04-18T13:34:28.644Z     at async PushDataCommand.run (/usr/local/lib/node_modules/apify-cli/src/commands/actor/push-data.js:25:9)
2024-04-18T13:34:28.644Z     at async PushDataCommand._run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/command.js:43:20)
2024-04-18T13:34:28.645Z     at async Config.runCommand (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/config/lib/config.js:173:24)
2024-04-18T13:34:28.646Z     at async Main.run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/main.js:28:9)
2024-04-18T13:34:28.647Z     at async Main._run (/usr/local/lib/node_modules/apify-cli/node_modules/@oclif/command/lib/command.js:43:20)
2024-04-18T13:34:28.648Z     at async /usr/local/lib/node_modules/apify-cli/src/bin/run:7:9
I think I exhausted all the debug option I found online, so I kindly ask for help here. What am I missing? [1] https://github.com/netmilk/actor-curl/blob/d31d4805e5be4a6ef06b3f2e291bbb44da517000/bin/start.sh#L6
GitHub
actor-curl/bin/start.sh at d31d4805e5be4a6ef06b3f2e291bbb44da517000...
Apify Actor Curl demo [WIP]. Contribute to netmilk/actor-curl development by creating an account on GitHub.
4 Replies
Pepa J
Pepa J•13mo ago
Hi @netmilk, Thank you for your feedback I will reach out to our tooling team for more information.
flat-fuchsia
flat-fuchsiaOP•13mo ago
Hey @Pepa J , thank you, I appreciate that. I can't wait to crack this one. 🙂
flat-fuchsia
flat-fuchsiaOP•13mo ago
Just a quick note. Please, prove me wrong, but I think I found out it couldn't ever work. This [1] condition is always true and the cli then always uses the MemoryStorage regardless the ENV variables exposed by the Run for the Actor. If I overrode the forceCloud to true it uses the ApifyClient adaptor, it works as expected and I can see the data in the "Run Detail > Storage > Dataset" tab in the Apify Cloud Console. I'll file a Github Issue later and let's follow up there. Ok? [1] https://github.com/apify/apify-cli/blob/8f7b1fd84c4070a11b0af26b64c6a5002f7a7218/src/lib/actor.ts#L30
GitHub
apify-cli/src/lib/actor.ts at 8f7b1fd84c4070a11b0af26b64c6a5002f7a7...
Apify command-line interface helps you create, develop, build and run Apify actors, and manage the Apify cloud platform. - apify/apify-cli
flat-fuchsia
flat-fuchsiaOP•13mo ago
Ok, let's follow up here in Github Issues: https://github.com/apify/apify-cli/issues/553
GitHub
actor:push-data doesn't work in the Apify Platform · Issue #553 ·...
I found out the apify actor:push-data command works locally, but doesn't work when I apify push it to the platform and then I apify run iit. I tried my best to replicate the remote environment ...

Did you find this page helpful?