CA
Crawlee & Apify•2y ago
multiple-amethyst

How to call apify actors from a separate server?

Context: - I'm currently using playwright on my nextjs api routes - these playwright scripts scrape a website & persist the data in my database (postgres) - since I need IP roration with session management though, I'd love to offload the scraping to crawlee - I'm considering apify as the deployment platform as this seems to be the recommended setup and apify apparently also supports webhooks -- e.g. can I get notified about the status of my actor run, such as job.started or job.finished events? - I'd love for the crawlee scraper on apify to persist data to my database so that my nextjs api refers to the database as the source of truth I imagine I'd call something like this within my nextjs api route: 
import { ApifyClient } from "apify-client"
const apify = new ApifyClient({ token: 'my-token' });


// Resource clients accept an ID of the resource.
const actor = apify.actor('john-doe/my-actor');
// Fetches the john-doe/my-actor object from the API.
const myActor = await actor.get();
// Starts the run of john-doe/my-actor and returns the Run object.
const myActorRun = await actor.start();
import { ApifyClient } from "apify-client"
const apify = new ApifyClient({ token: 'my-token' });


// Resource clients accept an ID of the resource.
const actor = apify.actor('john-doe/my-actor');
// Fetches the john-doe/my-actor object from the API.
const myActor = await actor.get();
// Starts the run of john-doe/my-actor and returns the Run object.
const myActorRun = await actor.start();
The actor should ideally update my postgres database directly. All I need to do is set up a webhook receiver for incoming webhooks to get updated about what's happening.
1 Reply
Pepa J
Pepa J•2y ago

Did you find this page helpful?