W
Wasp•7d ago
Elliot_bai

How to handle scheduled jobs in a load-balanced Wasp deployment?

Hi Wasp community! 👋 I'm deploying my Wasp application with load balancing using Nginx, and I have a question about scheduled jobs (using PgBoss). My setup: - 2 Wasp server instances behind Nginx load balancer - Using dailyStatsJob for analytics - PostgreSQL database shared between instances My concern: Will scheduled jobs run multiple times if I have multiple server instances? For example, will dailyStatsJob execute on both server instances and potentially create duplicate data? Questions: 1. Does PgBoss automatically handle job deduplication across multiple instances? 2. Do I need to designate one instance as the "primary" for running jobs? 3. What's the recommended approach for scheduled jobs in a load-balanced Wasp deployment? Current job definition: job dailyStatsJob { executor: PgBoss, perform: { fn: import { calculateDailyStats } from "@src/analytics/stats.js" }, schedule: { cron: "0 1 * * *" } } Any guidance would be greatly appreciated! Thanks! 🙏
2 Replies
kapa.ai
kapa.ai•7d ago
Hi @Elliot_bai. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave: ---------------------- Great questions! Here’s how scheduled jobs work in a load-balanced Wasp deployment using PgBoss: 1. Does PgBoss automatically handle job deduplication across multiple instances? Yes, PgBoss is designed for this scenario. When you have multiple Wasp server instances (all connected to the same PostgreSQL database), PgBoss ensures that a scheduled job (like your dailyStatsJob) is only executed once per schedule, regardless of how many server instances are running. PgBoss uses the database as a coordination point, so only one instance will pick up and execute each scheduled job occurrence. This prevents duplicate executions and data creation across your instances.
"pg-boss provides many useful features... By using PostgreSQL (and SKIP LOCKED) as its storage and synchronization mechanism, it allows us to provide many job queue pros without any additional infrastructure or complex management."
Declaring Jobs - API Reference
2. Do I need to designate one instance as the "primary" for running jobs? No, you do not need to designate a primary instance. All instances can safely run PgBoss, and the coordination is handled at the database level. This is one of the main benefits of using PgBoss with PostgreSQL in a distributed environment. 3. What's the recommended approach for scheduled jobs in a load-balanced Wasp deployment? - Run PgBoss on all instances: Let each Wasp server instance start PgBoss. The job will only be picked up by one instance at a time, thanks to the locking and coordination provided by PgBoss and PostgreSQL. - No special configuration needed: As long as all instances share the same PostgreSQL database, you’re set. - Avoid CPU-heavy jobs: Since jobs run alongside your server code, avoid CPU-intensive tasks in jobs, as they could impact your server’s responsiveness.
"Wasp starts pg-boss alongside your web server's application, where both are simultaneously operational... Wasp does not (yet) support independent, horizontal scaling of pg-boss-only applications, nor starting them as separate workers/processes/threads."
pg-boss considerations
Summary:
You do not need to worry about duplicate scheduled job executions in a load-balanced Wasp deployment with PgBoss. PgBoss and PostgreSQL handle coordination, so each scheduled job runs only once per schedule, no matter how many Wasp instances you have. If you need more advanced scaling (e.g., running job workers separately from your web servers), that’s not yet supported natively in Wasp, but for most use cases, the current setup is robust and reliable. Let me know if you have any more questions! ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ---------------------- Sorry @Elliot_bai, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist: Perhaps try again in 15mins?
miho
miho•5d ago
As I understand it, your setup should be perfectly fine with how Wasp uses PgBoss for its job scheduling. There is no difference between scheduled or non-scheduled jobs in terms of guarantees of PgBoss (it won't run the same job twice if you have two app instances). 1. Regular jobs: when you invoke them in Wasp, they get inserted in the pgboss.job table and then one of the workers will pick them up 2. Scheduled jobs: PgBoss itself keeps checking the pgboss.schedule table if some job needs to be invoked, and when the time comes, PgBoss inserts the job in the pgboss.job table Here's a Copilot generated diagram 🙂 
+------------------+         +---------------------+
|  pgboss.schedule |<------->| Application (schedule MyJob) |
+------------------+         +---------------------+
        |
        | (SQL poll: is it time to run MyJob?)
        v
+------------------+
|   pgboss.job     |  <--- new row inserted for MyJob at scheduled time
+------------------+
        |
        | (Worker polls for state='created')
        v
+------------------+
| Worker process   |
+------------------+
        |
        | (Job processed, state updated in pgboss.job)
        v
+------------------+
|   pgboss.job     |  <--- state: completed/failed/etc.
+------------------+
+------------------+         +---------------------+
|  pgboss.schedule |<------->| Application (schedule MyJob) |
+------------------+         +---------------------+
        |
        | (SQL poll: is it time to run MyJob?)
        v
+------------------+
|   pgboss.job     |  <--- new row inserted for MyJob at scheduled time
+------------------+
        |
        | (Worker polls for state='created')
        v
+------------------+
| Worker process   |
+------------------+
        |
        | (Job processed, state updated in pgboss.job)
        v
+------------------+
|   pgboss.job     |  <--- state: completed/failed/etc.
+------------------+

Did you find this page helpful?