Task scheduler infra?
I'm looking for something that can do the following (or do essentially the same thing). It feels like something that would be some kind of open source tech / AWS service / something.
- I can /PUT an id, JSON blob, URL and timestamp
- When the timestamp occurs, the provided URL gets called with the json blob
- If multiple tasks are within a close enough timestamp (1-2 seconds), they get batched into a single API call with an array
- Retries are handled intelligently with some kind of exponential backoff. Ideally I can specify whether a task should be retried or discarded after a certain period of time
- If the service is about to DDOS the URL with a ton of tasks, it spreads them out
- I can /POST an id to edit the task
- It scales effectively and has no problem handling 1m+ scheduled tasks
Nice to have:
- a UI where I can inspect currently scheduled tasks / see failures
I don't need this to be super precise. Triggering anywhere within 5-20 seconds of the timestamp is basically good enough.
We currently have a bunch of cron scrips that are all doing some version of this to schedule emails, notifications, reminders, etc. Anyone have thoughts on how they typically handle timestamp-based task scheduling? If this doesn't exist, someone please build it!!
0 Replies