Task scheduler infra?
I'm looking for something that can do the following (or do essentially the same thing). It feels like something that would be some kind of open source tech / AWS service / something.
We currently have a bunch of cron scrips that are all doing some version of this to schedule emails, notifications, reminders, etc. Anyone have thoughts on how they typically handle timestamp-based task scheduling? If this doesn't exist, someone please build it!!
- I can /PUT an id, JSON blob, URL and timestamp
- When the timestamp occurs, the provided URL gets called with the json blob
- If multiple tasks are within a close enough timestamp (1-2 seconds), they get batched into a single API call with an array
- Retries are handled intelligently with some kind of exponential backoff. Ideally I can specify whether a task should be retried or discarded after a certain period of time
- If the service is about to DDOS the URL with a ton of tasks, it spreads them out
- I can /POST an id to edit the task
- It scales effectively and has no problem handling 1m+ scheduled tasks
- a UI where I can inspect currently scheduled tasks / see failures
We currently have a bunch of cron scrips that are all doing some version of this to schedule emails, notifications, reminders, etc. Anyone have thoughts on how they typically handle timestamp-based task scheduling? If this doesn't exist, someone please build it!!
