How would you design a distributed task scheduling system?
Answer
Requirements: schedule tasks for future, recurring tasks, exactly-once execution, high availability. Architecture: scheduler service (determine when to run), executor workers (run tasks), task queue (pending tasks), task store (persistent). Scheduling: check upcoming tasks, push to queue at execution time. At-least-once: queue with acknowledgment, retry on failure, idempotent tasks. Exactly-once: distributed lock per task instance. Recurring: cron parser, generate next run time. Scale: partition by task ID, multiple schedulers with leader election. Features: priorities, dependencies, rate limiting. Examples: Airflow, Celery Beat.
Master These Concepts with IIT Certification
175+ hours of industry projects. Get placed at Bosch, Tata Motors, L&T and 500+ companies.