How would you design a real-time analytics dashboard system?
Answer
Architecture: event ingestion (Kafka) -> stream processing (Flink, Spark Streaming) -> serving layer (pre-aggregated results) -> dashboard (WebSocket updates). Processing: windowed aggregations (tumbling, sliding, session windows), exactly-once semantics. Serving: materialized views in fast store (Redis, Druid), denormalized for query patterns. Real-time updates: poll or WebSocket to frontend. Historical: Lambda architecture (batch + stream) or Kappa (stream only). Scale: partition by dimension, parallel processing. Challenges: late data (watermarks), high cardinality dimensions. Query: OLAP databases (ClickHouse, Druid) for ad-hoc analysis.
Master These Concepts with IIT Certification
175+ hours of industry projects. Get placed at Bosch, Tata Motors, L&T and 500+ companies.