Partition Lag Heatmap & Cluster Topology
Real-time visual partition health, consumer group lag skew detection, and interactive broker topology maps for enterprise streaming clusters.
The Problem with Traditional Lag Tables
In high-scale Kafka and streaming deployments, topics frequently have dozens or hundreds of partitions consumed by horizontally scaled application instances. Traditional UIs display lag as long paginated lists of numbers:
Partition 0: Lag 12 | Partition 1: Lag 15 | Partition 2: Lag 42,910 | Partition 3: Lag 8
In a 128-partition topic, spotting that Partition 2 is stalled due to a single poisoned poison-pill message or thread deadlock is like finding a needle in a haystack.
The Partition Lag Heatmap
EventOre's LagHeatmapView groups partitions into compact visual severity tiles color-coded by lag thresholds:
- Nominal (< 100 msgs): Emerald green tiles indicating healthy consumers keeping up with real-time producer velocity.
- Moderate (100 - 1,000 msgs): Amber tiles signaling elevated processing times or minor traffic bursts.
- Critical (> 1,000 msgs): Glowing crimson tiles highlighting immediate consumer stalls, backpressure, or deadlocks.
Automated Partition Skew Detection
EventOre automatically analyzes the standard deviation and max-to-median ratio across all partitions assigned to a consumer group. If a subset of partitions has significantly higher lag than others, EventOre raises an automatic Skew Warning:
order-service-pod-2 for thread stalls or poison pill processing errors.
Cluster Broker Topology Map
The topology viewer displays an interactive map of your messaging cluster:
- Broker Nodes: Node ID, host/port, controller / leader status, and rack awareness.
- Partition Distribution: Visual audit of leader partitions vs follower replicas across physical brokers to detect hotspotting.
- Under-Replicated Partitions (URP): Instant visual alerts when replicas fall out of the In-Sync Replicas (ISR) set, preventing data loss before broker failures.
Topology & Heatmap REST APIs
GET /api/v1/connections/{id}/kafka/topics/{topic}/lag-heatmap
GET /api/v1/connections/{id}/kafka/cluster/topology
GET /api/v1/connections/{id}/kafka/consumer-groups/{group}/skew