Partition Lag Heatmap & Cluster Topology

Real-time visual partition health, consumer group lag skew detection, and interactive broker topology maps for enterprise streaming clusters.

Monitoring Innovation (REQ-112): Instead of scrolling through text tables of hundreds of partition offsets, EventOre's visual lag heatmap provides an instant high-density matrix that flags struggling consumer instances in seconds.

The Problem with Traditional Lag Tables

In high-scale Kafka and streaming deployments, topics frequently have dozens or hundreds of partitions consumed by horizontally scaled application instances. Traditional UIs display lag as long paginated lists of numbers:

Partition 0: Lag 12  | Partition 1: Lag 15  | Partition 2: Lag 42,910 | Partition 3: Lag 8

In a 128-partition topic, spotting that Partition 2 is stalled due to a single poisoned poison-pill message or thread deadlock is like finding a needle in a haystack.

The Partition Lag Heatmap

EventOre's LagHeatmapView groups partitions into compact visual severity tiles color-coded by lag thresholds:

Automated Partition Skew Detection

EventOre automatically analyzes the standard deviation and max-to-median ratio across all partitions assigned to a consumer group. If a subset of partitions has significantly higher lag than others, EventOre raises an automatic Skew Warning:

Lag Skew Alert Detected: Partition 7 lag (38,410) is 94x higher than median partition lag (408). Check consumer worker order-service-pod-2 for thread stalls or poison pill processing errors.

Cluster Broker Topology Map

The topology viewer displays an interactive map of your messaging cluster:

Topology & Heatmap REST APIs

GET /api/v1/connections/{id}/kafka/topics/{topic}/lag-heatmap
GET /api/v1/connections/{id}/kafka/cluster/topology
GET /api/v1/connections/{id}/kafka/consumer-groups/{group}/skew