Real-Time Anomaly Detection
An end-to-end Fabric streaming pattern: ingest from Event Hubs or Kafka, land in an Eventhouse, score with KQL anomaly functions, alert through Activator and Teams. Ship live monitoring in days.
Five years ago "real-time anomaly detection" was a six-month engineering project. With Fabric it's a few hundred lines of KQL plus some Activator rules. This accelerator gives you the exact pattern we deploy on manufacturing OEE, application observability, and security telemetry use cases — wired together and battle-tested.
Drop in your source, point KQL at the right columns, tune the dwell windows, and you have an alerting system within the week.
Who this is for
- Engineering teams who already have an event source (Event Hubs, Kafka, Application Insights, IoT Hub) and want to act on it inside Fabric instead of building yet another streaming stack.
- SRE/ops teams who want to replace a brittle Logic Apps or custom alerting chain with KQL + Activator.
- Manufacturing IT teams who need OEE drift detection across sites.
- Security teams piloting Fabric for telemetry that currently lives in Sentinel/Splunk.
You need familiarity with KQL or willingness to learn it (it's worth the investment). You do not need to be a streaming expert — the accelerator does the heavy lifting.
What's in the box
A full streaming stack: ingest, store, detect, alert.
| File | Type | What it does |
|---|---|---|
infrastructure/eventhouse_deploy.ps1 | PS | Provisions Eventhouse, KQL DBs (raw + curated), retention & caching policies. |
infrastructure/eventstream.json | JSON | Eventstream definition: Event Hubs/Kafka source, inline transforms, Eventhouse destination. |
kql/01_ingestion_mapping.kql | KQL | Table schemas + JSON ingestion mappings + update policies for raw → curated. |
kql/10_spike_detection.kql | KQL | Function: sudden-spike detection using series_decompose_anomalies over rolling windows. |
kql/11_drift_detection.kql | KQL | Function: gradual drift detection comparing rolling means to baseline. |
kql/12_missing_data.kql | KQL | Function: detects expected-but-missing events ("heartbeat" pattern). |
kql/13_threshold_breach.kql | KQL | Function: static + dynamic threshold breaches with hysteresis. |
kql/20_materialized_views.kql | KQL | Pre-aggregated views for sub-second dashboard tile queries. |
activator/rules.json | JSON | Four Activator rules wiring the KQL functions to Teams + Power Automate destinations. |
dashboard/realtime-monitor.json | JSON | Real-Time Dashboard with live tiles, time-bound filter, drill-through. |
powerbi/historical-trends.pbix | PBIX | Companion Power BI report on the OneLake mirror of your KQL data. |
tools/event_generator.py | Python | Synthetic event generator for testing: spike, drift, gap patterns. Feeds Event Hubs. |
docs/RUNBOOK.md | MD | Operations runbook: tuning thresholds, suppressing alert storms, replay strategy. |
docs/KQL_PRIMER.md | MD | Just-enough KQL guide for teams new to the language. |
The four detection patterns
- Spike. Sudden surges (3+ standard deviations from the rolling mean). Catches incidents, attacks, viral events.
- Drift. Gradual shifts over hours or days. Catches slow degradations, calibration loss, behavior change.
- Missing. Heartbeats that didn't arrive. Catches silent failures — the most dangerous kind.
- Threshold breach. Static or dynamic thresholds with hysteresis to prevent flapping. The simple-but-essential case.
Every pattern includes a dwell window so single-point spikes don't fire pages, and a quiet period after firing to prevent storms.
Pricing
Frequently asked questions
Does it require my source to be Event Hubs or Kafka?
What if my events are non-numeric (text, categorical)?
How are alerts delivered?
Does it survive the Eventhouse autoscaling?
What's the smallest F-SKU this needs?
Refund policy?
From "we should alert on this" to "alerts firing" in a week
That's the gap this accelerator closes.