Monday, March 7, 2022

How to spot time-series issues in real-time with Anomaly Detection

Application or system events that are not detected early can lead to catastrophic consequences.  The impact could result in defects, injuries, theft, failures, loss of money and much more.  Azure Anomaly Detector uses time-series data to identify unusual behavior patterns, events, outliers, errors, or rare changes in data.  We are excited to announce a new MS Learn module on Azure Anomaly Detector that teaches you how to use this AI service to foresee problems before they occur.


Hands-On Exercise


In this short new Microsoft Learn module, you will learn how to use a smart device, IoT Hub and Azure Anomaly Detector on time-series data to quickly catch unexpected abnormalities.  It will enable developers and data scientists of all skill levels to easily add AI decision-making to their apps.   Here are the hands-on exercises in the module you’ll be working on:

  • Connect a smart device to IoT Hub to collect time-series data
  • Build a solution to leverage the prebuilt Azure Anomaly Detector API model on real-time IoT data streams.
  • Train and evaluate a custom model with Azure Anomaly Detector using your unique time-series data.
  • Find the root cause of an anomaly when you have more than one metric/variable in your data.



Machine Learning challenges that Anomaly Detector handles


The learn module will address the different challenges in training an anomaly detection model.  For example, there are multiple types of data patterns: Spikes/Dips, periodic wavy pattern, inclining pattern, declining pattern etc.  In most real-life situations, data patterns are inconsistent and unpredictable.  There is no one algorithm that fits all the patterns.  Next, it will address the difficulty in pinpointing where the root cause is; when you have many metrics that could be the culprit.


Industry use cases:


Azure Anomaly Detector can be used across industries.  Here are a few examples:

  • Manufacturing, automotive, agriculture: Collecting data from sensors to identify machinery malfunctions; and using predictive maintenance for early anomalies that can lead to damage or faults.
  • Healthcare: Examining medical devices and log to spot outliers; or gathering patient diagnostics data behavior for disorder detections.
  • IT, DevOps: Monitoring system networks for security breaches and intrusion detection; or watching computer and system pipeline metrics to catch unusual events that could cause system downtime.
  • Financial: Tracking consumer spending abnormality behaviors in real-time for fraud detection (in banking, credit card or insurance).  Observing stock market trends to detect deviations to make predictions.


Robust reliable benefits of Azure Anomaly Detector:


  • Pretrained Automatic model selection: Azure Anomaly Detector API is part of Azure Cognitive Services prebuilt AI models that assess your time-series dataset and automatically selects the best algorithm in real-time based on the data pattern.  So, no matter what pattern your data has, the model can adapt accordingly.
  • Root cause detection: Whether your time series data has one metric or many metrics, the service can find and rank the contributors of the anomaly at the timestamp.
  • Custom Training & deploying AI model: The Anomaly Detector service provides you with the flexibility and ease to use APIs to train, export, inference or delete your model with your unique dataset.  You also have APIs to review the JSON results with anomalies.
  • Containerized model: The Anomaly Detector models are containerized for edge, local or cloud deployments.
  • REST API & SDK support:  Python, C#, Java, JavaScript etc.


Let’s get started


Get started on this quick Microsoft Learn module that is hands-on and self-pace that uses an end-to-end IoT scenario to identify abnormal time-series data using Azure Anomaly Detector.   You will collect data from a smart meter device; send data to the cloud; and detect anomalies.







After completing this lesson, you will be able to use the Azure Anomaly Detector to build AI solutions that are reliable and ensure that your processes run smoothly.  You will understand how the Azure Anomaly Detector APIs work and how to apply the AI service to you time-series data analytics.  Get started here.  Happy Learning!

Posted at