One Pager Cheat Sheet

• Anomaly Detection is when we define what is usual or expected in a given situation and then, based on that, determine whether an observation fits the established normal pattern or not.
• Anomaly detection involves accurately categorizing outliers into Global Outliers, Contextual Outliers, and Collective Outliers to yield improved results.
• Collectively, Apple and Microsoft stock prices deviate significantly from the rest of the dataset, creating a collective outlier; however, individually the stock prices are not unusual.
• Inspecting the data manually is simple, yet not practical and prone to human error, while using Machine Learning algorithms to detect anomalies is a costlier yet more accurate, faster, and scalable solution.
• Anomaly Detection with ML can be done using either Unsupervised or Supervised Learning, with Unsupervised Learning being the most common method.
• The ML model can be trained to distinguish normal data from anomalous data using supervised learning with labelled data.
• We implemented an Long Short-Term Memory Network (LSTM) Model and used Manual Anomaly Detection techniques to find anomalous data points in an S&P 500 Daily Prices 1986 - 2018 dataset.
• We used Unsupervised Learning to cluster the data and detect anomalies without any human intervention.
• Anomaly detection automates the process of identifying unusual data points in a dataset using Machine Learning methods, making it efficient for larger datasets.