Hi, all!
Welcome to a new series on Azure Sentinel Notebooks! In this post, we want to introduce everyone to the Notebooks feature of Azure Sentinel and provide some basic knowledge that we’ll build on throughout this series.
The series will take the following form:
- Part 1: What are Notebooks and when do you need them? – this post
- Part 2: How to get started with notebooks and tour of the features
- Part 3: Overview of the pre-built Notebooks and how to use them
- Part 4: How to create your own Notebooks from scratch and how to customize the existing ones
What are Notebooks and when do you need them?
We have a number of features built into Azure Sentinel that share the “books” nomenclature, i.e., Playbooks, Workbooks, and Notebooks – so it can be confusing at times.
Playbooks, of course, are based on Logic Apps and supply some of the automation capabilities for Azure Sentinel. Workbooks are provided for analysts and SOC managers to build interactive views and reports of the Azure Sentinel data.
Notebooks should be an integral part of the security team’s daily processes, particularly those security teams using Azure Sentinel as their SIEM of choice.
The Notebook feature in Azure Sentinel is built on Jupyter Notebooks which is an open-source web application that allows anyone to create and share documents that contain live code, equations, visualizations, and narrative text. Its name is derived from the scripting language it is based on: JUlia, PYThon, and R.
The Jupyter Notebooks service has gained its popularity in various data science and scientific computing communities such as Genome research, Astronomy, finance, and stock market predictions, among others. It's effective and reliable proficiency to dynamically parse and present data enabled a logical pathway of interest to the cybersecurity field and has increasingly become a key tool for cybersecurity operations.
In Why Use Jupyter for Security Investigations?, @ianhelle provides some great context for using Jupyter Notebooks for cybersecurity operations including the capability for accessing and including external data, providing a true scripting and programming environment, and providing a set of steps that are restartable and repeatable.
Think of a Notebook like OneNote on steroids. Just like OneNote, you can store valuable information like text and pictures, but in the case of Notebooks, that data is interactive.
In short…
- Notebooks can be artifact storage – data persistence, repeatability and backtracking allows analysts to collect and store evidence and collateral to improve response the next time a similar event occurs.
- Notebooks can be interactive – storing more than just pieces of information, Notebooks can process the scripts it stores and produce data results inline to be used to build a more efficient and more intelligent approach to investigations and hunting.
- Notebooks can be interoperable – Notebooks enable deeper programmatic abilities to connect to, store, and use external data to be used dynamically.
- Notebooks can be guides – through sophisticated data processing, machine learning, and visualization, Notebooks guide analysts through every step of an investigation or hunt to expose, mitigate, and remediate threats to the environment.
Notebooks in Azure Sentinel extend the capabilities of the overall product. Out-of-the-box, every Azure Sentinel instance comes with several ready-made Notebooks that provide use cases for things like:
- Scanning for credential leaks in your database environment
- Detecting malicious base64-encoded commands on Linux hosts
- Generating baselines in network activities
To have a better understanding of who needs to use which “book” and when, the following table breaks these areas down side-by-side per suggested role along with providing the uses and pros and cons for each.
|
Playbooks |
Workbooks |
Notebooks |
|
|
Roles |
• SOC engineers • Analysts of all tiers |
• SOC engineers • Analysts of all tiers • SOC managers |
• Threat hunters/Tier 2-3 analysts • Incident investigators • Cyber data scientists • Security researchers |
|
Uses |
Automation of simpler, repeatable tasks:
|
|
|
|
|
|
|
|
Cons |
• Not suitable for ad-hoc & complex chain of tasks • Not great for documenting & sharing evidence |
• Cannot integrate with external data |
• Higher learning curve - requires coding knowledge * • Limited automated execution (this should be improved in the next few months) |
* Anyone can use our built-in Notebooks without coding knowledge. But the additional skillsets are involved in taking Notebooks to an advanced level. This is one of the reasons for the effort behind this Azure Sentinel Notebook Ninja series, but also a big reason for an upcoming public-facing, free training series for Azure Sentinel Notebooks.
The actual date for the training has yet to be announced, but you can already register to attend and be notified when the date has been set.
To register visit https://aka.ms/NotebookTraining and fill out the form.
We are super-excited to be bringing this series (and the training) to you! Look for more great knowledge on Azure Sentinel Notebooks as we supply new installments of this series.
Stay tuned!
More reading/tutorial resources:
- Blog: Why Jupyter notebooks are a key tool to SecOps
- Webinar: How to get started - Azure Sentinel notebooks
- Webinar: Software-defined monitoring - Using automated notebooks and Azure Sentinel to improve SecOps
- Webinar: Customizing Azure Sentinel with Python - MSTICPy and Jupyter Notebooks
- Notebook examples on the Azure Sentinel GitHub Repository
- Project Jupyter
Special thanks to my dear colleague, @rodtrent, for his major collaboration on this series and drafting this post!
Posted at https://sl.advdat.com/3Bx9U2v
