Notebooks - Sentinel Code Snippets
As discussed in Part 1 of this series, Notebooks service is a powerful feature and an integral part of Microsoft Sentinel that provides additional capability to help augment your analysis during threat hunting, incident triage and investigation.
When creating your custom notebook, you can leverage the Sentinel code snippets to quickly add the foundation structure to set up the environment for MSTICPy, configure the parameters, authenticate into Azure and Log Analytics, and query the data.
Four code snippets are currently available for Sentinel Notebooks:
- Get Configuration parameters
- Set up environment for msticpy
- Authenticate into Azure resources
- Authenticate into Azure Log Analytics
You can access the code snippets by simply typing the key-word “Sentinel” in the Notebook code cell.
Let’s explore each code snippet individually. You can also refer to a sample custom notebook using the code snippets and follow the tutorial below to practice along.
Set up environment for msticpy
The “Set up environment for msticpy” code snippet allows you to quickly add the needed code to install and configure MSTICPy into the Notebook Kernel.
MSTICPy is a Python library for InfoSec investigation and hunting in Jupyter Notebook developed by Microsoft Threat Intelligence Center (MSTIC) team - Read more about MSTICPy at “MSTIC Jupyter and Python Security Tools”.
#####################################################
# Sentinel - Set up environment for msticpy #
#####################################################
# import some modules needed in this cell
from pathlib import Path
from IPython.display import display, HTML
display(HTML('Checking upgrade to latest msticpy version'))
%pip install --upgrade --quiet msticpy
REQ_PYTHON_VER=(3, 6)
REQ_MSTICPY_VER=(1, 2, 3)
REQ_MP_EXTRAS = ['keyvault']
display(HTML('<h3>Starting Notebook setup...</h3>'))
if Path('./utils/nb_check.py'):
from utils.nb_check import check_versions
check_versions(REQ_PYTHON_VER, REQ_MSTICPY_VER)
# intialize msticpy
from msticpy.nbtools import nbinit
nbinit.init_notebook(
namespace=globals(),
extra_imports=['urllib.request, urlretrieve']
)
pd.set_option('display.html.table_schema', False)
At completion of the above tasks, you’ll have MSTICPy installed. One caveat, the code line “%pip install --upgrade --quiet msticpy” installs MSTICPy with default most functionality (approx 75%) and Kqlmagic Jupyter basic. If you wish to install specific functionality, add it explicitly into the command code (i.e. %pip install --upgrade --quiet msticpy[azsentinel]).
A few of the MSTICpy functionalities:
|
extra |
Functionality |
|
[none] |
|
|
keyvault |
|
|
azure |
|
|
kql |
|
|
azsentinel |
|
|
ml |
|
|
splunk |
|
|
vt3 |
|
|
all |
|
|
dev |
|
|
test |
|
Get Configuration parameters
The “Get Configuration parameters” code snippet enables loading the configuration parameters from the config.json file. The config.json file is automatically generated and properly configured when you launch a notebook from Sentinel to an Azure ML studio. The config.json file will have all the Sentinel workspace’s information such as the Tenant ID, workspace ID and both the user alias and object ID, etc.
The “Get Configuration parameters” Code snippet will read and import in memory the information stored in the config.json, so that the subsequent code cells can reference it, for example in authentication to your Log Analytics workspace.
####################################################
# Sentinel - Get Sentinel Configuration parameters #
####################################################
import json
def read_config_values(file_path):
'This loads pre-generated parameters for Microsoft Sentinel Workspace'
with open(file_path) as json_file:
if json_file:
json_config = json.load(json_file)
return (json_config['tenant_id'],
json_config['subscription_id'],
json_config['resource_group'],
json_config['workspace_id'],
json_config['workspace_name'],
json_config['user_alias'],
json_config['user_object_id'])
return None
# Calling the above function to populate Sentinel workspace parameters
# The file, config.json, was generated by the system, however, you may modify the values, or manually set the variables
tenant_id, subscription_id, resource_group, workspace_id, workspace_name, user_alias, user_object_id = read_config_values('config.json')
In the event you are missing the config.json, create a new file under your directory and paste the following by editing the information as relates to your Azure Subscription, Tenant ID, Log Analytic Workspace used by Sentinel, your account Alias and your account ID.
{
"tenant_id":"<Azure_Active_Directory_Tenant_Id>",
"subscription_id":"<Azure_Subscription_Id?",
"resource_group":"<Sentinel_Resource_Group>",
"workspace_id":"<Sentinel_Workspace_id>",
"workspace_name":"<Sentinel_Worspace_Name>",
"user_alias":"<User_Alias_With_Permissions_to_Access_Sentinel>",
"user_object_id":"<User_Object_Id>"
}
Authenticate into Azure resources
The “Authenticate into Azure resources” code snippet is a handy and simple line of code that prompts the user to login and authenticate into Azure. Once authenticated the notebook can communicate with your Azure resources.
####################################################
# Sentinel - Authenticate into Azure resources #
####################################################
# Azure CLI is used to get device code to login into Azure, you need to copy the code and open the DeviceLogin site.
!az login --use-device-code
Authenticate into Azure Log Analytics
This code snippet allows you to authenticate for the specific Log Analytics workspace associated to the Sentinel instance to which you want to query the data from.
The snippet has the same sample commented code that you can use to test the accessibility by running a KQL query against the workspace.
####################################################
# Sentinel - Authenticate into Azure Log Analytics #
####################################################
# Azure CLI is used to get device code to login into Azure, you need to copy the code and open the DeviceLogin site.
# !!! You need [tenant_id] and [subscription_id] to login into Azure !!!
from azure.common.client_factory import get_client_from_cli_profile
from azure.common.credentials import get_azure_cli_credentials
from azure.loganalytics import LogAnalyticsDataClient
from azure.mgmt.loganalytics import LogAnalyticsManagementClient
from azure.loganalytics.models import QueryBody
!az login --use-device-code
la_client = get_client_from_cli_profile(LogAnalyticsManagementClient, subscription_id = subscription_id)
creds, _ = get_azure_cli_credentials(resource='https://api.loganalytics.io')
la_data_client = LogAnalyticsDataClient(creds)
# Query sample:
# query = 'union withsource = SentinelTableName * | distinct SentinelTableName | sort by SentinelTableName asc'
# result = la_data_client.query(workspace_id, QueryBody(query=query))
# print(result.as_dict())
Summary
Regardless, if you are a seasoned developer or simply a newbie, having a library of existing pre-built code that you can quickly import into your notebooks and use it, is useful for efficient programming. Almost all engineers and developers use code snippets in their daily work to save time and avoid having to type repetitive code.
These are the four code snippets to help you get started in building or customizing your own Sentinel notebooks. We continue expanding the library so stay tuned!
Further resources:
- For sample or template Notebooks please refer to the Microsoft GitHub repository at Azure-Sentinel-Notebooks/MSTICPy-TutorialNotebooks at master · Azure/Azure-Sentinel-Notebooks · GitHub
- MSTICPY documentation
- Create Azure Sentinel Custom Notebooks with Code Snippets - YouTube
Posted at https://sl.advdat.com/3cb3RWF
