Send notification on a Microsoft Teams channel from a Data Factory pipeline

There is often a need to send notifications during or after job execution. It helps with proactive alerting and reduces the need for reactive monitoring to find out the issues. We had published ways to send email notifications using logic apps that the Data Factory pipeline can invoke.

 

Many enterprises are leveraging Microsoft Teams for collaboration; this helps you easily integrate yet another critical thing: proactive pipeline alerts into Microsoft teams. 

Here is how to send a notification on a teams channel from an Data Factory pipeline.

 

Prerequisite:

• Create an incoming webhook in your Microsoft Teams channel. If you are new to Microsoft Teams or would like to create a new Teams channel for this purpose, please refer to the Teams documentation.  

1. Open Microsoft Teams and go to Apps > Search for 'Incoming Webhook' and select the 'Incoming Webhook' connector.
   

    

2. Add the connector to the Team site where you want to send the notifications. 
   

    

3. Name the Webhook as appropriate and Optionally upload an icon to identify your messages.
   

    

4. Copy the store of the Webhook URL that is generated on creation for later use in ADF.
   

    

5. You will see the notification in the channel where you add the Webhook connector. 
   

    

Steps to send notifications on Teams channel from a Data Factory pipeline:

1. Create a new 'pipeline from template'. We have added a pipeline template which will make it easier to get started with teams notifications. 
   

    

2. Search for teams, select and use 'Send notification to a channel in Microsoft Teams' template. 
   
   
   
   
   

    

3. It is recommended to add the current Data Factory's Subscription ID, Resource Group, and the Teams webhook URL (refer prerequisite) under the default value of the relevant parameter placeholders (<update value>).    
   

    

   Note: It is used to construct the monitoring URL. Suppose you do not provide a valid subscription and resource group (of the same data factory where the pipelines belong). In that case, the notification will not contain a valid pipeline monitoring URL, but the messages will still work. 
   Also, it helps prevent the need to always passing those values from another pipeline. But if you intend to control those value through a metadata-driven approach, then you should modify accordingly. 
   
   
4. Add an 'Execute Pipeline' activity into the pipeline from which you would like to send notifications on the Teams channel. Select the pipeline generated from the 'Send notification to a channel in Microsoft Teams' template as the 'Invoked pipeline' in the 'Execute Pipeline' activity. 
   

    

5. Customize the parameters accordingly based on activity type. 
   

    

Note: You can leverage ADF system variables and expressions to make your messages dynamic. As an example -  

• @activity('CopyData').output.errors[0].Message
• @activity('DataFlow').error.Message expression 

The above expressions will get you the relevant error messages which can be sent out as notification on a Teams channel. 

You can find additional 'Copy' activity output properties here.

 

You're encouraged to look into the Microsoft Teams supported notification payload schema and further customize the above template to your needs. 

 

If you would like to be notified by the Teams application (cross-platform) through push notifications, then make sure you are subscribed to the channel notification. 

 

 

 

 

Posted at https://sl.advdat.com/2S7wgX1