Thursday, November 11, 2021

Express Virtual Network Injection for SSIS in Azure Data Factory

If you want to lift & shift/migrate your existing SQL Server Integration Services (SSIS) packages to the cloud, so they can run on SSIS integration runtime (IR) in Azure Data Factory (ADF), you’ll need to inject/join your SSIS IR to a virtual network (VNet) in the following scenarios:

 

 

  • You want to use Azure SQL Database server/managed instance that’s configured with a private endpoint/IP firewall rule/virtual network service endpoint or managed instance that joins a VNet to store your packages in SSIS catalog database (SSISDB)/SQL Server database (MSDB).

 

  • You want to use Azure Storage that’s configured with a private endpoint/IP firewall rule/virtual network service endpoint to store your SSIS packages or custom setup files.

 

  • You want to access other Azure data stores/resources that are configured with a private endpoint/IP firewall rule/virtual network service endpoint.

 

  • You want to access other cloud data stores/resources that are configured with an IP firewall rule.

 

There are two methods for you to inject your SSIS IR into a VNet: standard and express that are in General Availability (GA) and Public Preview, respectively.  

 

To see their different networking requirements, compare the diagram for standard VNet injection method on the left-hand-side (LHS) and that for express VNet injection method on the right-hand-side (RHS):

 

standard-express-virtual-network-injection.png

 

The standard method injects both SSIS IR virtual machine (VM) and container into your VNet, while the express one uses a new networking technology that injects SSIS IR VM into Microsoft-managed VNet, but SSIS IR container into your VNet.

 

This technology allows the express method to provision/start your SSIS IR faster, since it can now leverage our pools of standby VMs that have been created and prepared in advance, see https://techcommunity.microsoft.com/t5/sql-server-integration-services/express-provisioning-of-azure-ssis-integration-runtime/ba-p/719058.

 

The express method is also enterprise-friendlier than the standard one, since it needs not create additional network resources, such as public IP address, load balancer, and network security group (NSG) at network interface card (NIC) level, inside your VNet (see the purple boxes in LHS diagram).

 

Consequently, the express method has no inbound traffic and minimal outbound traffic requirements compared to the standard one (see the incoming and outgoing arrows, into and from your VNet in LHS and RHS diagrams).

 

That said, there are still some limitations with the express method that can be worked around for now, e.g. using your own DNS server requires a standard custom setup on SSIS IR, using multiple SSIS IRs requires multiple VNets that can be peered/connected together, etc.

 

For an overview of VNet injection, see https://docs.microsoft.com/azure/data-factory/join-azure-ssis-integration-runtime-virtual-network#configure-a-virtual-network

 

For a detailed comparison of the standard and express VNet injection methods, see https://docs.microsoft.com/azure/data-factory/azure-ssis-integration-runtime-virtual-network-configuration.

 

For more information on the express VNet injection method, see https://docs.microsoft.com/azure/data-factory/azure-ssis-integration-runtime-express-virtual-network-injection.

 

For more information on the standard VNet injection method, see https://docs.microsoft.com/azure/data-factory/azure-ssis-integration-runtime-standard-virtual-network-injection.   

 

I hope you’ll find this article useful for you to inject your SSIS IR into a VNet.  Please don’t hesitate to contact us if you have any feedbacks, questions, or issues, and we’ll follow up ASAP.  Thank you as always for your support.

Posted at https://sl.advdat.com/3C85IWL