Pre-requirements:
Before starting, you must have an existing Batch account.
Before we start:
The aim of this blog is to show you how to create a Batch pool using PowerShell command (PowerShell command to create Batch pool) with several features enabled:
Pre-Setup (Step 1 – 5): Setup subscription and get Batch context.
Scenario 1 (Step 6 - 12) (Optional): Add Batch pool to the Azure Virtual Network with or without public IP address: Create an Azure Batch pool with specified public IP addresses and Create an Azure Batch pool without public IP addresses.
Scenario 2 (Step 13 - 15) (Optional): Add application package to the Batch pool: Deploy applications to compute nodes with Batch application packages.
Scenario 3 (Step 16 - 17) (Optional): Add certificate to the Batch pool: Security with certificates.
Scenario 4 (Step 18) (Optional): Add user account to the Batch pool: Run tasks under user accounts in Batch.
Scenario 5 (Step 19 - 22) (Optional): Add start task to the Batch pool: Run start task in a pool.
Scenario 6 (Step 23) (Optional): Mounting the file system to the Batch pool: Mount Azure file share with PowerShell.
Scenario 7 (Step 24 - 25): Given the image reference (built-in image and custom image) to the Batch pool: Use the Azure Compute Gallery to create a custom image pool.
Create Batch pool (Step 26): Create a Batch pool with all configurations from above scenarios.
You can skip any of the above optional scenarios (Scenario 1-6) but remember to remove the related reference when creating the pool from last step (Step 26).
Steps to create Batch pool:
Please modify the following parameters as per your own requirement.
Pre-Setup
1. Login with your own account.
Connect-AzAccount
2. Provide Batch name, resource group name, subscription id for later use.
$batchResourceGroupName = "xxxxx"
$batchAccountName = "xxxxx"
$subscriptionId = "xxxxx"
3. Here we will select the target subscription where the Batch account locates.
Select-AzSubscription -Subscription $subscriptionId
4. Get the current Batch context information. This parameter will be used all the time in below commands.
$context = Get-AzBatchAccount -ResourceGroupName $batchResourceGroupName -AccountName $batchAccountName
Creating Batch pool with different feature enabled:
Now, we will create Batch pool. The below scenarios will show you several features that can be added during the creation.
5. Given a pool name.
$poolName = "xxxxx"
Scenario 1 (Optional): Add Batch pool to the Azure Virtual Network with or without public IP address.
6. We would recommend to create a new subnet for Batch pool.
$virtualNetworkName = "xxxxx"
$subnetName = "xxxxx"
7. In my example, I created a new virtual network and a new subnet for testing.
$batchSubnet = New-AzVirtualNetworkSubnetConfig -Name $subnetName -AddressPrefix "172.23.0.0/24"
New-AzVirtualNetwork -Name $virtualNetworkName -ResourceGroupName $batchResourceGroupName -Location $context.Location -AddressPrefix "172.23.0.0/16" -Subnet $batchSubnet
8. We will save the virtual network object in a parameter.
$virtualNetwork = Get-AzVirtualNetwork -ResourceGroupName $batchResourceGroupName -Name $virtualNetworkName
$subnetId = "/subscriptions/"+$subscriptionId+"/resourceGroups/"+$batchResourceGroupName+"/providers/Microsoft.Network/virtualNetworks/"+$virtualNetworkName+"/subnets/"+$subnetName
9. Now we will configure the settings to add pool to virtual network, here we have two option: with public IP address (step 10) and without public IP address (step 11 and step 12):
10. If you would like to add pool to the virtual network with public IP address:
$vnetConfig = New-Object Microsoft.Azure.Commands.Batch.Models.PSNetworkConfiguration
$pip = New-Object Microsoft.Azure.Commands.Batch.Models.PSPublicIPAddressConfiguration -ArgumentList @("BatchManaged")
$vnetConfig.publicIPAddressConfiguration = $pip
$vnetConfig.SubnetId = $subnetId
11. Note that you must disable the private link and endpoint network policies before adding the virtual network without public IP address, otherwise, you may encounter following error message. Please refer this document for more information to disable the network policy: Disable subnet private endpoint policies.
Update the virtual network to disable private link and endpoint policies.
($virtualNetwork | Select -ExpandProperty subnets | Where-Object {$_.Name -eq $subnetName} ).privateLinkServiceNetworkPolicies = "Disabled"
($virtualNetwork | Select -ExpandProperty subnets | Where-Object {$_.Name -eq $subnetName}).PrivateEndpointNetworkPolicies = "Disabled"
$virtualNetwork | Set-AzVirtualNetwork
You will see the policies are disabled in the result.
12. Setup the network configuration within the pool.
$vnetConfig = New-Object Microsoft.Azure.Commands.Batch.Models.PSNetworkConfiguration
$pip = New-Object Microsoft.Azure.Commands.Batch.Models.PSPublicIPAddressConfiguration -ArgumentList @("NoPublicIPAddresses")
$vnetConfig.publicIPAddressConfiguration = $pip
$vnetConfig.SubnetId = $subnetId
Scenario 2 (Optional): Add application package to the Batch pool.
13. We need to firstly create an application package in the Batch account. Note that you may need to link an Azure Storage account to your Batch account, otherwise, you may encounter the following error. Please refer to this document to create a link between Azure Batch and Azure Storage: Link a Storage Account.
14. Please fill out your own application name, application path, and application version. And then create a new package in the Batch account.
$applicationName="mountFile"
$applicationPath= "C:\Users\mos\Desktop\mountFile.zip"
$applicationVersion= "1"
New-AzBatchApplicationPackage -AccountName $batchAccountName -ResourceGroupName $batchResourceGroupName -ApplicationName $applicationName -ApplicationVersion $applicationVersion -FilePath $applicationPath -Format "zip"
15. Setup the package reference configuration within the pool.
$applicationPackageReference = New-Object Microsoft.Azure.Commands.Batch.Models.PSApplicationPackageReference
$applicationPackageReference.ApplicationId=$applicationName
$applicationPackageReference.Version=$applicationVersion
$applicationPackageArrayReference = New-Object System.Collections.Generic.List[Microsoft.Azure.Commands.Batch.Models.PSApplicationPackageReference]
$applicationPackageArrayReference.Add($applicationPackageReference)
Scenario 3 (Optional): Add certificate to the Batch pool.
16. Again, we need to firstly add the certificate in the Batch account. Please fill out with your certificate path and certificate password.
$certificatePath="C:\Users\mos\Desktop\xxxxx.pfx"
$certPwd = "xxxxx"
$securePassword=ConvertTo-SecureString $certPwd –asplaintext –force
$rawData = [System.IO.File]::ReadAllBytes($certificatePath)
New-AzBatchCertificate -RawData $rawData -Password $securePassword -BatchContext $context
17. Setup the certificate reference configuration within the pool.
$cert = Get-AzBatchCertificate -BatchContext $context
$certificateReference = New-Object Microsoft.Azure.Commands.Batch.Models.PSCertificateReference
$certificateReference.Thumbprint = $cert.Thumbprint
$certificateReference.ThumbprintAlgorithm = $cert.ThumbprintAlgorithm
$certificateReference.StoreLocation = "LocalMachine"
$certificateReference.StoreName = "My"
$certificateReference.Visibility = "StartTask, Task, RemoteUser"
$certificateArrayReference = New-Object System.Collections.Generic.List[Microsoft.Azure.Commands.Batch.Models.PSCertificateReference]
$certificateArrayReference.Add($certificateReference)
Scenario 4 (Optional): Add user account to the Batch pool.
18. Setup the user account configuration within the pool. Please fill out with you own account name and password.
$userAccount = New-Object Microsoft.Azure.Commands.Batch.Models.PSUserAccount -ArgumentList @("moshitest", "xxxxx")
$userAccount.ElevationLevel = "Admin"
$userAccount.WindowsUserConfiguration = New-Object Microsoft.Azure.Commands.Batch.Models.PSWindowsUserConfiguration -ArgumentList @("Interactive")
Scenario 5 (Optional): Add start task to the Batch pool.
19. Please fill out with your own command.
$startPoolTask="cmd /c echo hello"
$startTaskReference = New-Object Microsoft.Azure.Commands.Batch.Models.PSStartTask
20. Here I will use the user account created at scenario 4 (step 18) to execute the start task.
$userIdentity = New-Object Microsoft.Azure.Commands.Batch.Models.PSUserIdentity -ArgumentList($userAccount.Name)
21. (Optional) If you would like to use auto account, you can use the below setting.
$userIdentity = New-Object Microsoft.Azure.Commands.Batch.Models.PSAutoUserSpecification -ArgumentList @("Pool", "Admin")
22. Setup the start task configuration.
$startTaskReference.CommandLine = $startPoolTask
$startTaskReference.UserIdentity= $userIdentity
$startTaskReference.WaitForSuccess=$true
$startTaskReference.MaxTaskRetryCount=1
Scenario 6 (Optional): Mounting the file system to the Batch pool.
23. Set up the mount configuration within the pool. Please fill out with Storage account name, Storage File URL, and Storage access key.
$fileShareConfig = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSAzureFileShareConfiguration" -ArgumentList @("moshibatchstorage", "https://moshibatchstorage.file.core.windows.net/xxxxx", "S", "u9SuzbDN2ne+xxxxxxx==")
$mountConfig = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSMountConfiguration" -ArgumentList @($fileShareConfig)
The following commands are only used for testing connection after the pool is created with mounting file, you could add a task to the pool and run below PowerShell command to connect the file from the pool node.
$connectTestResult = Test-NetConnection -ComputerName moshibatchstorage.file.core.windows.net -Port 445
if ($connectTestResult.TcpTestSucceeded) {
# Save the password so the drive will persist on reboot
cmd.exe /C "cmdkey /add:`"moshibatchstorage.file.core.windows.net`" /user:`"localhost\moshibatchstorage`" /pass:`"u9SuzbDN2ne+xxxxxxx==`""
# Mount the drive
New-PSDrive -Name Z -PSProvider FileSystem -Root "\\moshibatchstorage.file.core.windows.net\xxxxx" -Persist
} else {
Write-Error -Message "Unable to reach the Azure storage account via port 445. Check to make sure your organization or ISP is not blocking port 445, or use Azure P2S VPN, Azure S2S VPN, or Express Route to tunnel SMB traffic over a different port."
}
Scenario 7: Given the image reference to the Batch pool.
This blog is using Virtual Machine Configuration but you could use either Cloud Service Configuration (Example for Cloud Service Configuration) or Virtual Machine Configuration at this step.
24. You could use built-in image reference when creating a pool.
$imageReference = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSImageReference" -ArgumentList @("WindowsServer", "MicrosoftWindowsServer", "2019-Datacenter", "latest")
$configuration = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSVirtualMachineConfiguration" -ArgumentList @($imageReference, "batch.node.windows amd64")
25. (Optional) If you would like to use a custom image. You may need to provide the image resource ID as below.
$nodeAgent = "batch.node.ubuntu 20.04"
$imageId="/subscriptions/xxxxxxx/resourceGroups/moshiforubuntu/providers/Microsoft.Compute/galleries/moshiforubuntusig/images/moshiforubuntud/versions/0.1.3"
$imageReference = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSImageReference" -ArgumentList @($imageId)
$configuration = New-Object -TypeName "Microsoft.Azure.Commands.Batch.Models.PSVirtualMachineConfiguration" -ArgumentList @($imageReference, $nodeAgent)
Create Pool with above configuration.
26. Now, we have prepared all configuration that we need so we could run command to create the pool.
New-AzBatchPool -Id $poolName `
-VirtualMachineSize "standard_Ds1_v2" `
-VirtualMachineConfiguration $configuration `
-TargetDedicatedComputeNodes 1 `
-NetworkConfiguration $vnetConfig `
-ApplicationPackageReferences $applicationPackageArrayReference `
-CertificateReferences $certificateArrayReference `
-StartTask $startTaskReference `
-UserAccount $userAccount `
-MountConfiguration @($mountConfig) `
-BatchContext $context
You may access to the Azure Portal to check the pool you just created. You could click the ‘Json view’ button under ‘Properties’ blade to check all the settings.
Posted at https://sl.advdat.com/3pCy7RC
