Azure Kubernetes Services (AKS) is a managed Kubernetes service on Microsoft Cloud Azure that can be used to rapidly deploy Kubernetes clusters, integrate with other Azure services and features to achieve simple operations and elastic business applications. The Kubernetes community is booming, and version upgrades are frequent. AKS manages version upgrade so that AKS clusters can be upgraded at the click of a button, rolling through the working nodes within a cluster. This makes it easy to upgrade AKS for small or non-mission-critical clusters. However, the in-place upgrade method is not very suitable for large-scale clusters or mission-critical applications, because it has a long overall upgrade time and the possibility of upgrade failure. And the failure cannot be rolled back. The safest approach is to create a new AKS cluster, switch upgrades using blue-green deployments. This approach can switch in seconds, and roll back the backup cluster at any time, which is faster and safer. Correspondingly, the architecture and operation of this approach is more complex. This article leads you to gradually build such an architecture scheme. This time we're using a scenario for a classic Web application, which can be expanded into other scenarios.
Architecture overview
This is a classic Web application architecture diagram, and we've selected the most streamlined resources to keep the demonstration and instructions as concise as possible.
Place 2 subnets in a virtual network, and place the application gateway in the first subnet as a load balancer for external services.
The second subnet has only 1 AKS cluster daily and deploys the business system. The AKS cluster uses advanced network CNI to simplify network patterns and facilitate application gateway communication with them.
Deploy Pod Identity is deployed in the AKS cluster to authorize pods in AKS to manage the application gateway through Azure AD's Pod Identity.
AGIC, known as Application Gateway Ingress Controller, not only distributes network traffic from the app gateway to the appropriate Pod, but also monitors changes in some Kubernetes resources, automatically updates the back-end pool of the app gateway as AKS scales, and updates the back-end pool of the app gateway synchronously as the AKS cluster switches. The core idea for our blue-green deployment is dynamically updating the back end of the app gateway with AGIC.
Resources Deployment
Basic resources such as the network
We quickly create the underlying resources of the current environment with CLI. Resource group:
AZ_REGION=ChinaNorth2
RESOURCE_GROUP=AKS_Upgrade
az group create -n $RESOURCE_GROUP -l $AZ_REGION
Create VNET and subnets.
VNET_NAME=AksVnet
APPGW_SUBNET=AppGwSubnet
AKS_SUBNET=AksSubnet
network vnet create on $VNET_NAME \
-g $RESOURCE_GROUP \
-l $AZ_REGION \
--address-prefix 10.0.0.0/8 \
--subnet-name $APPGW_SUBNET --subnet-prefix 10.1.0.0/16
az network vnet subnet create \
-g $RESOURCE_GROUP \
-n $AKS_SUBNET \
--address-prefixes 10.240.0.0/16 \
--vnet-name $VNET_NAME
The current AKS cluster
Create a public IP.
APPGW_IP=AppGatewayIp
az network public-ip create -n $APPGW_IP \
-g $RESOURCE_GROUP \
--allocation-method Static \
--sku Standard
Create an app gateway.
APP_GATEWAY=AppGateway
az network application-gateway create -n $APP_GATEWAY \
-g $RESOURCE_GROUP \
-l $AZ_REGION \
--vnet-name $VNET_NAME \
--subnet $APPGW_SUBNET \
--sku Standard_v2 \
--public-ip-address $APPGW_IP
Create an old AKS cluster with the current default mainstream AKS version. Get the subnet ID of the AKS cluster we created earlier.
AKS_SUBNET_ID=$(az network vnet subnet show -g $RESOURCE_GROUP --vnet-name $VNET_NAME --name $AKS_SUBNET --query id -o tsv)
Create the old AKS cluster.
AKS_OLD=old
az aks create -n $AKS_OLD \
-g $RESOURCE_GROUP \
-l $AZ_REGION \
--generate-ssh-keys \
--network-plugin azure \
--enable-managed-identity \
--vnet-subnet-id $AKS_SUBNET_ID
At the time of writing, the mainstream AKS version is 1.19.11.
The app gateway integrates with the current version of AKS
We use the Azure service principal to authorize the AKS cluster to manage the configuration of the application gateway.
Connect to the AKS cluster
az aks get-credentials --resource-group $RESOURCE_GROUP --name $AKS_OLD
We can then manage the AKS cluster with the familiar kubectl.
Install Helm and run the following command to add the application-gateway-kubernetes-ingress Helms package. Our AKS cluster has enabled Kubernetes RBAC, so use the following command.
kubectl create serviceaccount --namespace kube-system tiller-sa
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller-sa
helm repo add aad-pod-identity https://raw.githubusercontent.com/Azure/aad-pod-identity/master/charts
helm install aad-pod-identity aad-pod-identity/aad-pod-identity
The command will return:
NAME: aad-pod-identity
LAST DEPLOYED: Tue Jun 29 08:14:30 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed AAD Pod Identity in your Kubernetes cluster!
…
Wait a minute or two and use
kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
aad-pod-identity-mic-787c5958fd-kmx9b 1/1 Running 0 71s 10.240.0.33 aks-nodepool1-94448771-vmss000000 <none> <none>
aad-pod-identity-mic-787c5958fd-nkpv4 1/1 Running 0 72s 10.240.0.63 aks-nodepool1-94448771-vmss000001 <none> <none>
aad-pod-identity-nmi-mhp86 1/1 Running 0 72s 10.240.0.4 aks-nodepool1-94448771-vmss000000 <none> <none>
aad-pod-identity-nmi-sjpvw 1/1 Running 0 72s 10.240.0.35 aks-nodepool1-94448771-vmss000001 <none> <none>
aad-pod-identity-nmi-xnfxh 1/1 Running 0 72s 10.240.0.66 aks-nodepool1-94448771-vmss000002 <none> <none>
See that several of the related pods are already running.
Install Application Gateway Ingress Controller with helm.
helm repo add application-gateway-kubernetes-ingress https://appgwingress.blob.core.windows.net/ingress-azure-helm-package/
helm repo update
Copy the following YAML file and save it as helm_agic.yaml to configure AGIC.
# This file contains the essential configs for the ingress controller helm chart
# Verbosity level of the App Gateway Ingress Controller
verbosityLevel: 3
################################################################################
# Specify which application gateway the ingress controller will manage
#
appgw:
subscriptionId: <subscriptionId>
resourceGroup: <resourceGroupName>
name: <applicationGatewayName>
environment: AzureChinaCloud
# Setting appgw.shared to "true" will create an AzureIngressProhibitedTarget CRD.
# This prohibits AGIC from applying config for any host/path.
# Use "kubectl get AzureIngressProhibitedTargets" to view and change this.
shared: false
################################################################################
# Specify which kubernetes namespace the ingress controller will watch
# Default value is "default"
# Leaving this variable out or setting it to blank or empty string would
# result in Ingress Controller observing all acessible namespaces.
#
# kubernetes:
# watchNamespace: <namespace>
################################################################################
# Specify the authentication with Azure Resource Manager
#
# Two authentication methods are available:
# - Option 1: AAD-Pod-Identity (https://github.com/Azure/aad-pod-identity)
# armAuth:
# type: aadPodIdentity
# identityResourceID: <identityResourceId>
# identityClientID: <identityClientId>
## Alternatively you can use Service Principal credentials
armAuth:
type: servicePrincipal
secretJSON: <<Generate value with: "az ad sp create-for-rbac --sdk-auth | base64 -w0">>
################################################################################
# Specify if the cluster is RBAC enabled or not
rbac:
enabled: true # true/false
Let's fill in the parameter values in the above profiles one by one.
<subscriptionId>
Use command az account show --query id -o tsv to get.
<resourceGroupName>
Use the value of $RESOURCE_GROUP environment variable.
<applicationGatewayName>
Use the value of $APP_GATEWAY environment variable.
The value of secretJSON is obtained using the az ad sp create-for-rbac--sdk-auth | base64 -w0 command, which is a long string of more than 800 bytes encoded by base64.
Because our AKS cluster has RBAC enabled, the last configuration rbac is set to true.
Finally, run the following command.
helm install agic application-gateway-kubernetes-ingress/ingress-azure -f helm_agic.yaml
returns
W0629 08:16:47.733467 16087 warnings.go:70] apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
NAME: agic
LAST DEPLOYED: Tue Jun 29 08:16:48 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing ingress-azure:1.4.0.
Your release is named agic.
The controller is deployed in deployment agic-ingress-azure.
Configuration Details:
----------------------
* AzureRM Authentication Method:
- Use AAD-Pod-Identity
* Application Gateway:
- Subscription ID : 3d07553f-f6a8-455f-9de6-876fbcc00bb4
- Resource Group : AKS_Upgrade
- Application Gateway Name : AppGateway
* Kubernetes Ingress Controller:
- Watching All Namespaces
- Verbosity level: 3
And then
kubectl get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
aad-pod-identity-mic-787c5958fd-kmx9b 1/1 Running 0 4m54s 10.240.0.33 aks-nodepool1-94448771-vmss000000 <none> <none>
aad-pod-identity-mic-787c5958fd-nkpv4 1/1 Running 0 4m55s 10.240.0.63 aks-nodepool1-94448771-vmss000001 <none> <none>
aad-pod-identity-nmi-mhp86 1/1 Running 0 4m55s 10.240.0.4 aks-nodepool1-94448771-vmss000000 <none> <none>
aad-pod-identity-nmi-sjpvw 1/1 Running 0 4m55s 10.240.0.35 aks-nodepool1-94448771-vmss000001 <none> <none>
aad-pod-identity-nmi-xnfxh 1/1 Running 0 4m55s 10.240.0.66 aks-nodepool1-94448771-vmss000002 <none> <none>
agic-ingress-azure-8d9d85dd9-z8dwh 1/1 Running 0 2m37s 10.240.0.70 aks-nodepool1-94448771-vmss000002 <none> <none>
We can see that the new agic-ingress-azure pod is also working.
Posted at https://sl.advdat.com/2TEms7X