Security considerations for Azure Kubernetes Service

Kubernetes is a complex system with multiple components working in tandem. Though AKS is a managed Kubernetes offering, it still requires operators to take care of the security across the components. In this context, the document outlines the various security measures recommended across the Host, Cluster, API Server, Pods, and the supply chain for images.

 

Access Control

Authentication and Authorization are the primary mechanism to restrict access to a cluster. It is essential to use strong authentication and authorization to limit user and administrator access as well as to limit the attack surface.

 

Authentication

• AKS-managed Azure AD integration simplifies the Azure AD integration process
• Local accounts are enabled by default, and it is good best practice to disable them on an AAD enabled cluster
• In the event where local accounts are required, it is recommended to limit access to the kubeconfig file

Authorization

• AKS supports Azure RBAC as well as Kubernetes RBAC for authorization
• Leverage Azure RBAC for all identities that exist inside Azure AD, and Kubernetes RBAC for those outside of AAD (i.e., Kubernetes Service Accounts)
• Leverage Azure RBAC built-in roles

Identity

• Though AKS supports Service principals it is recommended to use Managed Identities for simplified management.
• The identity could be system-assigned or user-assigned (BYO)
• User assigned identities can be managed separately and works great with IaC
• Control Plane identity controls the creation of supporting Azure resources like load balancers, disks etc.
• Kubelet Identity controls authenticating with ACR
• Enable AAD Pod Identities to let workloads access cloud resources securely (i.e. KeyVault, databases etc.)

 

Traffic Control

Use firewalls to limit unneeded network connectivity.

• Namespace should be used to define boundaries to apply security controls in an easy manner
• Do not deploy any application workloads to the kube-* namespaces
• Network policies control East-West traffic (i.e. pod to pod communication) through Network policies applied at the namespace level.
• Avoid public exposure to the API server using private AKS cluster or limit exposure by using the authorized IP ranges 
• Control Egress traffic through a firewall

             

Encryption

Consider encryption to protect confidentiality

• By default, all disks in AKS are encrypted at rest with Microsoft-managed keys. Consider BYO Keys for control over encryption keys. 
• AKS encrypts Etcd at rest and does not allow a KMS integration to bring your own key.
• Use Sealed Secret to store the secrets in source control (for ex. gitops)
• Use Azure Key Vault secrets in Azure DevOps pipelines or Github Actions
• Use TLS for all ingress definitions through a trusted CA like Let's Encrypt

 

Container Security    

Kubernetes cluster security involves securing the host, cluster, and pods. Each of these layers should be hardened. Apart from it, the container image supply chain should be secured by scanning for known vulnerabilities and misconfigurations.

 

Host/Cluster

• Azure Defender for Kubernetes provides both Host level and cluster level protection
• Host Level has a limitation with AKS cluster on VMSS as there is a dependency on Log Analytics agent which is currently not supported in this setup.  An enhanced daemonset approach will be released soon to address this limitation.
• Integrated alerts for Cluster level and Host Level
• Advanced analysis through AKS data connector for Azure Sentinel

Workload protection

• Protection enabled through Kubernetes admission control via Azure Policy
• Ensure Policy Add-on is auto provisioned on AKS clusters
• Review and Configure the built-in policy bundle
• Pod Security Standard Considered to be the replacement for PSP. Review and deploy the built-in baseline and restricted initiatives
• Library of policies and tests that can be used with Custom Policy

Image Security

• Static image scanning for all images in ACR. Image scans are performed by Qualys and includes all images when they’re pushed to the registry, imported into the registry, or pulled within the last 30 days.
• Shift left in image scanning by integrating scans in the CI/CD pipelines. Azure Defender's CI/CD scanning is powered by Aqua Trivy
• Task available for GitHub workflows. ADO tasks are in the roadmap
• Continuous image scanning and Drift detection is in development
• Use opensource Falco or other commercial offerings for immediate needs

Pod Security

• Explicitly control the resource utilization of the pods
• Ensure that the containers are run in a specific security context
  • privileged: ability to run privileged container should be false unless required
  • hostPID, hostIPC: ability to share host process namespace should be false unless required
  • hostNetwork: ability to use host network namespace should be false unless required
  • allowedHostPaths: restricts to host paths and should be set to a dummy path for it to be effective
  • readOnlyRootFilesystem: Should be set to true by default and exempt writes as required
  • runAsUser: set to a non-root use
  • runAsGroup: set group to avoid defaulting to root i.e., 0
  • fsGroup: set group to specify ownership of files
  • allowPrivilegeEscalation: defines if the pod can assume root privileges
  • seLinux labels: lets you define access policies for services, processes, and filesystem access
  • AppArmor Profiles: Limit container actions
  • seccomp Profiles: filter the syscalls allowed
• Enforce the security context to Policy initiatives
• Automation of loading the AppArmor and SecComp profiles should ensure that they are applied to the nodes before any pods get scheduled. Consider a daemonset approach to overcome the race condition.
• Opt out of automounting API credentials for a service account by setting automountServiceAccountToken: false on the service account

Audit

Kubernetes emits variety of logs across various components, and it is essential to log, monitor and alert on all of these to identify potential threats. Monitoring the logs ensures that the services are secure and operating as intended. 

 

Monitoring

• Effective logging and monitoring is essential to secure any environment, and enabling Container Insights on an AKS cluster provides the required observability.
• It is recommended to monitor the below logs

                   Logs

                   Events

                   Metrics

                   API Server Logs

Alerts

• Enable the recommended metrics alerts bundle
• Create log alert rules for Resource utilization and Resource availability
• Create custom alerts as required. Refer sample log queries for table information

Conclusion

Securing the complete surface of the Kubernetes is essential and the above guidance provides a decent coverage. Though most of the concepts are specific to azure services, the security considerations are generic and applies to any Kubernetes distribution. Also, the concepts can be implemented through third party/OSS components in an AKS cluster

Posted at https://sl.advdat.com/3BDMBnb