Advanced Data Solutions : Use SAP Virtual Host Names with Linux in Azure

For many customers the concept of SAP virtual hostnames is commonly used and expected for the SAP landscape. It allows logically decouple the actual virtual machine hostname and an easy to understand and unique SAP system hostname. The hostname - often a cryptic string to comply with a company’s overall naming policy (example x17qus80) – is bound to the only network interface card. SAP system running on this host, let’s say with SID of T01, operates with a hostname unique for every instance. One virtual hostname for the ASCS instance, one for PAS, another for database and each application server has one each, too. Example sapt01ascs, sapt01pas, sapt01db etc.

These virtual hostnames correspond to DNS A/PTR entries and unique IP address is used for each virtual hostname, as seen in above example. These secondary IP addresses are bound to the same network interface card, so within the operating system the same NIC has – example above – 3 individual IP addresses. The virtual hostname concept hides the physical hostname from the application and allows very easy move of the SAP instances to new VMs – for example during major OS upgrades where typically a new VM with higher OS version is deployed, using new physical/VM hostname.

Figure 1 - Example of secondary IPs for SAP

NOTE: Please note that the configuration described in this article does NOT apply to any high-availability solution using internal load balancer such as Pacemaker or other, 3^rd party cluster solutions. The load balancer fulfills the same role – virtual hostname IP is provided by the load balancer. The configuration described in this blog post can be applied to SAP Application Servers including (A)SCS, non-HA SAP systems, as well as 2-tier SAP systems (application, DB and SCS instances installed on a single VM.)

This blog post addresses this topic for Linux systems only. For Windows environments, please see the following excellent blog post on the very same subject. Use SAP Virtual Host Names with Windows on Microsoft Azure

Azure configuration for SAP virtual hostnames

Now that you went through a quick refresher how virtual hostnames are used with SAP, how does this work in Azure? On-premises you add a secondary IP within the OS and often the networking stack automatically reconfigures. With software defined networking, as utilized in public cloud, some additional steps are needed.

An Azure VM running SAP workload typically requires only one NIC card, there is no performance benefit using multiple NICs. Hence, before adding the secondary IP within the OS, the IP needs to be added to the Azure network interface first. Each Azure NIC can support up to 256 secondary IP addresses.

To add the secondary IP(s) to a network interface you modify the network interface and add IP config(s) for each secondary entry. As always, you can use the Azure portal, CLI, tools like Terraform or the Azure API itself. Secondary IPs must be in the same subnet as the network interface itself.

There is NO downtime or interruption required to add or remove secondary IP addresses from Azure network cards, this can be done online.

Figure 2 - Example of secondary IPs configured in Azure for NIC

Once configured, as shown in picture, the Azure networking stack knows to route network packets for IPs 10.185.106.5, .60 and .61 to this network interface.

That is it from Azure perspective, make sure your DNS is updated with A/PTR records for this hostname/IP. Next step is the operating system of the VM.

OS specific steps for SAP virtual hostnames

Network packets addressed to the secondary IP(s) are now sent to this NIC and VM. But the OS must know how to deal with the network packets for the SAP virtual hostname/IP , too.

Most Linux distributions operate the NIC card with DHCP by default. That means Azure provides the IP (and possibly DNS servers, if configured on Azure virtual network) to the OS. SAP relevant OS images should pick-up the secondary IP address(es) that you configured within Azure immediately through DHCP. Don’t rely on DHCP entirely, however, since for some configurations like SAP LaMa or HA setups with Pacemaker require you to disable SUSE Network Manager and thus you need to add secondary IPs manually.

Should you use the default and use DHCP, once your DNS is configured correctly and the hostname entries are added to DNS, then there should be no action needed within the OS. You can try and reach the virtual hostname from other hosts – database host, from client PCs, etc.

virthost01:~ # ip a s eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000

link/ether 00:22:48:9b:ff:71 brd ff:ff:ff:ff:ff:ff

inet 10.185.106.5/25 brd 10.185.106.127 scope global eth0

valid_lft forever preferred_lft forever

inet 10.185.106.60/25 brd 10.185.106.127 scope global secondary eth0

valid_lft forever preferred_lft forever

inet 10.185.106.61/25 brd 10.185.106.127 scope global secondary eth0

valid_lft forever preferred_lft forever

inet6 fe80::222:48ff:fe9b:ff71/64 scope link

valid_lft forever preferred_lft forever

Without the use of DHCP, you need to add the secondary IPs manually within the OS. To add them online, without any stopping of applications/OS, you can use the ip command

[root@virthost01 ~]# ip a add 10.185.106.60/25 dev eth0

[root@virthost01 ~]# ip a add 10.185.106.61/25 dev eth0

And verify (same result as if the IP is picked up with DHCP)

[root@virthost01 ~]# ip a s eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000

link/ether 00:0d:3a:b1:d4:82 brd ff:ff:ff:ff:ff:ff

inet 10.185.106.4/25 brd 10.185.106.127 scope global noprefixroute eth0

valid_lft forever preferred_lft forever

inet 10.185.106.60/25 scope global secondary eth0

valid_lft forever preferred_lft forever

inet 10.185.106.61/25 scope global secondary eth0

valid_lft forever preferred_lft forever

inet6 fe80::20d:3aff:feb1:d482/64 scope link

valid_lft forever preferred_lft forever

Using ip command is however not persistent and after each reboot or VM start the secondary IPs in such case – if not automatically handled by dhcp client inside OS – would need to be re-added. For persistent changes modify /etc/sysconfig/network-scripts/ifcfg-<adapter> (RHEL) or /etc/sysconfig/network/ifcfg-<adapter> (SLES) scripts and restart networking or the OS.

Verify the name resolution

As clearly stated in SAP note 962955 - Use of virtual or logical TCP/IP host names both forward and reverse name lookups need to return the same hostname-IP pair. Failure often leads to SAP services not starting at all.

Now, in our example just below we will be working with this setup.

Hostname: x17qus80 (10.185.106.5)

SAP hostnames: sapt01ascs (10.185.106.60) and sapt01pas (10.185.106.61)

ht1adm@x17qus80:/usr/sap/HT1/tmp> ./niping -v -H sapt01ascs

Hostname/Nodeaddr verification:

===============================

Hostname of local computer: x17qus80 (NiMyHostName)

FQHN of local computer: x17qus80.virthost.com (NiGetOwnFQDN)

Lookup of hostname: sapt01ascs (NiHostToAddr)

--> IP-Addr.: 10.185.106.60

Lookup of IP-Addr.: 10.185.106.60 (NiAddrToHost)

--> Hostname: sapt01ascs

As seen, both forward and reverse lookup return the correct values, everything is in order. Would the reverse lookup not work correctly or return a miss-matched hostname than forward name resolution, SAP message server and other services would not start.

Additional validation and troubleshooting step is to use nslookup directly for both forward and reverse lookups (hostname and IP).

Alternative - Using multiple NICs

First alternative architecture to using secondary IPs on a single NIC is to simply use more than one NIC. Very often we carry over our previous thinking and complicate system design. Using multiple NICs in Azure when its not required is one such typical example. Each NIC attached to an Azure VM must be in same virtual network but can be in different subnets. The network bandwidth limit of a virtual machine in Azure is set regardless how many network interfaces are attached, in other words you do NOT get additional network throughput with additional NICs.

Figure 3 - Example of an unsuitable solution for SAP virtual hostnames

The typical usage scenario is requirements to use network segmentation through subnets and thus dual/multi-heading VMs into two or more subnets, such as administrative and SAP-application subnet. Remember that each subnet within a vnet can talk directly, if routes and network security groups (NSGs) are not restricted.

Using multiple NICs for SAP virtual hostnames brings negatives of additional overhead for management, as each NIC requires set NSGs, user defined routes, manual routes and management within OS as dhcp clients in operating system typically only handles first NIC only you will need to set IP and routes for the second interface. It also increases complexity and more difficult troubleshooting.

Use of a single, well managed and configured (NSG, routing) network interface on a VM with secondary IP configurations/addresses as described above is the far better solution for SAP virtual hostnames, since SAP virtual hostname/IP(s) are in same subnet as actual VM. If however the additional overhead of multi-NIC is not seen as a burden, then this is a valid alternative.

Alternative - Using Azure Internal Load Balancer for secondary IP addresses

An Azure Internal Load Balancer (ILB) is the correct service to provide virtual/floating IPs in a highly available (clustered) architecture. The IP of such virtual hostnames, used by SAP, is bound to the Load Balancer which is attached to the NICs of the VMs. This is the recommended solution for such HA-systems.

Figure 4 - Example of single VM with ILB providing virtual hostname

For single-VMs a Load Balancer can be created, attached, and serve the same purpose as in HA setup or with secondary IPs on primary NIC. It is however not possible to setup a load balancer on secondary IP configurations for floating IP usage, so you cannot combine the two methods. As per the note at beginning of this blog post, load balancer should be used for any HA setup and not secondary IPs on a NIC.

The drawbacks to this architecture are higher costs, overhead of building, managing and troubleshooting when compared to architecture without ILB and using NIC with secondary IP configs attached to it. Additionally an Azure Standard ILB attached to a VM modifies how a VM can access Internet – for example OS update repositories. ILBs are typically only used for clustered systems due to these drawbacks.

Bad alternative - Using DNS alias for SAP virtual hostname

The last alternative architecture to secondary IPs on a NIC is the use of DNS aliases. Very often DNS aliases, also known as canonical names or CNAME entries, are used to resolve the SAP virtual hostname, with the alias pointing to the actual VM IP. The benefit of this is no additional IP address space is needed, as it’s merely DNS ‘magic’.

As an example, SAP system T01 is running on VM with a CNAME for SAP virtual hostname sapt01pas.

Figure 5 - Example of DNS CNAME for SAP virtual hostname

Resolving that alias hostname, reveals that it points to the VM hostname, thus 10.185.106.5 is returned as IP.

ht1adm@x17qus80:/usr/sap/HT1/tmp> nslookup sapt01pas

Server: 168.63.129.16

Address: 168.63.129.16#53

Non-authoritative answer:

sapt01pas.virthost.com canonical name = x17qus80.virthost.com.

Name: x17qus80.virthost.com

Address: 10.185.106.5

ht1adm@x17qus80:/usr/sap/HT1/tmp> ./niping -v -H sapt01pas

Hostname/Nodeaddr verification:

===============================

Hostname of local computer: x17qus80 (NiMyHostName)

FQHN of local computer: x17qus80 (NiGetOwnFQDN)

Lookup of hostname: sapt01pas (NiHostToAddr)

--> IP-Addr.: 10.185.106.5

Lookup of IP-Addr.: 10.185.106.5 (NiAddrToHost)

--> Hostname: x17qus80

The CNAME setup would fail for SAP with just DNS, as the reverse lookup for the IP address will return the VM hostname, not the SAP virtual hostname.

We can overcome this issue by either modifying PTR records – and thus likely breaking something else running on the VM like security scan tools – or through VM-local hosts file entry which are likely to cause problems at some point in (near) future.

A DNS alias solution is not recommended for SAP environments. As per afore mentioned SAP note 962955 - Use of virtual or logical TCP/IP host names “Do not use any TCP network alias names to virtualize physical host names within the intra-server communication.”

Off-topic 1: Azure VM name does not have to equal OS hostname

When you deploy a VM in Azure, an OS image – from Azure Marketplace or custom image – is used to provide the OS within. The OS, due to DHCP client, sets the hostname from Azure along with the IP. VM names (think Azure resource) and OS hostnames do not have to match!

For SAP the hostname limit is often 13 characters – see SAP note 2718300 - Physical and Virtual hostname length limitations for details. This same applies in Azure, of course, too.

However the VM itself – the Azure resource – can be much longer. 15 characters for Windows, up to 64 for Linux. Thus, you can leverage Azure naming for a more descriptive name and inside the VM, on the OS modify /etc/hostname and benefit from separating the two.

Off-topic 2: VM outbound IP for 3^rd party applications

When we deal with SAP virtual hostnames, we think of incoming network traffic and name resolution from the perspective of the client/user or 3^rd party application. The SAP server process thinks its hostname is something like sapt01pas and listens on communication coming in on that IP.

However, we often don’t think of egress traffic, how our SAP systems looks to the other application. By default, SAP answers back with the primary IP of the network interface. For some applications which do a reverse lookup (IP -> hostname) this might be a security issue. From its perspective it tries to talk with John (the SAP virtual hostname) but the response (IP traffic back) comes from an address where Peter (VM hostname) is registered at.

SAP has parameters for different services like message server, ICM, etc to set the IP to bind to and use for response. Some years ago, SAP introduced a ‘master’ parameter for SAP NetWeaver kernel to better and easier manage this. Parameter is/local_addr = <SAP virtual hostname> will ensure all outbound communication is using a specific IP – for details see SAP note 2157220 - Kernel parameter is/local_addr

Figure 6 - IP addresses used for inbound/outbound communication paths

With this parameter set, the SAP virtual hostname will be used for outbound communication as well.

Conclusion

SAP ‘s virtual hostname concept is very easy to use in Azure and there are different possibilities to utilize it. Changing and moving of SAP instances between VM, also used by SAP LaMa, can be done very quickly.

To recap some main topics in this blog post

You can have many secondary IPs bound to an Azure VM and its NIC
Azure handles networking and thus you need to make a change on the NIC, before the OS
Adding/removing secondary IPs on Azure and OS side can be done online, without a VM stop/deallocate or reboot
Do not use multiple NICs for virtual hostnames, as it adds layers of complexity. Secondary IPs are recommended instead.
Alternatively use ILBs for virtualizing hostnames/IP
DNS aliases are not recommended by SAP for virtual hostnames
VM name does not have to equal OS hostname