Thursday, March 31, 2016

NetScaler VPX - load balance Exchange - Part 3 (load balance Outlook - RPC)

After configuring load balancing for SMTP traffic (on port 25), I will now configure load balancing for Outlook (MAPI over RPC). This is what some may now call "legacy outlook" since Outlook connectivity is (primarily) using HTTP in more recent versions of Exchange, notably Exchange 2016. This now applies to "Outlook" in general unlike previous versions of Exchange (2010 and before) where only Outlook Anywhere used HTTP (and OWA of course). 

Note: yes, I am still using Exchange 2010. However, the version of Exchange has little or no effect on general load balancing concepts.

As for the SMTP exercise (see my previous blog post), we need to create (if they are not already created):
  • Servers
  • Services
  • Virtual Servers (with a "VIP")
  • Monitors (optional - there is always a default monitor that checks the status of the (Exchange) servers but not of the services. It is possible that the server is available (functional) but the actual services are stopped. Therefore, we can optionally configure a monitor if we want to fine-tune the awareness of service availability).

We have already created server entries, representing our two Exchange servers, so there is no need to repeat this process. Please refer to my previous blog post if you need to see how to create these entries. So we will create two entries for the RPC service and then a virtual server (with a VIP) to which Outlook clients will be directed. Once again, we can optionally create a custom monitor for the service but for now (to keep matters as simple as possible) I will simply use the default monitor.


As a reminder, here are the server entries that we created in the previous blog post:

Just as we created a service for SMTP, and that we associated with each of the servers (see below), we have to create a service for RPC. Click on "Add":

Note: as mentioned in my previous blog post, we could optionally use a "service group".

I configure a RPC service entry associated with each of the Exchange servers:

Yes, for the protocol and the port, we select "TCP" and "*" respectively.

We now have the following services:

Next, in the virtual server section, we create a "virtual server" (with a virtual - but perfectly functional - IP address) to which Outlook clients will connect. Click on "Add":

Configure the settings as follows (I use the naming convention presented in the Citrix training course referenced in the first post of this blog series. You can use another name of course):

Likewise, use an IP address appropriate for your environment.

Now we need to link (or "bind") the virtual server and the RPC services we created earlier. Click on the link indicated below (... Service Binding):

Click on the arrow indicated by the red dot:

Check the RPC services and click on "Select":

That brings us to the following section where we click on "Bind":

Unlike SMTP load balancing, which does not require persistence, we do need to configure a persistence type for Outlook. Click on "Continue" as shown below:

On the far right side of the screen, we should see a menu with various categories including "Persistence":

Select SOURCEIP as the persistence type (then "OK" and "Done"):

I also want to change the load balancing method to "ROUNDROBIN" so I click on "Method" in the same menu on the right and then "ROUNDROBIN":

Click OK and then Done (above).

We have completed the configuration of load balancing for Outlook (RPC). However, we have to adjust the DNS record for our CAS array so Outlook clients are directed to the Outlook VIP ( on the NetScaler rather than the IP address of the Exchange servers:

Now, on the client machines, the Outlook E-mail AutoConfiguration test shows that Outlook is connecting to the CAS Array - which is now associated with the NetScaler rather than one of the Exchange servers. In fact, without a load balancer that the CAS Array can designate, the CAS Array (essentially an Active Directory logical object) cannot provide effective redundancy or high availability. 

Sunday, March 27, 2016

NetScaler VPX - load balance Exchange - Part 2 (load balance SMTP traffic)

Having completed the basic configuration of the NetScaler VPX, I will now configure load balancing for SMTP traffic, specifically inbound mail traffic entering my network via the perimeter firewall.

These are the IP addresses that we will use. Some will be configured on the NetScaler. Others are for reference. - EX13-1 (Exchange server 1) - EX13-2 (Exchange server 2)

Note 1: despite the "13" in the hostname, these servers are Exchange 2010 servers.
Note 2: make sure all the related network nodes are up and running. - VPX NSIP - VPX SNIP

Note 3: these IP addresses were configured on the VPX in the previous blog post.


Verify connectivity

I will take advantage of this project to present some of the diagnostic tools of the NetScaler and test connectivity with the remote network nodes listed above.

If we browse to this location in the NetScaler GUI...

NetScaler > System > Diagnostics

We have several common diagnostic tools at our disposal, among others PING, TRACEROUTE and a command line interface:

If I open PING, I can enter the hostname or IP address of the remote target and the number of pings I want to send. There are a number of other options as well (not shown below):

When I have finished, I click on "Run" at the very bottom of the page:

Connectivity is verified for the first Exchange server (EX13-1):

Note : we can also open a command line and enter the commands directly there.

I also verified connectivity with the other Exchange server (success).

Configure load balancing for SMTP traffic (port 25)

Now I will configure load balancing. To begin, I will load balance SMTP traffic (port 25). Most organizations have one or more firewall appliances at the perimeter of their network and often perform what is known as "1 to 1 NAT".

Note: the paragraph that follows assumes basic knowledge of DNS, mail flow and networking. Also, there is some degree of simplification (some organizations may have more complex networks).

In the case of SMTP traffic, the MX records designate an A record that, in turn, designates the IP address of the external interface of the perimeter firewall. The external IP address is a routable address and usually needs to be associated with an internal non-routable IP address that would otherwise be inaccessible from the Internet. The association of the external routable address and internal non-routable address is an implementation of what we call "1 to 1 NAT". The internal IP address is often that of a mail hygiene appliance such as Barracuda or Ironport, but could be that of the Exchange server itself. If we have more than one Exchange server, and want to ensure some level of high availability, we can direct incoming mail to a load balancer that will monitor the status of the Exchange servers and direct SMTP traffic only to the active server(s) if the other(s) is (are) unavailable. As its name suggests, the appliance can also "balance" the traffic between the two nodes (and perform even more tasks that I will not address here).

Here is a very simple illustration:

Internet -> Firewall -> Load Balancer -> Exchange Server(s)

Load balancing is one of the many features of the NetScaler. We can enable the feature here...

By checking the appropriate box:

We configure load balancing itself here:

NetScaler > Traffic Management > Load Balancing

We need to create:
  • Servers
  • Services
  • Virtual Servers (with a "VIP")
  • Monitors (optional - there is always a default monitor that checks the status of the (Exchange) servers but not of the services. It is possible that the server is available (functional) but the actual services are stopped. Therefore, we can optionally configure a monitor if we want to fine-tune the awareness of service availability).

I will follow the order of creation used the in Citrix training course  (see reference in previous blog post), although creating the items above in a different order is possible.


First, I create "Servers". These servers represent the Exchange servers to which SMTP traffic will be redirected after it reaches the load balancer (yes, click on "Add"):

Enter the necessary details and click on Create:

Note: I use the naming convention used in the Citrix course: srv for server (followed by an underscore and the name of the server), svc for service, and lb_vs for load balancing virtual server (see below). However, you can name these elements according to your own conventions.

I then add a second server (note the first server we created in the server list):

Now we have designated our two Exchange servers:


Next we must configure a "service" (in this case for SMTP) associated with each of the servers:

Configure a SMTP service for each of the Exchange servers as shown below

Here are the services we have configured:

Note: if we need to make adjustments to the configuration, we would need to make the changes for each service. If we had many servers (and one service configured per server) it might be preferable to use a "service group" so we could make a change once and have it apply to all the server members of the service group. Please consult the documentation for additional information about service groups.

Virtual Servers

Now we will create a "virtual server" for SMTP, associated with the two SMTP services, each of which is associated with one of the two Exchange servers. The virtual server, and its virtual IP address, or "VIP", in particular, represent the service (and indirectly the "real" servers)  to clients who will direct their communications to this VIP.

For the virtual server, we provide the details entered in the screenshot below. The IP address is the address that clients will access for the service in question (as opposed to the IP address of the Exchange servers themselves). In my case, the "client" for SMTP connections is the perimeter firewall that will forward email from the outside to this IP address (

Once we click OK, we will arrive at this page where we need to bind the two services (each linked to one of the Exchange servers) to the virtual server:

We click on "No Load Balancing Virtual Server Service Binding" (see above) and then "Click to select" services:

Select both of the services we configured earlier...

And click on "Bind":

We have almost finished! We still need to select a load balancing "method", the most common being "Round-Robin" which we will use here. Once we click on "Bind" (above), we should see a page similar to the one below. Select "Method" in the menu on the far right...

And select ROUNDROBIN as the Load Balancing Method:

The result should look like this:

Lastly, we should direct incoming SMTP traffic to the VIP of the virtual server ( In my case, the perimeter firewall is a Cisco ASA device so I make the adjustments here:

Remember to save your configuration by clicking on the floppy disk icon in the upper right-hand corner. If you shut down the NetScaler from the NetScaler, you will be prompted to save the "running configuration". If you shutdown the NetScaler from VMware workstation, you will NOT be prompted to save your configuration...


I tested the configuration above by sending email to an internal test user from an external email account (Gmail or Hotmail for example). The email arrived in the Inbox successfully. Of course, this requires that we have correctly configured a number of other elements that I have not presented here, such as DNS MX and A records as well as 1 to 1 NAT on our perimeter firewall.  

Friday, March 25, 2016

NetScaler VPX - load balance Exchange - Part 1 (Installation and Configuration)

In this blog post (and the following), I would like to share my experiences with the Citrix Netscaler VPX, used as a load balancer for Microsoft Exchange 2010. The NetScaler can be deployed in several forms: as a "simple" physical appliance (MPX), as a virtual machine that we can host on common hypervisors (VPX), or as an appliance hosting XenServer with one or more virtual Netscaler VPX instances (SDX). I will use the VPX version as a guest in VMware Workstation.

In this scenario, our NetScaler VPX will load balance for a pair of Exchange 2010 (SP3) servers but the general concepts would apply to Exchange 2013 and 2016 as well. In fact, the Exchange servers themselves usually do not require additional configuration to interact with the VPX. Most often, it is simply a matter of designating them as "Servers" to which SMTP, RPC, SSL and possibly POP/IMAP traffic will be forwarded. "SSL Offloading" and work with SSL certificates in general are two examples where we might have to work on the Exchange servers themselves.

I will concentrate on the following aspects in this blog post and in following posts: installation and initial configuration of the NetScaler VPX, configuration of load balancing for various types of traffic (SMTP, RPC, SSL) and possibly some experiments with certificates.

At the end of this first blog post, I will provide a list of NetScaler VPX resources (see below).

Step 1: download and import VPX into the hypervisor

The first step is to download the "NetScaler VPX Express" virtual appliance package at this URL:

Expand the version number and select the image compatible with your hypervisor (XenServer, ESX, HyperV, etc.). For VMware Workstation, we would use the ESX version.

I downloaded version 11.0-64.34 (for ESX):

Note: this is, of course, for a practice environment, the objective being to become better acquainted with the Citrix NetScaler. It is very unlikely that VMware Workstation would ever be used otherwise as a host for VPX.

I extract the content of the .zip file which leaves me with these three files:

If necessary, I copy these files to another location and then import the virtual machine from inside VMware Workstation: File | Open (browse to the .ovf file shown above).

We now have a NetScaler VPX virtual machine with the following configuration:

It is often recommended to remove the second network adapter since licensing is based on the MAC address and having two network adapters can apparently cause confusion. So I will remove the second NIC, at least for the time being.

Next, I click on the green arrow (see the screenshot above - "Power on this virtual machine") and wait for the NetScaler to start. I learned that the NetScaler OS is based on FreeBSD and we may see certain references to this effect in the verbose boot information that displays:

In my case, the system seemed to linger a moment at the "FreeBSD prompt" which is not a prompt at all (just wait for the boot process to continue). In fact, before we logon, we need to configure a first IP address (with mask and gateway) for the NetScaler. We will later use this IP address for management via a web interface. 

Only then can we logon, using the default username nsroot and password nsroot (yes, both username and password are nsroot):

Note: in a production environment, we would change the default password in accordance with the password policy of our organization.

Step 2: initial configuration (IP addresses, DNS, hostname, licensing)

Once logged on (above), we can further configure the NetScaler but only at the command line. If we prefer the GUI, we have to open a browser (I use Chrome here) and enter the IP address we assigned to the NetScaler earlier ( for example):

After logon, the first thing we may see is a message about the "Citrix User Experience Improvement Program", which we can close.

We are then prompted to complete the initial configuration of the NetScaler: 2) at least one subnet IP address; 3) host name, DNS and time zone; 4) licensing (visible on a later screenshot):

We had already configured the "NetScaler IP Address" or "NSIP". We need to configure at least one "Subnet IP Address", also known as a "SNIP".

Unlike network nodes with a single address, the NetScaler usually has several: the NSIP, the SNIP (one or more) and also virtual IP addresses (VIP). For example, the VPX may load balance for Exchange, IIS, and other application servers. Each one of these services would be presented to clients as a separate and distinct IP address.

Essentially, managers access the NetScaler using the NSIP and the NetScaler communicates with backend servers using one or more SNIPs.

Note: the Netscaler VPX virtual image comes with two interfaces. I removed the second, as having two apparently complicates the installation of the license. After intial configuration, we can re-establish the second interface.

In my case, I will leave the NetScaler with a single interface, using what is known as a "one arm" configuration. In summary, this means that clients and servers are on the same subnet/VLAN. This article explains the concept further:

Otherwise, you can search for other articles explaining the differences between "one-arm" and "two-arm" load balancing. The general concepts apply to both Citrix and non-Citrix devices.

So, for the Subnet IP Address, or SNIP (we must have at least one SNIP), I click on the number 2 icon which causes the screen below to display. It explains the concept of a SNIP and allows us to configure the IP settings in the lower left-hand corner. I will use IP address for the SNIP (the NSIP is

Next, I'll assign a hostname, a local DNS server, and the time zone (UTC by default):

And now, it is time to license the NetScaler VPX.

I will use the free VPX Express license which activates many (but not all) NetScaler features for a duration of one year after which the license must be renewed.

Now, licensing the NetScaler is the most complicated part of the initial configuration tasks

Please take note of the value shown on the right-side of the screenshot below.

This is the MAC address of the (first) network interface of the NetScaler.

We must enter this value later when we request a VPX license from Citrix.

Note: yes, prudent or paranoid (?), I have partly concealed both my MAC addresses and license numbers.

We may see confusing references to the "hostname" or "hostid" but it is indeed the MAC address that we should use. I'll show you how we can find the hostname of the NetScaler, but also what happens if we attempt to license the NetScaler based on the hostname.

This is the URL where we can obtain a (free) VPX Express license:

Click on Get License (you may have to scroll to the bottom of the page):

There is a serial number - click on it:

And now the confusion begins! If we noted the identifier above, we might have (correctly) concluded that we would use the MAC address of the NetScaler for licensing. But when we click on the serial number of the license, we obtain a warning about... the HOST NAME:

If we click on the "Determine License Server Name (host name) or Host Id", we read:

So I do this (type "hostname" at the command prompt which gives us NSVPX1):

Note: in the web interface, we can see the MAC Address under:

NetScaler > System > System Information

We then enter NSVPX1 in the Host ID field:

And obtain this result:

*** We need to use the MAC address - NOT the hostname ***

So copy the MAC address from the VPX configuration page above and enter it in the "Host ID" field instead of the host name (and click "Continue"):

This time the license was granted and we can download it to the computer from which we are managing the NetScaler (via the web interface).

Click on download...

Note: save the file to a location on the local computer (I will not demonstrate how to download a license file here). The license file should look something like this:

Returning to the NetScaler web interface, we click on the number 4 to the right of the licensing section (unfortunately cut off in my screenshot):

We click on "Add New License" and browse to the location of the license file. Once installed, we should see a screen similar to this:

We must reboot the NetScaler at this point, so click on the blue Reboot button and be sure to save your configuration:


Once the Netscaler reboots, we can log back in and configure load balancing (for example). I'll take a look at that in my next blog post.

NetScaler VPX resources

Citrix Documentation

We can access Citrix documentation at this URL (select the product and product version):

Here is the section for the NetScaler VPX:

Citrix Education courses

I was fortunate enough to attend an official Citrix NetScaler training course (instructor led):

CNS-205 - Citrix NetScaler 11.0 Essentials and Networking

I would recommend it for anyone responsible for managing NetScaler in a production environment.


For those who prefer books (that may summarize essential points under a single cover), you might want to consider the following titles:

Implementing NetScaler VPX by Marius Sandbu

Mastering NetScaler VPX by Marius Sandbu

Note: perform a search at your preferred bookseller.

Video training (3rd party)

CBTNuggets offers a course which I found useful for my objectives:

PluralSight also offers a NetScaler course (I have not viewed it myself but ratings seem to be favorable):

Citrix VPX Forum

Lastly, I am certainly not the first to blog about the Citrix NetScaler. There is an abundance of other blog posts and videos of varying quality available online.