Monitoring Netscaler with Operations Manager 2012

This guide has been written with Netscaler build 73 and Operations Manager 2012 SP1 (on WS2012) with the management pack from Citrix.

Operations Manager 2012 supports monitoring network devices either through SNMP (v1, 2 and 3) or through just basic ICMP.
Citrix has made a management pack solution, which you can use to enhance the monitoring capabilities in SCOM.
The pack also includes VMM PRO management pack (Which is not gone through in this guide, just the basic management pack)

The management pack can be downloaded from mycitrix (Requires login)
https://www.citrix.com/downloads/netscaler-adc/components/netscaler-management-pack-for-operations-manager-2012.html

(Just a side note: Comtrade is a Citrix Partner who is currently making a new management pack for Netscaler so stay tuned for the new release )

So when we have a functional Operations Manager server up and running we have to install the SNMP service on one of the servers.
This can be done via Server Manager.

After that is installed go into services.msc and choose “Accept SNMP packets from any host” or just enter the IP of the Netscaler server.
Make sure that firewall on the OpsMgr server allows for SNMP traffic in.

After that is done you can install and open the management pack folder.

You will see that it includes a Guide and MP folder (which contains the Management Packs)
Now open Operations Manager console and go to administration and choose Management Packs, right-click and choose import.

And from there browse to the directory and choose the regular NS MP (Not the PRO)

And choose Install.

After that is installed, go back to monitoring and you will see that a new folder has appeared under Citrix Netscaler

by default all Performance monitoring are mostly disabled so we have to enable these to actually get some data.
So go into Authoring -> Rules and scope it to Citrix Netscaler

First of we can enable Virtual Servers current up

So we create a override rule for Netscaler Devices

and choose Enabled and save it into a Management Pack where we save our overrides.
After that is done we alter the SNMP settings on the Netscaler devices, im going it in CLI

add snmp manager IP
add snmp community enternamehere ALL (The last one is used to define which rights this community string has)

Add the IP of the SCOM MS and add a community string (In my case I used “com”)

After that is done we have to add the network device into Operations Manager.
Open Administration -> Network Management -> Right Click and choose Discovery Wizard from the wizard choose Network Devices ->

From there specify a name and which MS and resource pool to manage the device

Click next -> choose Explicit

Click Next -> Here we add the community string which we will use to authenticate with the NS
We have to add a new run as account which includes the Community String

Next we add the device IP and choose what type of service it will use to communicate with the device

After the Device Discovery Wizard is done, go into Discovery Rule and choose Run.
After a while the Device will appear under Network Devices pane.

You can check the Application Log on the Operations Manager server for info and you can check the snmp stats option in Netscaler.
So after this is complete we can see the device health properties

We also have some Performance counters for CPU and Memory we can see.

After you have enabled other Performance Monitors they will appear here as well, this allows you to create a baseline for how connections should be on your box.
This also allows for Operations Manager to generate alarms in case of DDoS attacks.

Citrix and Microsoft cloud solutions

Now If people see any wrong facts here please leave a comment below, I can’t cover everything but this post is written purely based on my opinions and my own knowledge and not everything may be correct.
The purpose of this post is to show capabilities and features that Citrix and Microsoft’s cloud solutions possess and how they can benefit each other. Let’s face it, Microsoft or Citrix (which are huge huge huge! in the cloud market)  for that matter can’t do everything themselves, they both have a solid solution around cloud, but each solution has their strengths and their weaknesses.

Now Cloud is a term that consists in 3 shapes. Private, Public or Hybrid.
And there are some common terms that describe a Public cloud solution
* Self-Service
* Elasticity
* Scalable
* Pooling of resources
(Some Public Cloud solutions: Amazon, Windows Azure)
With Private Cloud you have more control of the resources and it is easier to do customization. More are thinking of going with the Hybrid solution which gives you the advantages that a private cloud brings and the benefits of the low cost with a Public cloud. Both Citrix and Microsoft have the ability to support a Hybrid cloud approach. Then again there are may ways to offer a cloud solution for a customer, either it is an application, infrastructure or a platform.

Now Microsoft’s cloud solution consists of the following core components

* System Center
* Windows Server
* Windows Azure (Public Cloud)

Now what do you get with System Center ?
A brief overview
App Controller ( Self-service of their services & virtual machines for system owners, with support for on-premise and public cloud)
Operations Manager ( Monitoring capabilities, SLA monitoring with support for on-premise and also allows for Azure monitoring)
Data Protection Manager ( Backup solution for Windows and Windows Applications ( Physical and Virtual)
Service Manager (IT Service Management, Helpdesk solution, self-service for the users )
Virtual Machine Manager (Manages your virtual infrastructure, network and storage. With support for Citrix, VMware and Hyper-V of course with most capabilities with Hyper-V allows for creating of 1, 2 and 3 tier templates)
Configuration Manager ( MDM, client/server management, antivirus, patch management, can integrate with Azure as well)
Orchestrator (Automation with runbooks across all kinds of different products)
And of course the corner stone of this is Windows Server.

Now a problem with Microsoft’s as of now is that System Center 2012 does not support Windows Server 2012 until Service Pack 1 is released, this will most likely be released Q1 2013, which will close the “gap” that many are waiting for.
Microsoft has acknowledged that everyone isn’t running just Microsoft and has added much more support and functionality for Unix/Linux based servers.

 
And Citrix’s cloud solution consists of the following core components

Much of Citrix strategy on Cloud is based upon Project Avalon which has the key components (Any Cloud, Any Hypervisor, Any Device)
Which comes in this nice wrapping.

Citrix is part owner of the OpenStack solution that Apache has, and has made some changes to it and have their own solution called CloudPlatform
Which is very similar to virtual machine manager. It has support for multiple hypervisors such as
* XenServer
* KVM
* VMware
But their solution has more benefits against XenServer.
It also supports storage solutions and network. So this is the main product for administrating your “cloud”.
Then we have the other products such as
* Cloudbridge (Allows you to “bridge” your private and public cloud” this is actually an add-on to Netscaler which uses IPSEC)
* Cloudgateway (Which is the gateway in for end-users (Which again consists of Netscaler and Storefront )
* Netscaler ( A Network appliance which provides for ADC (Application Delivery Controller) features
* CloudPortal (Which allows for provisioning of users and services, control panel solution)

So depending on what kind of cloud and service you wish to offer your users, both companies provide a solid cloud solution. With automation and multiple hypervisor support.
Microsoft has made a solid improvement to Hyper-V in the latest release so it provides with more advanced features then XenServer it also has support for larger workloads and scalability. So if you choose Hyper-V you need to have VMM, if you choose the latest XenServer you would need CloudStack (VMM has XenServer support but not for the latest release and not for the more advanced features)
Citrix is building much of their solutions based upon XenServer (and some VMware) for instance the AppController that is part of the CloudGateway will not function in Hyper-V

Microsoft also offers a more complete monitoring solution with System Center (You have the capability to monitor all of Microsoft’s products, Network devices, Citrix Products + inlcuding Netscaler (With ComTrade MP) and Unix/Linux services)
And I don’t have enough insight on the automation part of CloudStack go give it a good overview but Orchestrator has also the ability to run commands against SSH devices which allows for running commands against Network devices it also has a broad support of hardware and storage vendors. You can also use it to run PowerShell commands which allow for automation of Citrix installation. (And more and more vendors are implementing PowerShell cmdlets with their products, PowerShell 3 also supports CIM which many vendors support)

But what Microsoft is missing is the network component that Citrix provides with it’s Netscaler product.
* Advanced load-balancing features for all applications running on TCP with or without SSL (With hardware acceleration on the hardware appliance)
* Protection against DDoS attack (SYN flood, ICMP floods) and can also provide with defense against application level attack (XSS, HTTP DoS)
* URL responders, rewrite, filtering
* Intelligent SQL load balancing
* GSLB
* Caching and compression
You can also integrate it with System Center to provide automation of new solutions that should be load balanced. You can also use Orchestrator to automate other options with the SSH options.

When regarding device access Citrix has provides a better solution with support for all types of Mobile devices, which makes it possible for full BYOD. Microsoft on the other hand also promises that you can bring your own (as long as it is running Windows) This solution  requires that you can Citrix on your terminal servers. Citrix also has more MDM capabilities then System Center has (as of today), and with the coming of MDX technology, Citrix is going to gain more ground there.
Microsoft also offers a VPN solution with allows you to connect with your Azure cloud, but this does not provide the same throughput that a dedicated Netscaler with Cloudbridge would provide (Cloudbridge again has limited support against Azure )

And I forgot to mention that Citrix has also their own monitoring and helpdesk tools which are part of the GoTo package (GoToAssist and ) But I am unsure how they compete against Operations Manager which has been around for a long time and against Service Manager which is a core part of the Self-service solution to System Center.

System Center with SPF (Service Provider Foundation) provides the capabilities for hosting providers to create their own control panel solution to automate activities against Orchestrator and VMM) This is a feature that is still in the early stages with an open API. Citrix on the other hand has a more mature product with their CloudPortal solution which can provision users, set up full services on Lync, Exchange, SharePoint, CRM, XenApp and XenDesktop ++ for customers.
But the weakness is that it does not have any integration against System Center to complete the circle on management and monitoring ( and of course backup)
But again this feature is more suited for hosters, for enterprise businesses not so much.

So a little conclusion on my part. What do I think makes a good combination of what these two deliver. This solution will consist of a few products that are yet to be released (But are in beta)
1: Hyper-V 2012 as my main Hypervisor
2: System Center for infrastructure & cloud management and monitoring (SP1 with provides support for WS2012)
3: Project Excalibur next generation XenApp / XenDesktop which provides the best BYOD support (And Provides support for WS2012)
4: XenServer for components that need XenServer
4: Cloudgateway with Netscaler ADC

So it would look like something like this
(Just a glimpse)

And I would appreciate some feedback on your thoughts

Netscaler and DDoS

A part of many Network admins day to day tasks involves mitigating DDoS attacks. And they come in many types of shape and size.
But they are share a common goal, disrupting the service for the users. These types of attacks make the service unresponsive and therefore cannot service the regular users who actually need to access the service. Trough out the years there have been many DDoS attacks on many of the HUGE online services.

For instance PayPal, Visa many online banks (Such as DNB in Norway) have suffered of these kinds of attacks, and if you think about it what happens if an online bank is offline ? The business loses a lot of income and the regular users cannot access their online bank account.

Now back to the kind of DDoS attacks. The most common ones are

SYN Flood:
Happens when a host sends a flood of TCP/SYN packets, which are often from an forged address. Each of these packets is handled like a connection request, causing the server to spawn an half-open connection. This is actually just a simple exploit of how TCP connections are established.
I like to think of it as an old lady (who is in disguise) to gives a bag to the server and says (can you hold this bag for me ? ) and of course the server is happy to oblige and therefore holds the bag and then the old lady runs of, and the server is standing there with the bag yelling (“Old lady?”) and then again there comes another old lady (in disguise) which comes with another bag which again the server is happy to help out and again is stuck with two bags.
And as you can see it is only a matter of time before the server cannot hold anymore bags.

ICMP Flood:
Which again is split up in more different types.
The primary with these kinds of attacks is that it uses ICMP. Now the Ping command is pretty simple when run it asks a server, “Are you alive?” and the server says “yes” If you have thousands upon thousands of these kinds of requests they can quickly use up much of the network bandwidth at the server.

Smurf Attack (which is an ICMP flood Attack)
Is another type of attack (Which is usually used where the network isn’t configured correctly) What happens is that a attackers, sends a spoofed IP address ping to an broadcast address in a network, and the reply to address is set to a server address. What happens is that all the clients in that subnet (which are alive and gets the ping request, will repond to the ICMP packet to the server)
These kinds of attacks are usually easy mitigated at the network.
For instance with Cisco you can set a pretty ACL to limit the ICMP traffic

config t
Access-list 100 permit icmp any {your network} {your subnet} echo-reply
Access-list 100 permit icmp any (your Network) (your Subnet) echo
Interface e1
Rate-limit input access-group 100 512000 8000 8000 conform action transmit exceed action drop

Or what you should do is use the command no direct ip-broadcast

Now these 2 are the most common types of low-layer attacks. There are a bunch of layer 7 attacks I will discuss in a later post.
So how does Netscaler come into the picture ?

SYN Flood:

A NetScaler appliance defends against SYN flood attacks by using SYN cookies instead of maintaining half-open connections on the system memory stack. The appliance sends a cookie to each client that requests a TCP connection, but it does not maintain the states of half-open connections. Instead, the appliance allocates system memory for a connection only upon receiving the final ACK packet, or, for HTTP traffic, upon receiving an HTTP request. This prevents SYN attacks and allows normal TCP communications with legitimate clients to continue uninterrupted.

SYN DoS protection on  NetScaler appliance requires no external configuration. It is enabled by default.

ICMP Flood:

The NetScaler also protects network resources from ICMP based attacks by using ICMP rate limiting and aggressive ICMP packet inspection. It performs strong IP reassembly, drops a variety of suspicious and malformed packets, and applies Access Control Lists (ACLs) to site traffic for further protection.
Now if you type sh ns ratecontrol

You can see the allowed ICMP packets per MS. 100 is the default value. And by default there are no rate control set on UDP and TCP.

Citrix Netscaler and SDN

There is a lot of talk now about SDN, so I would like clarify what is actually meant with SDN.
First of the name SDN is for Software Defined Networking, the concept Is actually pretty simple. It is built upon that a network administrator can share traffic from a single console Instead of having to configure each switch individually.
And for instance, it allows for an admin to allow SharePoint traffic but not regular HTTP traffic, it steps away from the regular ACL deny/grant and performs at a more granular level.
Hence the name, Software Defined

So you have to think that instead of configuring the switches, acls, router etc., an it-admin configures the software that goes trough the network(application control)
So there is a lot of speculation that this is going to be a Cisco killer or Cisco kill switch because this would allow for a multi vendor solution but still have that single control plane.

So again the network becomes app-driven

And back to the topic, Citrix has already laid plans for the future with its Netscaler product and SDN.
They have already established many partnerships with different vendor for this to happen.

Cisco has announced that they are going to discontinue their production of their ACE module, and has already gone into partnership with Citrix to work with SDN.
They have that that for instance Netscaler and their Cisco ASA (Their firewall product) will include integrations which are based upon SDN.

Citrix has also aligned themselves with a lot of other partners such as.

* Palo Alto
* Trend
* Aruba
* WebSense
* Bluecat
* RSA
* Splunk

You can read more about them here –>
http://www.citrix.com/products/netscaler-for-sdn/how-it-works/partners.html

And Citrix themselves has a lot they can offer with Netscaler
With for instance AppTemplates they have many finished templates a company can use (For instance SharePoint, Citrix has made their own template based on best practices which allow for the best kind of user experience )
a template can include (policies for caching, compression, rewrite and other NetScaler functionality used to optimize the application.) So instead of typing in all the information all the Netscaler needs is the IP of SharePoint and Netscaler will do the rest, pretty easy instead of writing ACLs, setup up the service adding load-balancing etc.
You can find them here –> http://community.citrix.com/display/ns/AppExpert+Templates

So it’s a step in the right direction
And with integration with System Center, an Netscaler deployment infront. a Netscaler can detect if there is need for more servers in the back and automatically deploy more.
But the conclusion is, SDN is become more and more popular subject so be ready for it

Troubleshooting Netscaler

Netscaler is a complex device, and lets face it a lot of things can go wrong. Either when setting it up or someone does something weird with the config and saves it. So therefore I wrote this basic troubleshooting guide, hopefully it will be some help for some
This guide is primarily written with CLI in mind, of course the appliance includes a pretty comprehensive GUI diagnostic menu as well.

Here you also have the “Call Home” option which (you have to enable the feature first enable feature ch)
Call Home requires that your Citrix NetScaler MPX appliance When the problem cannot be resolved using the old methods, you can decide to push “Call Home” which will then upload all the data for troubleshooting to Citrix Tech Support (https://taas.citrix.com) and optionally create a support case. Off course you must have an active Technical Support Contract to make use of this feature. Not having an active Technical Support will result in a registration failure.

Another useful tool is the “Revision history” that allows you to rollback to a previous config file, in case someone made a mistake to the config.

But! Lets first start talking about troubleshooting the network. If you have some traffic issues that you want to debug closer we can start a nstrace.sh or a nstcpdump.sh (for more low level debugging)
both of these store in cap (capture files) that you can analyze further via for instance Network Analyzer or Wireshark, and ill show you how.

If I run for instance the command nstrace –time 30

I will store trace data for 30 seconds in each a new cap file.
I could also for instance apply a filter to the trace in order to “filter out” stuff that I don’t need.

I could use –filter “SOURCEIP == 10.0.0.1” –time 30
This would fetch out traffic where the source IP is from 10.0.0.1

There are some other filters that you can use, they are listed here –>
http://support.citrix.com/article/CTX120941

In order to kill a trace push CTRL + C

Now if you want to fetch out these files you need a SCP client for instance WINSCP
After you have downloaded and installed the client you can establish an connection to the NSIP.

Now if you go to the root/var/nstrace catalog and copy over some of the cap files.  And I suggest that you open these in for instance Wireshark.
IF you open Wireshark and choose the import file option and choose one of the cap files

Here is the cap file where I put the filter on Source ip. If you can perform a raw dump on the netscaler ( This is pretty CPU intensive so be careful ) and then perform analysis in Wireshark.
Now that we’ve covered the networking part, we can move forward to moving event viewers.
If you connected to the netscaler console you can run the command nsconmsg you have to run the command shell first.
And after you start it the console might get spammed.

You can also open log files from –> /root/var/nslogs (and there are some useful logs there)

Httperror.log

cron

Dr_error.log

Httpaccess.log

License.log

Nscollect.log

Nsvpn.log

Nsvpnd.log

Snmpd.log

and you can use the same command to view archived logs. For instance if you have a newnslog.100 file.
cd /var/nslog
tar xvfz newnslog.100.tar.gz
/netscaler/nsconmsg -K /var/nslog/newnslog
(Remember to use K not k in the command a small k is used to write logs not read)

This will read archived logs.
You can also use the common unix command top to view utilization.
Also you can use the command ns commands:

Stat ns

Stat cpu

Stat interface

Common stat commands for vserver and service:

Stat lb vserver

Stat cs vserver

Stat service

Other common stat commands:

Stat dns

Stat ssl

Stat http

Also when In doubt doublecheck all the settings for a service. For instance if you have a high-availability setup use the
show node

There is also numerous show commands that you can use to check the status of each service.

show node

show info

show license

show lb vserver

show cs vserver

show service

show persistencesession

show connectiontable

show route

show ip

show dns addrec -type proxy

Another useful command is when you are looking at the config file.

sh run | grep XA
this will show the running config but just the lines containing anything with XA

Hopefully this will get you along the way when troubleshooting a Netscaler device
Would also recommend that you check this URL below for reference for logs and messages.

Other useful links: logs message reference
http://support.citrix.com/article/CTX132382

Netscaler 101

The last couple of days I’ve seen a lot of traffic on my blog regarding the posts on Netscaler ( And I don’t have so many of them!) And with the recent events regarding Cisco ACE and Microsoft Forefront TMG, I’m guessing that a lot of people are looking into the option to switch over to Citrix.
Cisco has always been huge in the networking market, but in the ADC (Application Delivery Controller) market they have never gotten the huge market share that they were hoping for, therefore a couple of weeks ago they decided to stop further development of their ACE product. And in similar events Microsoft decided to stop further development on their TMG product. TMG is not the same product like Netscaler/ACE/BIG-IP thou it has a lot of the same functions and features.

So back to Netscaler what can it offer:
* Advanced load balancing
* Content and app caching
* Database load balancing
* Application Firewall
* Secure Remote Access
* Advanced server offload
* Application acceleration
* Integration with Citrix
      * Access Gateway features
      * Web interface
* Scale up and Scale Out features

You can read more about the different features here –>
http://www.citrix.com/products/netscaler-application-delivery-controller/features.html

Now the Netscaler product comes in 3 Different versions.

MPX: Which is the hardware appliance, is again split up into different models,
http://www.citrix.com/products/netscaler-application-delivery-controller/features/platforms/mpx.html
As you can see most of the models here have a “pay-as-you-grow” options, so for instance if you buy a MPX 7500 and your company is growing and you need more throughput you can upgrade your 7500 to a 9500. So it’s the same hardware as before you just “unlock” more features.
You can see all the different models and features here –> http://www.citrix.com/content/dam/citrix/en_us/documents/products/netscalerdatasheetaugust2012.pdf

VPX: Is a software based virtual appliance, which is available for Hyper-V, VMware and XenServer.
http://www.citrix.com/products/netscaler-application-delivery-controller/features/platforms/vpx.html
Here as well you have a “pay-as-you-grow” solution so you can upgrade it if you need more throughput, the downside to using a VPX is that it does not have  hardware based SSL acceleration (which the MPX has), which allows for a lot less SSL connections.

SDX: Is the best of both worlds. It is a hardware appliance like the MPX but in also has capabilities of running VPX. So it’s a piece of hardware which basically runs a stripped down XenServer which allows to run multiple VPX inside. And since this piece of hardware has SSL acceleration capabilities it does not have downside of a regular VPX. It allows for up to 40 VPX’s and that will allow for true multi-tenancy.
You also have the “pay-as-you-grow” option here.
http://www.citrix.com/products/netscaler-application-delivery-controller/features/platforms/sdx.html

Also Netscaler comes in 3 Different editions (Like most of Citrix products)
You can see the different editions and their limitations in this datasheet
http://www.citrix.com/content/dam/citrix/en_us/documents/products/netscalerdatasheetaugust2012.pdf

A summary,
Standard = Use for Load-balancing (Web and DB) also has Citrix Web interface and TCP optimization
Enterprise = For more advanced features – cloud bridge, edgesight for netscaler, branch repeater client.
Platinum = Includes all the features.

So what do I need for my organization ?
Well first of you need to figure out what your needs are.
1: Do I need just the load balancing for my Web-servers?
2: SSL VPN solution and/or SSL offloading?
3: Advanced Web load-balancing and caching and optimization?
4: Multi-tenancy solution ?
5: DDos defenses ? Or do I have a firewall in front which is fully capable ?
6: Just for my Citrix pieces (Access Gateway and Web interface)?
7: SQL load-balancing?
8: How many users do I have?

You also need to calculate the bandwidth usage the service you are going to load-balance, most of the products (for instance Lync) has well documented traffic usage for each feature.
Let’s take an example if I am a small business that just needs to load-balance my 2 webservers for my internal users (and I have 100 of them) the smallest VPX would suffice.
If I am a enterprise service provider and I offer fully multitenancy solution where customers can setup LB for all their services I would recommend a SDX (The best solution regarding version is to start with the lowest system you think you need and upgrade when you need to grow)

So after you have chosen the model (remember that you always need two of them, since if you only have 1 you have a single point of failure). The next part is setting up the device.
Remember that a Netscaler operating system consists of two parts.
1: Part is FreeBSD (The Appliance uses this part for booting and for logging)
2: Part is the core os (NSOS NetscalerOS) Which controls the traffic in / out of the appliance.

When a appliance boots, it will get system image from the flash and decompress and put it into the ram. The config file is also fetched from the flash and put into the ram. (Which is know as the running-config)
(You can show the running-config from CLI by running the command, show ns runningconfig if you want to see the saved config you can run the command show ns.conf )
You can access it either via a console (serial cable or console via the hypervisor )

And remember that you can save at anytime by running the command save ns config, if you screwed up you can restart the Netscaler (if you didn’t save your config)

But when you start the NS appliance the first thing you see is that it asks for an IP (Which is known as the NSIP Netscaler IP) Which is used for management purposes and clustering. You also enter a subnet mask and a gateway.

After that you can save and quit the config menu and you can now access the appliance via a webconsole. You can also see more info regarding the interface by running the command show ns ip 10.0.0.2

As you can see here it says that “Management Access is enabled” and FTP, Telnet, SSH and GUI is enabled.
So we should disable the insecure access methods before we continue. By running the commands set ns ip 10.0.0.2 –telnet disabled and same for FTP

And there are other things we should configure as well, change the default password for the user “nsroot”
You can do this by running the command config system user nsroot PASSWORD (something very very safe)

Also you SHOULD enable NTP sync with a authorized ntp server.
add ntp server IP –minpool integer –maxpool integer
enable ntp sync

Now we can log onto the Web GUI.  (Im using version 10 of the Netscaler VPX you can get a free trial for your hypervisor from citrix.com and might add that the web gui is much improved in V10)

The default username and password for the local system user on a netscaler is nsroot and nsroot
So after you have logged in you will come to main menu.

Its split up into 3 panes (Dashboard, Configuration and Reporting) and what you see here is the configuration pane.
If I go to the Dashboard, you see a lot of read-time information regarding well.. everything you want to see
I can choose if I wish to view SSL connections, TCP handshakes, HTTP traffic etc..

The reporting pane is just that, you can create reports and there are a bunch out of the box that we can view as well.
But most of the time we are going to be in the configuration pane.
Now what other things do we need to do in order to load balance a service?
First of we have to design how the netscaler should be placed in our infrastructure, most of the designs are based on
one-arm-mode or two-armed-mode.

In one-arm-mode the netscaler has ONE interface, and on that interface external traffic comes in and the inside traffic out on the same interface (traffic is split by using VLAN’s)
In two-arm-mode the netscaler has TWO interfaces, 1 for external traffic comes in and comes out and 1 for internal traffic. So this is the much more common deployment.

Now in both scenarios the traffic to the back-end servers are flowing as the following.

Now when the client connects to the web service as the virtual IP (90.90.90.90) The Netscaler (depending on the LB rules) make a connection to one of the servers which are connected to that virtual service with the Netscaler SNIP(Subnet IP)
The Subnet IP is an address that connects the netscaler to the servers in the backend, so you should have an SNIP address for each subnet you want to have services in.
So SOURCE IP —> VIRTUAL IP (NS) SNIP —-> WEB SERVER 10.0.0.4 (BASED ON LB) so for the web servers it will appear that the connections come from the same IP. And the same will go back to the clients
WEB SERVER –> SNIP (NS) VIRTUAL IP —> SOURCE IP, so for the clients all they see is that one IP address which may house loads of web servers.

Now is there a problem with this ?
Well yeah.. if you have a web server you probably want to have logging in place for the IP address of the client,  now you have the Netscaler option which known as use “Source IP mode”(USIP) which will allow for clients to do a direct connection with the backend servers.  But what is the downfall of this ?
1: TCP Multiplexing which allows for the netscaler appliance to have one connection to the webserver will be disabled when you use Source IP mode.
2: When backend servers see the source IP they will look at their default routing table instead of returning the traffic to the netscaler, so therefore the servers with go with the local gateway instead of the netscaler. When the backend servers try to connect to a TCP connection with the client, the client will drop connection since it is awaiting its response from the Netscaler VIP.
So in the case you use Source IP mode you need to set the default GW on the backend servers to point to the NS.

You can set USIP mode in modes.

Configuration –> Settings –> Configure Modes –> Use Source IP

Alternative enable ns mode usip
In case of logging we have another choice(  inject HTTP header option which allows the Netscaler to inject the source IP header into the http request which again allows logs on the webserver to contain the IP-address of the client. )
But in general I would recommend that you don’t use USIP.

Now lets setup a load-balancing configuration.
Before we continue remember that you need to setup at least 3 addresses on the NetScaler
1: NSIP
2: VIP
3: SNIP or MIP

There are a few things we need to find out before we can setup LB, what kind of service to we need to load-balance and what servers are hosting this service. And we need to setup a monitor towards that service as well, this monitor check is the service in the backend is responding on that server, if one server is not responding for a particular service it is taken out of the LB queue. So we need.
1: Servers (The list of servers that have a particular service running
2: Service (What kind of service is it ? Webhosting port 80?
3: Monitors (Checks if the service on the server is responding if not it is taken out of the LB-queue until it start responding again)
4: Virtual IP (a virtual IP address which the Netscaler will respond to)
All this is added together and it creates a load balanced service on a virtual Ip address which consists of the servers in the server list.

So lets go ahead and create a LB service. First we add a VIP and a SNIP

Go to configuration pane –> IPs and add a IP address. Remember that a VIP is the ip address that the end users are going to connect to, the SNIP is a ip which the netscaler uses to connect to the servers in the backend.
After that go to the load-balancing pane further down below.
Go to servers and add the servers that has a service.
(Remember that this is just a list of servers, you don’t define the services here)

After that go to monitors –>
As you can see the HTTP monitor is enabled by default
This does a HTTP HEAD command, and if it is working as it should you should get a code 200 response.
You can see this by opening the http monitor

After that we add the service,
We add a service that runs on port 80 on one server and add the HTTP monitor. (Remember to add this for both servers) And have a very descript full name each service on each server.)

Now that we have both services on both servers it should look like this
(In my case I don’t have any hosts on these IP addresses yet so therefore they are stated as Down) because the monitor is trying to do http request on them.

Now at alas we will add the virtual server that will point to the http server on these 2 servers in the backend. Go to Load balancing and virtual server –>

Remember to add both of the services on those servers (If you wish to load balance differently for instance it you have a more power on one of the server you can alter the weight on that server to 2, then this server will take twice the load)
You can also go to method and persistence to change how the service is load balanced. By default it is set to “least connection” that server with least connections will get the next connection, this will happen until they are even. You can also specify persistence (This will define if a client should talk with the same server it spoke with earlier) the most typical choice here is cookie insert for web services. But we will leave it at the default.

Now I’ve added a HTTP server with actually responds to HTTP

You can see that is responds to http request if I open a browser to IP 10.0.0.26
And if you are like me and would like to do it via the CLI you can do this.
Run the command add service servername ip http portname

Next we need to add the services to a virtual IP. (that will do the load balancing )
first we do a add lb vs servicename http ip 80
then we bind the services to that virtual ip
bind lb vs servicename serviceserver

After that you can do a

sh lb vs v1 to show that if the load balancing is active

 

Phuh! long post, next one will be regarding setting up a cluster on Netscaler, since you would always need 2 x Netscalers so you don’t have a single point of failure. And we are going to integrate authentication with LDAP.
Now I would also recommend that some user look at the command reference sheet from Citrix eDocs
http://support.citrix.com/servlet/KbServlet/download/20679-102-665857/NS-CommandReference-Guide.pdf

Microsoft Private Cloud and Application Delivery Controllers

An import issue to adress  in a private cloud setup is setup of HA “high availability”. There a multiple key components that make up a cloud service, and all of the core components need to have HA because if  one of the core components go down, your cloud goes down. C

The network must be designed properly in order to address the traffic the cloud service will generate. For instance if you have a big service like Facebook or Linkedin you need to have a proper network design in place to be sure that the solution won’t “kneel” on the first day because of the traffic. (Either it is regular requests to the site or because of an DDOS attack)
And as a part of that design you need ADC.

Of  course when you connect to a public service like facebook.com you don’t go directly to a webserver.  A typical deployment for a service (with HA would look like this)
End-user ————–> Internet ———-> Firewall -> ADC -> Pool of web servers.

An ADC can be described as an next generation load balancers.
They include features such as, compression, caching, ssl offloading, content switching and load balacing. There are of course other options as well (Some are different for each product, but these are the common criteria for an ADC)

The largest ADC products in the market are F5 BIG-IP and Citrix Netscaler.
(According to Gartner 2010)

And many of the largest web companies in the world use Netscaler or BIG-IP ADC’s
Like Facebook, Bank of America uses BIG-IP according to netcraft.com and sites like Visa use Netscaler.

(Of course if you wish to try out some of the features in these products, both of them offers virtual appliances that can be run within a hypervisor with some limitations)
F5 also has a nifty flash to show many of the features within a ADC and how they work -> http://www.f5.com/flash/product-demo/

But back to the cloud, when deploying new services in the cloud you can automate much of this with SCVMM 2012 out-of-the-box.
* Automate the deployment of new service.
* Installing the operating system / applying security updates on a virtual machine
* Installing the application or server roles (Terminal server / web server )
* Configure which users have access to the service, so on and so forth.

But of course this will only get you so far, if you have an ADC between your firewall (Which is connected to the internet) and your infrastructure you would need to make some settings on the ADC as well in order to deploy the service properly.

Microsoft has seen the value of working together with the ADC vendors, and because of this you can integrate your ADC’s into SCVMM and with it fully automate your service deployment. As of today there are 3 “connectors” avaliable.
BIG-IP -> https://devcentral.f5.com/tabid/1082224/Default.aspx?returnurl=%2fLinkClick.aspx%3flink%3dhttp%3a%2f%2fdevcentral.f5.com%2fdownloads%2fplugins%2fF5LoadBalancerPowerShellSetup-214-x64.zip%26tabid%3d73%26mid%3d3221

Citrix Netscaler -> http://community.citrix.com/display/ns/Citrix+NetScaler+LB+Provider+for+Microsoft+System+Center+Virtual+Machine+Manager+2012

Brocade -> http://www.brocade.com/partnerships/technology-alliance-partners/partner-details/microsoft/microsoft-systems-center/index.page

Im going to walk trough the deployment of Netscaler connector within SCVMM 2012. And how you can further use this when creating templates.

First of install the connector from the site. Click next, next and install.

After you have installed the connector you need to restart the virtual machine service.
(Just open it from services.msc)
Then it should appear under Configuration Providers

Before we can use it, we need to add it as a Load balancer,

From there you need to create a runas account which has access to the netscaler, and has access add LB rules.

Then you need to choose which host group this LB will be active for, then choose the manufacturer and model.

Then enter the IP address and port for the Netscaler device.

Now under Provider we check if the system has access to the device.

The system will try to perform basic functions on the device like
* Retrieve LBsysteminfo
* Open LBConnection
* Close LBConnection
* Retrieve LBknownVIP
* And so on..

Afther that is complete you can click complete. Now that the Load Balancer is in place and is configured correctly with access we must create a VIP template.
A VIP template contains a configuration setting for a hardware load-balancer for a specific type of network traffic. For instance, you could create a template that specifies the load balancing behavior for HTTPS traffic on a specific load balancer.

In this example we are going to create a VIP template for https traffic where the SSL is going to be terminated at the load-balancer

So give the template a name and define what the VIP port is going to be (since https is over port 443 I enter that)

Next I choose what type of load-balancer I wish to use

Click next, now we have to define which Protocol we are going to load-balance, and if we wish to terminate the https connection at the load-balancers.
We also need to enter a Certificate subject name here. For instance C=US,ST=WA,L=Redmond,O=Contoso,OU=Test,CN=www.contoso.com/emailAddress=contoso@contoso.com.

Click next,
Here we change the settings for Persistance, for instance if someone has the SSL session ID of = 12325345345 and has visited WEBSERV1 before then the user be routed back to that server.

Click next –>
Now we choose what kind of Load balancing method we are going to use, im going to stick with “Least Connections” since my web servers are equal in terms of hardware.

And last but not least Health Monitors.
Health monitors are in place to check if the servers in the back actually are alive and responding.
You can for instance add a GET / in the request box and type 200 under reponse (Which is the status for OK in HTTP)  and the device will perform a HTTP GET on each server so see if they are alive and well.

Click next then finish!
After this is done you can use this template in any service template deployment (I will get back to that in a later post)

Citrix Netscaler

After been attending a Citrix Xenapp course the previous week, there was one thing at the course that I’ve actually never heard of and that was Citrix Netscaler.

What is Citrix Netscaler?
After been reading a bit about it, I simply can’t describe what it can do( cause its so much!). Mostly its a hardware based load-balancer.
It can also be used for caching. So if placed infront of a web server klients don’t have to get the information from the web-server it can get it directly from the netscaler. This can take alot of traffic away from the webservers ( Of course you can use it on a fil-server as well) You can also use it for SSL-VPN and as a proxy solution.

It has alot of other features that I don’t know about I myself have just started to read about it.

Who uses it ?
Most of the largest websites in the world use Netscaler infront. Take Microsoft as an example.
http://uptime.netcraft.com/up/graph/?host=www.microsoft.com and of course Citrix themself use Netscaler.
http://toolbar.netcraft.com/site_report?url=citrix.com

Alot of the other big sites out there, like Facebook use BIG-IP F5 which is one of Netscalers competitores in the marked.

I want to learn more!
Head over to Citrix website -> http://www.citrix.com/English/ps2/products/product.asp?contentID=21679
Citrix also offers a virtual appliance (built upon FreeBSD) that you can download from their website. Which is useable in vSphere, Xenserver or Hyper-V.

In my case I wanted to try it on my vmware workstation 7.x
I had to download the vSphere version from Citrix

After that I had to use OVF Tool, convert the downloaded OVF format Virtual Appliance to VMX format:ovftool.exe can be located in C:\Program Files\VMware\VMware OVF Tool.  The following command can be used (where Source.ovf and Destination.vmx are replaced with the actual paths):
“C:\Program Files\VMware\VMware OVF Tool\ovftool.exe” C:\Source.ovf C:\Destination.vmx

Then I could use the Import function in Workstation, applied some default configuration settings and im good to go.

 

PS: If you want to login to the console using the CLI the username is nsrecover and password is nsroot. Default for the webinterface is nsroot and nsroot

When its done, open a browser and point it to the IP address of the netscaler.

Type in nsroot and nsroot, and accept the Java plugin. and voila!

The picture above shows what functions I have avalible on my Virtual Appliance, the picture below shows all the features that Netscaler has to offer.

Later im going to show how to load balance 2 webinterface infront of Xenapp and how to use the Netscaler as the webinterface server.