Real Server Down and Virtual Service Down are two different types of issue that may be reported by LoadMaster 360. However, the way you troubleshoot both issues are very similar so this topic outlines information that may help if either of these issues is reported.

Getting notified about this type of issue makes you aware of potential problems with your application delivery such as a risk to the service or a reduction in capacity.

Example scenario: The following text describes an example scenario where a Real Server Down issue may occur. Web development make a live update to a corporate web page but it results in an error when the process is restarted on the second server due to installed library version differences. This is not noticed because website connectivity is maintained.

A Virtual Service is considered to be down when user connections are not accepted. This includes the following LoadMaster Virtual Service states:
  • Sorry
  • Fail Message
  • Security Down
  • WAF Misconfigured
  • Down

One of the most important aspects of maintaining application availability is ensuring the health of Real Servers for which Virtual Service traffic is configured to be sent. The LoadMaster does this using the configured health check. If the health check fails, the Real Server is considered unhealthy and is taken out of service. While a load balancer should ensure application availability, you should investigate any unavailable Real Server.

Like all network application issues, there are a number of possible issues that may cause a Real Server to be down. Below are some suggested questions to ask and things to check to diagnose these issues:
  • Check the health of the Real Server: First, check if the Real Server is actually healthy. Typically this involves examining the Real Server directly:
    • Is the device up? Check the reachability of the server, the status of the Virtual Machine, and so on.
    • Is the application running? Can you access the application locally on the server?
    • Can I access the Real Server directly from my own client? Can I access the application by pointing my client directly at the Real Server (this may require a local DNS update)?
  • Check that the appropriate health check is in use: If the Real Server seems healthy, the next thing to examine is if the health check in use is appropriate:
    • Is the LoadMaster Virtual Service using the correct health check? You can check what health check is selected by going to Virtual Services > View/Modify Services > Modify > Real Servers > Real Server Check Method in the LoadMaster User Interface (UI).
    • Can I successfully perform the same health check from my own PC/client?
    • At what level is the health check failing? You can use the LoadMaster TCPDUMP feature to review the traffic.
  • Check that LoadMaster to Real Server traffic is allowed: Here are some questions to ask:
    • Is there a firewall blocking traffic? Check what firewall rules are in place between the LoadMaster and Real Server.
    • Is there a local firewall blocking traffic? Check the local device firewall rules.
    • Is traffic allowed from both units (if using a High Availability (HA) pair of LoadMasters)? You can use the LoadMaster TCPDUMP feature to review the traffic.
  • Other checks to perform: Some other checks you can do are as follows:
    • Is it possible there is another device using the same IP address as your Real Server? To investigate this, you can check the ARP table entries for the Real Server.
  • Is it possible that the Real Server is handling excessive traffic loads and is unable to process the health check request? Check the volume of connections being processed on the application.