The LoadMaster generates syslog messages using the following format:

YYYY-MM-DDThh:mm:ss+hh:mm Hostname Subsystem: Message Data
An explanation of the format is as follows:

  • Date/Time Stamp: YYYY-MM-DDThh:mm:ss+hh:mm

    This includes the full date and time along with the timezone offset.

  • Hostname: The name of the LoadMaster host that generated the message.

  • Subsystem: The component or module that originated the message (for example, kernel, wafd, ssomgr, and so on).

  • Message Data: The actual content of the message.

All syslogs are rotated every seven days (older logs are removed).

Disk Usage - This section provides an indication of the percentage used/free of the log partition. Color-coding is used to highlight different usage levels:

  • 0% to 50%: green
  • 50% to 90%: orange
  • 90% to 100%: red

Boot.msg File - contains information, including the current version, during the initial starting of LoadMaster.

Warning Message File - contains warnings logged during the operation of LoadMaster.

System Message File - contains system events logged during the operation of LoadMaster. This includes both operating system-level and LoadMaster internal events.

Nameserver Log File - show the DNS name server log.

Nameserver Statistics - show the latest name server statistics.

IPsec IKE Log - show the IPsec IKE log.

Note: The IPsec IKE Log button does not appear if VPN Management is not configured under System Configuration > Network Setup > VPN Management.

WAF Debug Log File - contains debug traces useful for debugging WAF issues. Only enable this option when requested to do so by Progress Kemp Technical Support.

Note: The WAF Debug Log File button does not appear if there are no WAF debug logs.

WAF Event Log File- contains logs for most recently triggered WAF rules.

Note: The WAF Event Log File button does not appear if there are no WAF event logs.

Audit LogFile - contains a log for each action which is performed by a user; either using the API or the WUI. This will only function if session management is enabled. For further information on session management, refer to the Admin WUI Access section.

Clear Logs - clears the warning and system message logfiles. You can either click Clear All to clear all the system log files or click the arrow to select specific log files to clear.

Save Logs - this option allows you to send saved system log files to Progress Kemp Support. Click Save All to save all system log files to your computer. The logs are compressed. You can view/read them using a text editor after you uncompress the file.

The LoadMaster automatically rotates logs so the hard disk does not fill up with log files.

Debug Options

The LoadMaster has a range of features that will help you and Progress Kemp Support staff with diagnosing connectivity issues. Clicking Debug Options brings up the screen shown below.

Note: WARNING – Progress Kemp does not recommend using debug commands during normal operation. They should ideally only be used in conjunction with a Progress Kemp Support Technician’s recommendations.
Note: Debug commands have performance impacts on the LoadMaster and may expose your system to additional security vulnerabilities during the time they are running.

Disable All Transparency

Disables transparency on every Virtual Service and forces them to use Layer 7. Use with caution.

Note: This option is only for debugging and does not replace the normal controls to enable and disable transparency on a per-Virtual Service basis.
Note: Using this option to disable transparency saves a copy of the configuration file before disabling transparency. When transparency is turned back on (not all Virtual Services may have had transparency turned on before the change), the original configuration is restored. Therefore, any changes to the configuration during this time are lost. This includes creating new Virtual Services.

Enable L7 Debug Traces

This option enables debugging on all Layer 7 (L7) connections.

Generates log traffic in the message files. Due to the large number of files being logged it slows down L7 processing.

CAUTION: Enabling this option can consume more resources and it is possible that some authorization parameters may be exposed. Only enable this option if recommended by Progress Kemp Support.

Enable Extended L7 Debug

Click Enable Extended Debug to enable L7 extended debug options.

In LoadMaster firmware version 7.2.53, the Enable Extended L7 Debug option was enhanced. Enabling this option may be needed when performing extensive testing.

Once the Enable Extended L7 Debug option is enabled, a Process Debug button appears. Clicking Process Debug displays the list of processes and the debug level.

The user has to sanitize the logs before providing them externally. Logs should only be enabled (debug level is set to 1) for debug purposes and disabled (debug level is set to 0) immediately after. Logs should be removed from the LoadMaster as soon as possible when the debugging is completed.

CAUTION: Enabling the Extended L7 Debug option can consume more resources and it is possible that some authorization parameters may be exposed. Only enable this option if recommended by Progress Kemp Support.

When extended debugging is enabled, an additional Extended Debug configuration section becomes available in the Virtual Service modify screen (Virtual Services > View/Modify Services > Modify) for all VSs. When using Sub-Virtual Services (SubVSs), the Extended Debug settings are also inherited by the SubVS, so that a single call can be logged in its entirety. It is also possible to enable debug on a single SubVS if required and a further option to limit the logging by specifying the client IP address. All logs associated with this feature are recorded in the system messages file messages.txt.

There are two fields that you can configure in this section:

  • L7 Debug Level: Set the level of Layer7 debugging for this Virtual Service. Possible values are as follows:

    • No Debug

    • Call Tracing

    • Full Debug

    • Full Debug + HTTP Headers

    Call Tracing is a basic level log that displays most relevant operations, while Full Debug displays all available debug logs, which is the same as the global setting of Enable L7 Debug Traces but on a per-VS level.

    Note: By default, the L7 Debug Level is set to No Debug for all Virtual Services and SubVSs. To enable logging for a particular Virtual Service or SubVS, you must set the L7 Debug Level to Call Tracing or Full Debug in the Extended Debug section of the Virtual Service or SubVS modify screen.
    CAUTION: Setting the L7 Debug Level to Full Debug + HTTP Headers may expose sensitive information.

  • Client to Trace: It is also possible to limit the debug information even further by specifying a client IP address (you can specify an IPv4 or IPv6 address). If an address is specified, only connections coming from that specific client IP are logged/traced. This allows debugging capability from a single address.

Enable IRQ Pinning

Click the button to enable Interrupt Request Line (IRQ) pinning. This is disabled by default.

CAUTION: Only enable this option in consultation with Progress Kemp Support.
Note: When you change the IRQ pinning option from off to on, IRQ pinning is enabled on all network interfaces that are assigned an IP address. When IRQ pinning is enabled and you add an IP address to an unconfigured interface, that interface will not have IRQ pinning enabled until you either toggle the IRQ pinning off and back on again, or the system is rebooted.

Perform an l7adm

Note: Displays raw statistics about the L7 subsystem.

Enable WAF Debug Logging

You can enable logging to get detailed information about traffic that is analyzed by Web Application Firewall. Information that is contained in the logs includes the time that LoadMaster WAF received the request from your LoadMaster resource, detailed information about the request, and the action for the rule that each request matched.

Enable WAF debug traces.

CAUTION: Please be aware that enabling this option will generate logs that may include Personally Identifiable Information as defined under the General Data Protection Agreement (EU GDPR). You should follow your organization's best practice to protect this information which may include anonymizing, deleting, or encrypting the data within the logs.
Note: This generates a lot of log traffic. It also slows down WAF processing. Only enable this option when requested to do so by Progress Kemp Technical Support. Kemp does not recommend enabling this option in a production environment.

The WAF debug logs are never closed and they are rotated if they get too large. WAF needs to be disabled and re-enabled in all WAF-enabled Virtual Service settings to re-enable the debug logs. Alternatively, perform a rule update, with rules that are relevant for the Virtual Service(s).

Enable IRQ Balance

Enable this option only after consulting with Progress Kemp support staff.

Enable TSO

Enable TCP Segmentation Offload (TSO).

Note: Only modify this option after consultation with Progress Kemp Technical Support. Changes to this option will only take affect after a reboot.

Enable TCP SACK

Click this button to enable TCP SACK (Selective ACKnowledgement) processing. This is a global setting that affects all Layer 7 Virtual Services. It only works if TCP SACK is enabled on a Virtual Service client and the LoadMaster.

Enable Layer 4 IPv6 Forwarding

This option, when enabled (the default), supports IPv6 forwarding behaviour as supported on LoadMaster versions previous to LoadMaster Operating System (LMOS) 7.2.50. When this option is disabled, it supports the IPv6 forwarding behavior required by the IPv6 standards. If IPv6 forwarding behavior that conforms to the IPv6 standards is required in your deployment, you should disable this option.

Enable/Disable CLI VS Management

Enable or disable the Command Line Interface (CLI) Service Management function.

GEO Debugging

Clicking GEO Debug displays a GEO Debug page. The GEO Debug page displays the GEO configuration related information and various GEO Debug Options. For more details on GEO Debugging, refer to the GEO Debug section.

Display RAID Information

Note: The Display RAID Information and Display RAID Disks Information buttons only appear if a RAID controller is installed on the LoadMaster.

Display the Redundant Array of Independent Disks (RAID) controller details. Some example information is below:

-------------------------------------------------------------------
Controller details
-------------------------------------------------------------------
- Chip ID................: 10
- Parent Controller Index: 255
- OS Physical Name.......: /dev/sda
- Serial Number..........: 427491329
- AES Power on State.....: 0
- Sata Ports.............: 2
-------------------------------------------------------------------
Raid Port 0 details
-------------------------------------------------------------------
- Raid Model Name..............: H/W RAID1
- Raid Serial Number...........: OUEYEXCXTQ53GE1BSOSN
- EZBackup Disk Support........: 0
- Port Multiplier port.........: 0
- Raid Capacity................: 953 (29 GB)
- Raid Capacity low word.......: 0
- Raid State...................: 1 (Active)
- Raid Status..................: 3 (Normal)
- Raid Level...................: 1 (Raid 1 (mirror))
- Mark Type....................: 0
- Active Member................: 15
- Active Level.................: 0
- Rebuild Priority.............: 3
- Standby Timer................: 0
- Total members in the RAID....: 2
Member disk 0
- Ready....................: 1
- Lba 48 Bit Support.......: 1
- SATA Page................: 0
- SATA Port................: 0
- SATA Base................: 0
- SATA Size................: 953
----------------------------------------------------------
Member disk 1
- Ready....................: 1
- Lba 48 Bit Support.......: 1
- SATA Page................: 0
- SATA Port................: 1
- SATA Base................: 0
- SATA Size................: 953
----------------------------------------------------------

Display RAID Disks Information

Display details about the RAID disks. Some example information is below:

-------------------------------------------------------------------
Sata Port 0 details
-------------------------------------------------------------------
- Disk Model Name..............: 32GB SATA Flash Drive
- Disk Serial Number...........: C0122916B01000000074
- Disk Firmware Version........: SFDC001D
- EZBackup Disk Support........: 1
- Port Multiplier port.........: 15
- Disk Capacity................: 954 (29 GB)
- Port Type....................: 2 (RAID)
- Port Speed...................: 2 (GB)
- Page 0 State.................: 2
- Page 0 Raid Index............: 0
- Page 0 Member Index..........: 0
- Page 0 Raid Name.............:
- Page 0 Raid Serial Number....:
- Page 0 Raid Segment Base.....: 0
- Page 0 Raid Size.............: 953
- Page 0 Raid EZ Backup Support: 0
- Page 1 State.................: 0
- Page 1 Raid Index............: 0
- Page 1 MemberIndex...........: 0
- Page 1 Raid Name.............:
- Page 1 Raid Serial Number....:
- Page 1 Raid Segment Base.....: 0
- Page 1 Raid Size.............: 0
- Page 1 Raid EZ Backup Support: 0
- PortErrorStatus..............: 0
-------------------------------------------------------------------
Sata Port 1 details
-------------------------------------------------------------------
- Disk Model Name..............: 32GB SATA Flash Drive
- Disk Serial Number...........: E011321290100000005A
- Disk Firmware Version........: SFDC001D
- EZBackup Disk Support........: 1
- Port Multiplier port.........: 15
- Disk Capacity................: 954 (29 GB)
- Port Type....................: 2 (RAID)
- Port Speed...................: 2 (GB)
- Page 0 State.................: 2
- Page 0 Raid Index............: 0
- Page 0 Member Index..........: 1
- Page 0 Raid Name.............:
- Page 0 Raid Serial Number....:
- Page 0 Raid Segment Base.....: 0
- Page 0 Raid Size.............: 953
- Page 0 Raid EZ Backup Support: 0
- Page 1 State.................: 0
- Page 1 Raid Index............: 0
- Page 1 MemberIndex...........: 0
- Page 1 Raid Name.............:
- Page 1 Raid Serial Number....:
- Page 1 Raid Segment Base.....: 0
- Page 1 Raid Size.............: 0
- Page 1 Raid EZ Backup Support: 0
- PortErrorStatus..............: 0

Reset Statistic Counters

Reset all statistics counters to zero and delete any old graphs. This also deletes the Round Robin Database (RRD) files but these files are automatically recreated when needed.

Flush OCSPD Cache

When using OCSP to verify client certificates, OCSPD caches the responses it gets from the OCSP server. This cache can be flushed by pressing this button. Flushing the OCSPD cache can be useful when testing, or when the Certificate Revocation List (CRL) has been updated.

Stop IPsec IKE Daemon

Stop the IPsec IKE daemon on the LoadMaster.

Note: If this button is clicked, the connection for all tunnels will go down.

Perform an IPsec Status

Display the raw IPsec status output.

Enable IKE Debug Level Logs

Control the IPsec IKE log level.

Flush SSO Authentication Cache

Clicking the Flush SSO Cache button flushes all Single Sign-On (SSO) records, resets all authentication server statuses, resets the KCD domain (if relevant) and re-reads the configuration. This has the effect of logging off all clients using Single Sign-On to connect to the LoadMaster.

SSO Internal State

Click SSO State to show the internal state for the SSO Manager.

SSO LDAP server timeout

Set the SSO LDAP server timeout value in seconds (default value is 5 seconds).

Linear SSO Logfiles

By default, older log files are deleted to make room for newer log files, so that the filesystem does not become full. Selecting the Linear SSO Logfiles check box prevents older files from being deleted.

Note: When using Linear SSO Logging, if the log files are not periodically removed and the file system becomes full, access to ESP-enabled Virtual Services will be blocked, preventing unlogged access to the virtual service. Access to non-ESP enabled Virtual Services are unaffected by the Linear SSO Logfile feature.

Kill LoadMaster

Permanently disables all LoadMaster functions. The LoadMaster can be re-enabled by being relicensed.

Note: Please do not kill your LoadMaster without consulting Progress Kemp Technical Support.
Note: The Kill LoadMaster option will not be available in LoadMasters which are tenants of the Progress Kemp Multi-Tenant LoadMaster.

Enable DHCPv6 Client

When this option is enabled, the DHCPv6 client will run on the primary interface. This provides the capability to obtain an IPv6 address on boot. If you want DHCPv6 to be run on every boot, keep this option enabled. However, this is a long running process and it keeps running in the background when it is enabled so if you only need an IPv6 address to be assigned and you do not need to renew and release the IPv6 address you should disable this option after the IPv6 address is assigned.