Overview

includes applications that run periodically on the server called Scheduled Tasks. These applications take care of maintaining a system and executing time-based actions such as delayed notifications and password expirations. The applications are run by two tasks in the Windows Scheduled Tasks list, which are added automatically during the installation.

There are two groups of Scheduled Tasks, with some applications being in both groups. The first group is the DayTime group, which, by default, runs every 5 minutes of every day between 2AM and 12AM. Applications in this group are those which need to run throughout the day, such as for issuing delayed notifications, and cleaning up cached entries in the database. The Windows Scheduled Tasks entry that runs this group is called MOVEitDayTimeTask.

The second group is the Nightly group, which, by default, runs every night at 1AM. Applications in this group are mostly responsible for checking the consistency of the system, archiving logs and secure messages, and expiring data. The Windows Scheduled Tasks entry that runs this group is called MOVEitNightlyTask.

In version 3.4 a batch file called RunOneTask was also introduced to allow administrators to selectively run individual tasks. More information about this utility can be found below.

Below is a list of the applications in each group.

DayTime Tasks

DeleteParmFiles

  • Cleans up stale parameter files needed for communication between the application and the MOVEitISAPI file transfer module.

EmailNotify

  • Sends delayed notifications to both senders and recipients.

TableCleanup

  • Deletes stale sessions and folder permissions granted to various sessions.
  • Reenables users and IP addresses locked out for signon violations.

Nightly Tasks

ArchiveLog

  • Archives and then deletes audit log entries from the database once they are older than the configured retention period. Stores the archives in the /Archive/Logs folder on the filesystem. This task also resets tamper-evident hash chains to reflect the new start of each organization's audit logs after logs are archived/deleted.

    To control Archive Log settings, see the following registry settings: ArchiveLogBatchSize, ArchiveLogBatchPauseMS, ArchiveLogTableName.

ArchiveMessages

  • Archives (optional) and then deletes package messages and attachments once they are older than the configured retention period. Stores the archives in the /Archive/Packages folder on the filesystem.

ConsistencyCheck

  • Makes sure records in various database tables are consistent. For example, it may check that a file, the folder that contains the file, and the user who uploaded the file all belong to the same organization.
  • Makes sure database records match file system data and vice versa. In this check, each file and folder record in the database is matched up with a file and folder on the Windows file system.

    If the ConsistencyCheck task finds errors, it will send a notification to the configured administrative e-mail address recommending that a DBFixup be performed to reconcile the errors. Run DBFixup from on the console through the Start | Programs | menu and it will step you through several prompts. You will need the password of the MySQL user that is used by to access the database. This is a password that is set up at installation. For each database error, the DBFixup will either delete the offending table entry or modify it to make it consistent. For file system errors, DBFixup will delete un-matched files and folders.

    Note: If is running in a web farm environment, DBFixup can be run on any node, but only by a user who has access to the shared filesystem on the NAS backend. If DBFixup is not run by a user with the appropriate access, an error message will be displayed noting the requirement.
    Note: DBFixup should be run as administrator.

In some cases, especially if there is a heavy overnight load while the consistency checker is running, it is possible to report false positives. The checker may find files that were in the process of being created, deleted, or moved at the exact time the checker ran. These errors will usually disappear the next day that the ConsistencyCheck task runs. If there is any doubt about a reported inconsistency, there is no harm in just waiting another day. The information in the notification email can also be used to research a file or folder ID through the Web interface.

CreateReports

  • Executes any scheduled reports that should be run and saves the report contents to their configured locations.

DeleteParmFiles

See above.

DeletePendingUsers

  • Removes deleted users from the database once they are no longer referenced by other elements.
  • Expires temporary users and issues warnings about pending temporary user expirations.

EmailNotify

See above.

GarbageCollection

  • Deletes old files from folders with the file cleanup option enabled.
  • Marks new files as not new once they are older than the folder's configured NewTime setting.
  • Deletes partial files older than 12 hours.
  • Deletes old and empty subfolders from folders with the subfolder cleanup option enabled.
  • Marks new secure messages as not new once they are older than the organization's configured NewTime setting.
  • Deletes old, unassigned SSH keys and SSL certs from the holding tank.

PasswordAgeUsers

  • Suspends users whose passwords are older than the configured password aging settings.
  • Sends warning notifications to users whose passwords are within the configured password age warning settings.
  • Sends notifications to interested administrators and GroupAdmins about password expirations and password warning notifications.

SyncLDAP

  • Synchronizes the properties of advanced-LDAP-authenticated users on the server with the associated user records on the LDAP server.
  • Optionally adds user records for users that are found by the LDAP authentication source but do not exist yet on .

SysCheck

  • Checks on the first day of every month to see if any of the system debug levels are at or above the Some Debug level. If so, sends an email message to the Send Errors To email address(es), noting that high debug levels can impact performance.
  • Checks the remaining space available on each local system drive. If any drive has less than a configurable number of megabytes remaining, sends an email message to the Send Errors To email address(es), noting which drives are low on space.

    By default, the minimum drive space remaining value is 1024MB (1GB). This value can be changed through the Config utility.

  • Checks every few minutes that the machine and ISAPI URLs are still valid. The main reason that these URLs suddenly become invalid is that someone makes a security change to the IIS server and neglects to run the Check utility to see whether their change is fatal or not. If either of these URLs time out, return a bad message or otherwise appear unhealthy, SysCheck sends an email message to the Send Errors To email address(es) with some suggestions to resolve the problem.
  • If Content Scanning (Anti-virus and/or Data Loss Prevention) is enabled, checks every few minutes that the scan engine is available. If the scan engine is not available, sends an email message to the Send Errors To email address and warns that the server will not be able to transfer files until this situation is addressed. When the scan engine becomes available again, sends an email that states that scanning is now working.

TamperCheck

  • Goes through all log entries (by organization) and ensures that each organization's chain of cryptographic hashes remains intact. If any tampering is detected, a notification with an explanation and logs are sent to the Send Errors To email address(es).

A Log Tamper Check link that manually starts TamperCheck (and displays running results in a command-line window) is available from the Start menu under the program group. Any TamperCheck that ends with the phrase Completed with errors should be considered a failed TamperCheck; the exact reason for the failure will be explained in the log and in the notification email messages.

Logging and Error Handling

Each one of these applications writes its own log file to the common MOVEit Logs folder. (Typically, this is something like D:\moveitdmz\logs). Each run of generates a new log (subsequent runs do not append to an existing log), so old logs are automatically grandfathered so that up to 5 old copies of each scheduled application's log file are available.

Each scheduler application also writes out a log file specifically for errors that it encounters. These files will end in a .err extension, and will also be automatically grandfathered, just as the normal log files are. When no errors occur, the error log file will be empty. Otherwise, the specifics of the error encountered will be written out in the file. Also, if an error does occur, an email message will be sent to the Send Errors To email address configured for the system informing the recipient which host the error occurred on and which application encountered the error. The contents of the error log file will be included in the body of the email, and the appropriate normal log file for the application will also be attached to the email.

Manually Running a Single Task (RunOneTask)

The daily and nightly sets of scheduled tasks may be run manually as a set at any time through the Windows Scheduled Task interface. However, to run individual tasks within a set, you must use a batch file utility called RunOneTask, also located in your Scheduler directory.

D:\MOVEitDMZ\Scheduler>runonetask 
Usage: RunOneTask NonWebDir moveitDSN TaskToRun

If you have installed into a folder whose name uses spaces, you should use the 8.3 version of any foldernames provided to ensure RunOneTask is run properly. For example, if I have installed into the D:\m i\mi dmz folder (instead of the usual D:\moveitdmz\midmz) then my RunOneTask command should resemble the following example.

D:\m i\mi dmz\Scheduler>runonetask d:\mi09f8~1\midmz~1 moveitdmz syscheck
NonWebDir=d:\mi09f8~1\midmz~1 SchedLogDir=d:\mi09f8~1\midmz~1\logs 
BEGIN One Task Run of syscheck 
	1 file(s) moved. 
	1 file(s) moved. 
END One Task Run of syscheck

MOVEit SysStat Service

In addition to the two groups of Scheduled Tasks, there is one more application which, while not directly scheduled, does run periodically. This is the SysStat service, which is responsible for periodically recording several performance statistics in a table in the database. These values can be used to get a good overall picture of the health of the server.

By default, the SysStat service wakes up every 323 seconds (roughly five minutes) and records samples of the various performance statistics that it keeps track of. The default 323 second sleep period is chosen to make sure the service remains offset from the more even 5 minute schedule of the daytime scheduled tasks. Every 72 cycles, the service also does a complete check of disk utilization for each of the various components. This operation is performed only periodically because the disk utilization values typically change far more slowly than other system performance statistics, and because the disk check takes longer to accomplish and requires more system resources than the sampling of the other statistics. Also for this reason, the SysStat service will not do a full disk utilization check on the first run after it starts up. Instead, the first disk check will be performed at a random future run, and then periodically after that.

The above configuration values are customizable, as is the length of time which the service keeps statistics in the database for (the default is 30 days). The values can be changed on the Miscellaneous tab of the Configuration Utility.

The SysStat service stores its statistics samples in the sysstats table in the database. The fields available in the table are listed below, along with descriptions of their contents. Fields prefixed below with an asterisk indicate those fields which are only populated every 72 cycles by default. During off-cycles, these fields are set to 0.

  • ID - Simple auto-incrementing ID number for each entry
  • StatTime - Date/Time stamp indicating when the samples in the entry were recorded
  • ResilNode - If the system is in a web farm DMZ cluster, this will contain the node number the samples in the entry were recorded on.
  • FilesDriveRootPath - Root drive path of the encrypted file store.
  • FilesDriveSpaceFree - Number of bytes available on the drive containing the encrypted file store.
  • FilesDriveSpaceUsed - Number of bytes used on the drive containing the encrypted file store.
  • *FilesSpaceUsed - Number of bytes used by solely the encrypted file store.
  • DBDriveRootPath - Root drive path of the MySQL database.
  • DBDriveSpaceFree - Number of bytes available on the drive containing the MySQL database.
  • DBDriveSpaceUsed - Number of bytes used on the drive containing the MySQL database.
  • *DBSpaceUsed - Number of bytes used by solely the MySQL database.
  • LogsDriveRootPath - Root drive path of the debug log store.
  • LogsDriveSpaceFree - Number of bytes available on the drive containing the debug log store.
  • LogsDriveSpaceUsed - Number of bytes used on the drive containing the debug log store.
  • *LogsSpaceUsed - Number of bytes used by solely the debug log store.
  • FilesTotalDB - Number of files on the system according to the database records.
  • FilesSizeTotalDB - Total bytecount of files on the system according to the database records.
  • CPUUsagePercentTotal - Total percentage of CPU usage by all running processes.
  • CPUUsagePercentDMZ - Percentage of CPU usage by the web application (aspnet_wp).
  • CPUUsagePercentISAPI - Percentage of CPU usage by the MOVEit ISAPI module. As an ISAPI module, MOVEit ISAPI is run under the DLLHOST.EXE application. The SysStats service automatically determines which running DLLHOST process is responsible for the MOVEit ISAPI module, and determines the CPU usage percentage of that process.
  • CPUUsagePercentIIS - Percentage of CPU usage by the IIS webserver (inetinfo).
  • CPUUsagePercentDB - Percentage of CPU usage by the MySQL database server (mysqld-nt).
  • CPUUsagePercentDMZFTP - Percentage of CPU usage by the FTP server (MIFTPSrv).
  • CPUUsagePercentDMZSSH - Percentage of CPU usage by the SSH server (MIDMZSSHSrv).
  • CPUUsagePercentSched - Percentage of CPU usage by the various scheduler applications (GarbageCollecti, EmailNotify, ArchiveLog, DeletePendingUs, PasswordAgeUser, ArchiveMessages, ConsistencyChec, DeleteParmFiles, SyncLDAP).
  • CPUUsagePercentCentral - Percentage of CPU usage by the application, if running (MICentral).
  • MemUsedTotal - Total bytecount of memory used by all running processes.
  • MemFreeTotal - Bytecount of available memory.
  • MemUsedDMZ - Bytecount of memory used by the web application.
  • MemUsedISAPI - Bytecount of memory used by the MOVEit ISAPI module.
  • MemUsedIIS - Bytecount of memory used by the IIS webserver.
  • MemUsedDB - Bytecount of memory used by the MySQL database server.
  • MemUsedDMZFTP - Bytecount of memory used by the FTP server.
  • MemUsedDMZSSH - Bytecount of memory used by the SSH server.
  • MemUsedSched - Bytecount of memory used by the various scheduler applications.
  • MemUsedCentral - Bytecount of memory used by the application, if installed.
  • VMSizeDMZ - Bytecount of virtual memory used by the web application.
  • VMSizeISAPI - Bytecount of virtual memory used by the MOVEit ISAPI module.
  • VMSizeIIS - Bytecount of virtual memory used by the IIS webserver.
  • VMSizeDB - Bytecount of virtual memory used by the MySQL database server.
  • VMSizeDMZFTP - Bytecount of virtual memory used by the FTP server.
  • VMSizeDMZSSH - Bytecount of virtual memory used by the SSH server.
  • VMSizeSched - Bytecount of virtual memory used by the various scheduler applications.
  • VMSizeCentral - Bytecount of virtual memory used by the application, if installed.
  • HandlesTotal - Total count of handles open by all running processes.
  • HandlesDMZ - Count of handles open by the web application.
  • HandlesISAPI - Count of handles open by the MOVEit ISAPI module.
  • HandlesIIS - Count of handles open by the IIS webserver.
  • HandlesDB - Count of handles open by the MySQL database server.
  • HandlesDMZFTP - Count of handles open by the FTP server.
  • HandlesDMZSSH - Count of handles open by the SSH server.
  • HandlesSched - Count of handles open by the various scheduler applications.
  • HandlesCentral - Count of handles open by the application, if installed.
  • ProcessesTotal - Total count of running processes.
  • ThreadsTotal - Total count of running threads by all running processes.
  • ThreadsDMZ - Count of running threads owned by the web application.
  • ThreadsISAPI - Count of running threads owned by the MOVEit ISAPI module.
  • ThreadsIIS - Count of running threads owned by the IIS webserver.
  • ThreadsDB - Count of running threads owned by the MySQL database server.
  • ThreadsDMZFTP - Count of running threads owned by the FTP server.
  • ThreadsDMZSSH - Count of running threads owned by the SSH server.
  • ThreadsSched - Count of running threads owned by the various scheduler applications.
  • ThreadsCentral - Count of running threads owned by the application, if installed.
  • SessionsTotal - Total ASP.NET sessions registered with .
  • SessionsActive - Total active (touched within last 5 minutes) ASP.NET sessions registered with .

If serious errors occur during a statistics gathering cycle, SysStat will report them by sending an email message to the Send Errors To email address configured on the system, as well as by logging the errors in the Windows Application Event Log. In most cases, SysStat will record the information it was able to gather and continue its work. If SysStat is unable to gather information on a particular field, it will typically record a value of 0 or -1 for that field. The value -1 indicates an unknown error occurred. The value 0 is used in the case of a known error occurring (most often a process that SysStat is gathering information on is not running). Note that a value of 0 for a field does NOT always imply that an error occurred (CPUUsagePercent is often 0 for processes that are running but not doing anything).

A value of -1 is also recorded in certain database- and filesystem-related fields when the database and/or filesystem is remote. This is usually the case when is in a webfarm configuration. Additionally, database performance statistics will only be recorded for a local MySQL database; local and remote SQL Server databases will not be queried for performance statistics.

Helper Service

The Helper service performs a number of utility functions for other services.

  • CA and Client Certificate Management - The web interface services (IIS, etc.) do not run with sufficient privileges to directly alter the Microsoft Certificate Store. The Helper service allows the web interface to indirectly create, import and delete certificates from the store.