Folders and Files

There is no specifically defined limit for the number of files or folders MOVEit Transfer can expect to manage. The number of files that can be stored is limited by the amount of disk space available to the folder defined as the Files folder on the Paths tab of the MOVEit Transfer Config Utility.

Folder information is stored in the MOVEit Transfer database. Folder scale is limited by the scaling and storage capacity of the database. If the database contains a very large number of folders (over 1 million), folder navigation and list commands might take longer (it takes the database more computing resources to query across a large and deep collection of folder information). Below are some considerations for configuring folders.

Folder Contents and Nested Folders

Folders with large numbers of files and folders which have deep nested trees can make folder navigation and any operations that touch many folders much slower. They can also make quota calculations slower on folders (when disk space and quota policies are enabled).

Recommendations

  • Keep the contents of a single folder to < 300,000 files

  • Keep nested trees of folders to < 7 levels of nesting

Folder Quotas

Using a quota on a large tree of files and folders can increase the time for a quota check to finish before file upload can begin (see section above on Folder Contents)

Recommendations

Use quotas only on folders that require them.

MOVEit Transfer Debug and Audit Logs

System debug logging presents a performance versus log detail tradeoff, which is why MOVEit Transfer provides separate output channels for "audit logging" (audit logs record a trail of file and system access needed to trace the end-to-end file transfer workflow) and "debug logging" (which can slow performance when set to more granular, low-severity, and high detail levels).

Debug log overview

MOVEit Transfer services record log entries for debugging and security purposes. When configured to verbose (the most detailed) output levels, logging can slow and introduce noticeable latency to the MOVEit Transfer system.

Debug log best practice and useful settings

  • Use “Connect” or “User Error” log levels
  • Use Full or All Debug log levels only for short periods of time when troubleshooting MOVEit Transfer errors
  • Use a folder on the local machine to store log files. This protects against potential loss of log data if a remote file share becomes unavailable.
  • See the Status tab in the MOVEit Transfer Config Utility documentation for details on setting log levels

Audit log overview

The Audit Log for MOVEit Transfer is stored in the database and is a record of user interactions with the MOVEit Transfer platform. When the Audit Log grows very large (up to 60 to 100 million rows) MOVEit Transfer performance might be reduced due to lengthy query processing times and resource utilization at the database.

Audit log best practice and useful settings

Users

User management is an important part of a healthy Managed File Transfer (MFT) system. It is best practice to follow the principle of least privilege. Having dormant or forgotten user accounts increases the possibility that an account can be used for purposes other than the original intent.

Recommendations
  • Configure user expiration policies and user lockout policies to automatically lock out and then delete users that have not signed on for a specified period of time.
  • Use temp user roles (when appropriate) and embed user expiration policies into reusable user profiles (which you can apply to these temporary users).

Groups

Recommendations:

Group management should also follow the principle of least privilege. For group management, it is best practice to:
  • Remove user groups that are no longer in use or useful to your goals.
  • Use user groups to apply permissions to designated groups of users (to team resources or folders, for example). For more information, see also Files & Folders recommendations.

Security

Recommendations for security best practices:

  • Use the IP Lockout, User Lockout, and Password Expiration UI controls to apply policy that aligns with your industry and your company's data security and user management standards.

  • Block endpoint access and disable protocol servers that are not used. For example, disable the MOVEit Transfer FTPS service within MOVEit Transfer if the FTPS protocol is not used at your site.

  • Use the Trusted Hosts feature in MOVEit Transfer only for trusted network devices, such as a load balancer or a Web Application Firewall (WAF) (or specific trusted applications such as MOVEit Automation).
    Note: Trusted Hosts should not be used as a mechanism to prevent a wide range of IP addresses from being locked out.

Endpoint or Local Antivirus

Endpoint or antivirus protection should be used as part of overall server hardening for MOVEit Transfers servers. In some cases, endpoint protection may impact MOVEit performance.

Content Scanning

It is not appropriate to use endpoint protection for scanning the files submitted to the MOVEit Transfer file store. For scanning files during upload or download, MOVEit Transfer supports integrating AV and DLP solutions using ICAP. MOVEit Transfer Help - System - Content Scanning provides more details on ICAP integration for scanning transferred files.

MySQL

MySQL may be installed as part of a MOVEit Transfer all-in-one deployment. Refer to the documentation and recommendations from the antivirus or endpoint protection vendor for proper configuration with MySQL.

Recommendations

  • Exclude the MOVEit Transfer file store. The file store path defaults to c:\MOVEitTransfer\files and is configured in the MOVEit Transfer Config Utility Paths tab

    • Files stored in the file store are encrypted during write and may be incorrectly flagged as viruses by some scanners. This could result in file upload errors, missing files, download errors or errors in the MOVEit Consistency Check task.

  • Configure MOVEit content scanning with an AV or DLP system that supports ICAP integration with MOVEit Transfer to scan files as they are uploaded or downloaded

  • The following executables and folder paths may need to be excluded from endpoint scanning if there are runtime errors or if specific executables or services repeatedly fail to start.

    • MIFTPSrv.exe

    • MIDMZHelper.exe

    • SysStat.exe

    • MIDMZSSHSrv.exe

    • DMZCLI.exe

    • CertificateManagementService.exe

    • GatewayTunnelService.exe

    • KeyManagementService.exe

    • Sftpserver.exe

    • MOVEit Transfer Program Files path (may be one of the following):

      • C:\PROGRA~1\MOVEit

      • C:\Program Files\MOVEit

      • C:\Program Files (x86)\MOVEit)

    • Executables within C:\MOVEitTransfer and sub-folders

High Availability (Web Farm) Deployments

The following recommendations are specific to MOVEit Transfer High Availability (Web Farm) deployments. For information on MOVEit Transfer Web Farm installation refer to the MOVEit Transfer Installation Guide
Note: High Availability (HA) Web Farm deployments with nodes distributed across different datacenters is not an officially supported deployment pattern.

Load Balancer

Recommendations

  • Use a load balancing algorithm that is based on node response time instead of simple ‘round robin’ load balancing

  • Session persistence (sometimes referrred to as “'sticky sessions”’) should only be used with FTPS connections. HTTPS and SFTP protocols do not require this option.

  • Configure the IP address of the client to be passed through to MOVEit Transfer. This is important for auditing and security.

Database

Recommendations

  • Use Microsoft SQL Serve Always-On or Azure SQL Premium service tier to provide read-only access to the database.

    • The read-only access can be used for reporting and other read-only queries, which improves the overall responsiveness of MOVEit Transfer.

    • Read-only access is configured on the Database tab in the MOVEit Transfer Config Utility

  • Refer to Microsoft’s SQL Server best practices to ensure optimal configuration of SQL Server and SQL Server Always-On deployments.

  • Azure SQL - Enable automated indexing. This feature of Azure SQL will attempt to modify database indexes specifically for your MOVEit Transfer deployment’s traffic patterns.

Single Node Deployments

The following recommendations are specific to MOVEit Transfer single node deployments. For information on MOVEit Transfer Single Node installation refer to the MOVEit Transfer Installation Guide

  • Start a single node deployment with a remote Microsoft SQL server or Azure SQL if there are plans to expand to a MOVEit Transfer High Availability (Web Farm) deployment in the future

  • Consider migrating from MySQL to a remote Microsoft SQL Server or Azure SQL database if the size of the MOVEit Transfer database in MySQL is approximately 2GB or larger. This is because a large MySQL database will compete against the MOVEit Transfer services for CPU, memory and disk I/O resources.

The size of the MySQL database can be verified by running the following SQL statement against the MySQL database.

SELECT table_schema AS "Database", SUM(data_length + index_length) / 1024 / 1024 AS "Size (MB)" FROM information_schema.TABLES GROUP BY table_schema;

MOVEit Knowledge Base Links