Backup/Restore Utility

Capabilities

The utilities can back up and restore the following:

• The MOVEit Transfer configuration database (MySQL only), configuration registry keys, folder structure, custom color schemes, logos, and templates.
• Encrypted files (they are not backed up by default), and Server and client SSL certificates (they are backed up by default) to create a complete backup of the MOVEit Transfer state.

The backup and restore utilities automatically determine where the critical MOVEit Transfer files are located on the server, as well as how to connect to the MySQL database to perform lock operations. Administrators can override these automatic detections.

Backups and restorations can be done across differing types of MOVEit Transfer installations (but NOT differing versions), such as web farm and standalone installations, and installations with differing path structures. This capability requires the backup and restore utilities to NOT backup MOVEit Transfer installation path information, database access information, or email server settings. These are typically configured on a per-installation basis and should not be replicated across distinct servers.

Limitations

The backup utility's ability to back up the MOVEit Transfer database tables applies only to MySQL databases. When using Microsoft SQL Server as the database for MOVEit Transfer, the utility must be run with the --without-database option, and you must back up the MOVEit Transfer database using other tools. If the utility detects that SQL Server is being used, and the --without-database option has not been specified, it will display an error message and exit.

The backup utility obtains a read lock on all MOVEit Transfer database tables before backing up the MySQL database information. This is to ensure that the data does not change during the backup process. As a result, any requests that come into MOVEit Transfer while the tables are locked will wait until the tables are available again before continuing (the read locks are released after the database tables have been backed up). For this reason, it is best if the backup process is run during off-hours.

The backup and restore utilities do NOT back up NTFS permissions, NT users/groups, or IIS settings. If you are using these utilities to maintain a "hot standby" —you must set up these items on the second machine ahead of time. (Frequently, people create a hot standby by restoring a full tape backup onto a new machine, then use periodic runs of the backup utility to keep data on the hot standby fresh.)

The backup utility cannot override the do not export private key setting you might have set when you imported your SSL server certificates. If the utility encounters an SSL server certificate export error related to private keys, it will immediately print the error encountered and then exit. You can use the --ignore-cert-export-errors option to prevent the utility from exiting like this; however, the problem certificates will still not be backed up.

Get the Backup and Restore Utilities

The backup and restore utilities are installed by the MOVEit Transfer setup program, and are placed in the MOVEitDMZ\Scheduler folder. Filenames are DMZBackup.exe and DMZRestore.exe. The programs must be in this location in order to use some requisite libraries provided by MOVEit Transfer.

In addition to the backup and restore utilities, an additional application is installed for use by the utilities:

• ExportCerts.exe: an SSL certificate extracting application that puts Microsoft SSL certificate stores into a format that is usable by the backup and restore utilities.

Using the Backup Utility

Open a command prompt and cd to the Scheduler subdirectory of your MOVEit Transfer non-web directory. Execute the following command.

dmzbackup

This creates a file named MOVEitDMZ_Backup_xxxxxxxx.7z in your Scheduler directory, where xxxxxxxx is the current date in YYYYMMDD format. This file will contain all the necessary files and information to reconstruct your MOVEit Transfer configuration. It will not backup the at-rest encrypted files in your MOVEit Transfer configuration, however, it will backup the existing folder structure. To backup the encrypted files as well, add the --with-files command-line option to the above command.

Because doing a complete backup with files requires a large amount of free disk space on the server (approximately twice as much free disk space as the size of your \MOVEitDMZ\Files directory), many customers opt to run a config-only backup, and then backup the encrypted files in a different way (NTBackup, read-only FTP server, etc.). Customers who use this method should add the --with-file-tables option to force the backup utility to back up the file-related database tables, which are normally skipped when a config-only backup is performed.

If you would like to see what the utility is doing while it runs, add the --debug command-line option to the above command. For a complete list of the options available in the MOVEit Transfer backup utility, see the Backup Utility Commands section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitDMZ\Scheduler>dmzbackup --help

Backup Utility Commands

The MOVEit Transfer Backup Utility is a .NET console application that runs in the Scheduler directory on a MOVEit Transfer system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command dmzbackup --help. The default values listed for the database location and DMZ directories are gathered from the registry on the MOVEit Transfer system. The values may be overridden by using the command-line options.

C:\MOVEitTransfer\Scheduler>DMZBackup.exe --help

Commandline Help for DMZBackup (help for utility shipped with 2020 release shown)

MOVEit Transfer Backup Utility

Usage: dmzbackup [options]

Options:

  --backup-database-settings

    By default the database connection settings are NOT backed up.

    It is assumed that the system being restored to already has

    its desired database connection configured. However, the

    --backup-database-settings option can be used to include the

    database connection settings in the backup file.

  --dbdatalocation=<directory>

    Location of database data files (default C:\MySQL\data\)

  --debug

    Turn debug mode on

  --dmznonwebdir=<directory>

    DMZ non-web directory path (default C:\MOVEitTransfer)

  --dmzprogramfilesdir=<directory>

    DMZ Program Files path (default 'C:\Program Files\MOVEit\MOVEitTransfer')

  --dmzwebdir=<directory>

    DMZ web directory path (default C:\MOVEitTransfer\wwwroot)

  --exit-on-cert-export-errors

    Exit immediately if an error is detected while backing up SSL

    client and server certificates. This option has no effect if

    --without-certs is enabled.

  -h, --help

    Display this help screen and exit

  --ignore-cert-export-errors

    Do not exit immediately if an error is detected while backing

    up SSL client and server certificates. This option has no effect

    if --without-certs is enabled. (default)

  --output=<output_filename>

    Relative or absolute output filepath

    (default MOVEitDMZ_Backup_YYYYMMDD.[zip|7z])

    The date macros [YYYY], [MM], and [DD] may be used and will

    be replaced by the current year, month, and day, respectively.

    The time macros [HH], [TT], and [SS] may also be used and will

    be replaced by the current hour, minute, and second respectively.

  --tempdir=<directory>

    Base temporary directory to use 

    (default C:\Users\Administrator\AppData\Local\Temp\2\)

  --with-certs

    Back up SSL client and server certificates from the Microsoft

    Certificate Store. Certificate backup files will be encrypted. (default)

  --without-certs

    Do not back up SSL server and client certificates

  --with-files

    Back up encrypted files along with DMZ filesystem structure

  --without-files

    Do not back up encrypted files along with DMZ filesystem structure (default)

  --with-filesystem

    Back up encrypted DMZ filesystem structure (default)

  --without-filesystem

    Do not back up encrypted DMZ filesystem structure

  --with-file-tables

    Back up the file tables from the database. This is implied when using

    --with-files, but is not on by default when --without-files is in effect.

    Since --without-files is on by default, file tables are not backed up by

    default.

  --without-database

    Do not back up database (use if managing database backup by a

    different method)

  --without-ftpnatmap

    Do not back up FTP NAT mappings (use if backup DMZ has different

    IP address)

  --without-ipbindings

    Do not back up IP bindings for FTP and SSH services (use if backup

    DMZ has different IP address)

NOTE: Paths with spaces can be entered by surrounding the entire argument 

with quotes. For example:

    "--tempdir=D:\Temp Dir\Sub Folder"

Backup Utility Specifics

The backup utility creates a backup file by copying all desired files, folder structures, and settings into a temporary directory, and then archiving the contents of that directory using either Zip or 7-Zip. The temporary directory is then removed at the end of the procedure. The backup utility performs the following actions:

• The temporary directory is randomly generated and then created. All backup files will be copied to this location, and then archived.
• Most registry keys in the HKLM\SOFTWARE\Standard Networks\siLock key are read up and copied to a Windows Registry Editor compatible file, which is placed in the temporary directory. Registry keys containing paths, database information, and administrator email information are skipped to allow restoring to machines with different configurations than the original.
• If the MySQL database is being backed up, a read lock is obtained on all MOVEit Transfer database tables, and the appropriate table data is exported to the temporary directory using the mysqldump utility. If the --with-files or --with-file-tables options are specified, all table data will be exported. Otherwise, the files, newfiles, folderfile, and log tables are skipped. Once all table data has been exported, the read locks are released.
• The complete encrypted filesystem folder structure is copied to the temporary directory. If the --with-files option is specified, the encrypted files themselves will also be copied, in their appropriate locations. If the --with-files or --with-file-tables options are specified, per-organization tamper checking hash information files will also be copied.
• The contents of the images subdirectory of the DMZ web directory is copied to the temporary directory to preserve any custom images provided by the server operator or organization administrators.
• Any custom CSS stylesheets, used to create MOVEit Transfer color schemes, are copied to the temporary directory.
• Any custom templates are copied to the temporary directory.
• The ExportCerts.exe utility program is run to export all SSL certificates in the Microsoft Certificate Store and store them as files in the temporary directory.
• The contents of the temporary directory are archived and optionally compressed into a single file. The temporary directory is then removed.

Using the Restore Utility

Prerequisites:

• Install MOVEit Transfer on your target system.

  For best results, the target system and the backed-up system should use the same MOVEit Transfer version.

  Note: If you are using different version; for example, if migrating the MOVEit Transfer server to a new platform and upgrading to a new version, then after the restoration, you must re-run the MOVEit Transfer installation program and choose the Repair option.
• Transfer the backup file to the target system.

Running the restore utility

1. In a command prompt, cd to the Scheduler subdirectory of the MOVEit Transfer non-web directory.
2. Issue the following command:

dmzrestore --debug path-to-your-backup-file

where

• path-to-your-backup-file is your correct path
• --debug shows the actions of the command as it runs. (Optional)

  The restore utility stops your webserver and database, decompress the backup file to a temporary location, copies the backed up files to their appropriate locations, and then restarts the database and webservers.

For a complete list of the options available in the DMZ restore utility, see the MOVEit Transfer Restore Utility section below, or execute the following command in your Scheduler subdirectory:

C:\MOVEitTransfer\Scheduler>DMZRestore.exe --help

Restore Utility Commands

The DMZ Restore Utility is a .NET console application that runs in the Scheduler directory on a MOVEit Transfer system. The utility is command-line driven, so it can easily be integrated into a batch file. A list of available options can be generated by entering the command dmzrestore --help. The default values listed for the database location, DSN, and DMZ directories are gathered from the registry on the DMZ system. The values may be overridden by using the command-line options.

C:\MOVEitDMZ\Scheduler>dmzrestore --help

Commandline Help for DMZRestore (help for utility shipped with 2020 release shown)

MOVEit Transfer Restore Utility

Usage: dmzrestore [options] <dmzbackupfile>

Options:

  --dbdatalocation=<directory>

    Location of database data files (default C:\MySQL\data\)

  --debug

    Turn debug mode on

  --dmznonwebdir=<directory>

    DMZ non-web directory path (default C:\MOVEitTransfer)

  --dmzwebdir=<directory>

    DMZ web directory path (default C:\MOVEitTransfer\wwwroot)

  -h, --help

    Display this help screen and exit

  --tempdir=<directory>

    Base temporary directory to use (default C:\Users\Administrator\AppData\Local\Temp\2\)

  --with-files

    Restore encrypted files along with DMZ filesystem structure if they exist

    in the backup file (default)

  --without-files

    Do not restore encrypted files along with DMZ filesystem structure even if

    they exist in the backup file

NOTE: Paths with spaces can be entered by surrounding the entire argument with quotes.

For Example:

  "--tempdir=D:\Temp Dir\Sub Folder"

Restore Utility Specifics

The restore utility, like the backup utility, first creates a temporary directory. It then extracts all files from the backup file into the temporary directory, and then copies the backed up files to their appropriate locations, overwriting any existing files. In order to correctly perform the restore, it first stops the webserver and database server. The services are restarted once the restore is complete, and the temporary directory is removed. The restore utility performs the following actions

1. The temporary directory is randomly generated and then created. The backup file is extracted to this location.
2. The IIS webserver is stopped to allow proper restoration of appropriate files.
3. The keys contained in the registry backup file are restored into the window registry.
4. The database export is read back into the database, restoring the table structure and data entries.
5. The encrypted filesystem folder structure, and any files backup up, are copied into the appropriate encrypted filesystem directory.
6. The contents of the backed up images directory are copied into the appropriate images directory.
7. Any backed up custom stylesheets are copied to the appropriate templates directory.
8. Any backed up custom templates are copied to the appropriate templates directory, automatically creating any necessary custom subdirectories.
9. Any backed up SSL certificates are copied to the Certs\C00X subdirectory of the DMZ non-web directory, where X is 0 (meaning the server is standalone). From here, the MOVEit Transfer Helper service will process the certificate files and load them into the Microsoft Certificate Store.
10. The IIS webserver is restarted, and the temporary directory is removed.

Creating an Automated Backup Process

It is best practice to back up the configuration structure of a MOVEit Transfer server every night, preferably during off-peak hours. The following steps show how to create a backup process using the DMZ Backup Utility, the MOVEit Xfer command-line secure transfer client and the Windows Scheduled Tasks system. This backup process requires the MOVEit Transfer web service to be up. The process will consist of a batch file that stops the web server, backs up the DMZ system, restarts the web server, and finally uploads the backup file to a location on the DMZ server itself. From there, the backup file should be pulled down and stored in a safe location (the MOVEit Automation super-client is recommended for this step).

Preparing for the Backup Process

Do the following before you create the backup process.

1. Obtain and install all necessary software:

   On the MOVEit Transfer server:

   • The DMZ Backup Utility
   • MOVEit Xfer ( a free command-line secure transfer client).
2. Create a backup user to handle uploads, temporary storage, and downloads of backup files. See below for instructions. This user should only be allowed to access MOVEit Automation from the internal machine that will be handling the downloading of backup files from the system.

Create a backup user:

1. Log on to MOVEit Transfer as an administrator.
2. Switch to the Users page and click Add User.
3. Enter a username such as backup_user and fill in the rest of the user account information. If you do not want this user to receive emailed notifications, leave the email address field blank. Set the permission level to User. Click Add User.
4. In the left pane, click Users, and in the list, click the username to open the user profile. In the User Authentication section, under the Remote Access line, click Select Ruleset.
5. On the Remote Access Settings page, under the Select Ruleset line, select Use Custom Rules and click Change Remote Access Settings.
6. Next to the Use Custom Rules radio option, click the View Custom Rules link. .Add a rule to allow access from the host that will be handling the downloading of backup files from the system.
7. You might also want to restrict this user to the HTTP Clients interface only and/or exempt this user from password expiration.

Backup Process Batch File

The batch file commands that will execute the backup process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols. You may also remove or add any additional -- options you want to be part of the process.

rem *** AUTOMATED MOVEit DMZ Backup Procedure *** SET Task=DMZBackup del %Task%.err 

rem Execute the backup process. This will create rem a file with the name MOVEitDMZ_Backup_YYYYMMDD.7z, rem where YYYYMMDD is the current year, month, and day. rem A log of the results and another containing just the errors will also be saved

dmzbackup > %Task%.Log 2>%Task%.Err --tempdir=<temp directory> --without-files

rem If there were any errors in the logs, notify the administrators 

for /F %%A in ("%Task%.Err") do If %%~zA equ 0 del %Task%.Err 

IF EXIST %Task%.Err GOTO errors 

GOTO ok 

:errors 

echo ERRORS 

ReportErrors.exe %Task% %Task%.err %Task%.Log 

GOTO done 

:ok echo OK 

rem If all OK... rem Upload the file to the DMZ server 

xfer -quiterror -z -user:<backup account username> 

-password:<backup account password> -s:dmz_bkup_commands.txt localhost 

rem If there were transfer errors, notify the administrators 

IF NOT %ERRORLEVEL%==0 GOTO xferbad 

:xferok 

rem Delete backup file 

del /Q MOVEitDMZ_Backup_* 

GOTO done 

:xferbad 

ReportErrors.exe %Task%_During_Xfer %Task%xfer.Log %Task%xfer.Err 

GOTO done 

:done

MOVEit Xfer Commands File

The command file that MOVEit Xfer will execute to upload the backup file should have the name dmz_bkup_commands.txt. It is shown below.

<backup-user-name><backup-user-password>put MOVEitDMZ_Backup_*quit

Scheduling the Backup Process

The Windows Scheduled Tasks service is used to schedule the running of the backup process. The process should be scheduled during off hours. Follow these steps to schedule the backup process:

1. Click Start | Programs | Accessories | System Tools | Scheduled Tasks. This will bring up the Scheduled Tasks window.
2. Double-click Add Scheduled Task. This will start the Schedule Task wizard. Click Next to start the wizard.
3. Click Browse to locate the backup process batch file. Click Next.
4. Select how often to run the backup process, and at what times. It is best practice to run the backup process nightly.
5. Select an appropriate Windows user account to run the backup process as. The account should have permissions to create directories and files in the temp directory, and to read files from the various MOVEit Transfer directories. It should also have permissions to run the MOVEit Xfer program, as well as the ZIP program found in the MOVEitDMZ\Utilities directory.
6. Click Finish to complete the setting up of the schedule task.

Finished

The automated backup process should now be set up and ready to run. You may test the process by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting Run.

Handling the Backup File

At this point, a client on the internal network can be used to pull the backup file off of the MOVEit Transfer system and store it to a safe location, such as a fileserver location that gets regularly backed up to tape. MOVEit Automation makes automating this task simple and effective. A MOVEit Automation system on the internal network can be used to pull down the backup file at some time after the file is created, and then forward that file off to a safe location. Follow the steps below to configure MOVEit Automation to perform this task.

1. Make sure MOVEit Automation has the target MOVEit Transfer server configured as a DMZ host. Also make sure that whatever destination system you choose for the file is also configured as a host.
2. Create a new Task for the backup file gathering task.
3. Create a Source element and set the source as the target MOVEit Transfer server. If the DMZ server is already being used in other MOVEit Automation tasks, you should probably set the source to use a non-default user account, and then enter in the backup account username and password. Browse the DMZ server and select the backup user's home folder. Turn on the option to delete the source file after successful transfer.
4. Create a Destination element and set it to the location you wish to forward the file to. The name the file will end up with at the destination can be configured at this point using the Filename setting. If the destination is where the backup file will be stored, it is a good idea to include the date of the backup in the filename, with the four-digit year first for easier sorting. In that case, Macros can be used to generate the correct filename: moveitdmz_backup_[yyyy][mm][dd].7z. If the destination is a hot-standby machine that will be restoring from the backup file, the filename should be a fixed name, to allow an automated restore process to function correctly. In this case, a name such as moveitdmz_backup.7z is recommended. Also, in this case, the Overwrite Existing Files option should be turned on.
5. Create a schedule element and set it to run approximately one hour after the backup process is scheduled to run on the DMZ server.
6. If you wish to test the Task, you can do so by right-clicking on it and selecting Run Now. You should place a test file in the correct location to make sure the file transfers are working correctly.

Creating an Automated Restore Process

An automated restore process can be useful for keeping an up-to-date hot-standby copy of MOVEit Transfer for use during loss of the primary DMZ system. The process relies on one or more clients such as MOVEit Automation to transfer the backup file from the primary DMZ to the backup DMZ. Once the backup file has been transferred to the backup DMZ, the restore process will download the file and restore it to the local system.

Preparing for the Restore Process

A copy of all the needed software should be obtained and installed. This should include the DMZ Restore Utility and a copy of MOVEit Xfer. No user configuration should be necessary, assuming the hot-standby machine has previously been manually synced to the primary machine. This process will use the same user account that the backup process used.

In order to allow the restore process to download and process the backup file correctly, it should always be put on the server with a constant name. Using MOVEit Automation, this can be accomplished by configuring the Filename section of the Destination element to a fixed name, such as moveitdmz_backup.7z.

Restore Process Batch File

The batch file commands that will execute the restore process are shown below. These commands should be copied to a batch file located in the MOVEitDMZ\Scheduler directory. Appropriate values must be substituted in for the pseudo-values surrounded by less-than (<) and greater-than (>) symbols.

rem First, download the backup file from the rem DMZ server xfer -z -user:<backup account username> -password:<backup account password> -s:dmz_rstr_commands.txt localhost 

rem Next, execute the restore process.. dmzrestore --tempdir=<temp directory> <DMZ backup filename> 

rem Delete backup file del /Q <DMZ backup filename>

MOVEit Xfer Commands File

The command file that MOVEit Xfer will execute to download the backup file and then remove it from the DMZ server should have the name dmz_rstr_commands.txt. It is shown below.

prompt 

mget moveitdmz_backup_*.7z 

mdel moveitdmz_backup_*.7z

Scheduling the Restore Process

As with the backup process, the Windows Scheduled Tasks service is used to schedule the running of the restore process. The process can be scheduled for any time after the file is scheduled to arrive on the DMZ server, since this DMZ server should be a hot-standby, and not currently in production use. Follow the directions in the Creating an Automated Backup Process section to create a scheduled process.

Finished

The automated restore process should now be set up and ready to run. You may test the process (assuming you have a file to download) by right-clicking the scheduled task entry for the backup process in the Scheduled Tasks window and selecting Run.

Disaster Recovery

With backup and restore processes properly configured to keep a hot-standby machine synced with a primary machine, as shown above, an organization can easily provide redundant DMZ services. If the primary machine goes down for any reason, the hot-standby should be able to jump in without any further configuration, since it should always be running and synced to the primary. The only configuration required to change over would be in the routing and DNS systems, to point customers at the standby instead of the primary.

Batch Files to Start and Stop MOVEit Services

Two batch files in the Scheduler folder allow administrators to easily start and stop all MOVEit Transfer services in a safe order with a single command.

• StartMOVEitDMZ.bat - Starts all MOVEit Transfer services, including MySQL
• StopMOVEitDMZ.bat - Stops all MOVEit Transfer services. MySQL will be stopped only if the administrator answers Y to the stop MySQL? prompt.