Pro2 can automatically detect, terminate, and restart unresponsive Pro2 threads, reducing the need for manual intervention when a thread hangs.

Supported thread types

Automatic hung thread recovery is supported for the following Pro2 thread types:
  • Replication
  • Replication-2ndPass
  • CDC
  • Replication-CDC

About hung thread detection

A Pro2 thread hangs when it stops processing data but continues to appear active. While the thread appears to be running, it no longer transfers records to the target database. This can cause the replication queue to grow, data to fall out of sync between source and target databases, and database locks to accumulate on the unresponsive session.

Pro2 detects hung threads using a heartbeat mechanism. Each active thread periodically updates a heartbeat timestamp to indicate that it is alive and processing. The Status Check scheduled job reads these timestamps at the interval defined by the HEARTBEAT_CHECK_MINUTES property and flags any thread that has not updated its heartbeat within the HEARTBEAT_CHECK_MINUTES interval as hung.

For more information on Status Check, see Types of scheduled jobs in Pro2.

Important: The HEARTBEAT_CHECK_MINUTES property defaults to 15 minutes. To disable hung thread detection, set this property to 0. For more information, see the Configuration section.

Recovery process

When Pro2 detects a hung thread, it attempts the following recovery steps automatically:
  1. Identify and terminate the process—Pro2 identifies and terminates the operating system process associated with the hung thread. If the process cannot be terminated, Pro2 sends an e-mail notification to administrators requesting manual intervention and stops the recovery sequence.
  2. Release database locks—After the process is terminated, Pro2 releases any database locks held by the unresponsive session and removes the associated control records.
  3. Restart the thread—Pro2 queues a restart pending jobs for the Job Runner. The Job Runner then restarts the thread automatically.
After recovery completes, whether automatic or requiring manual intervention, Pro2 sends an email notification to administrators that includes the details of the event and the actions taken.
Note: Automatic thread restart requires the Job Runner to be active. If the Job Runner is not running when recovery is triggered, the restart job is queued but does not execute until the Job Runner is started. For more information about the Job Runner, see the Use the job runner.

Configuration

Use the HEARTBEAT_CHECK_MINUTES property to enable and configure hung thread detection. Set this property on the Properties tab of the Pro2 web user interface.
Property Description Default
HEARTBEAT_CHECK_MINUTES The maximum number of minutes a thread may remain inactive (no heartbeat) before Pro2 treats it as hung and starts the recovery process. Set this value to 0 to disable hung-thread detection. 15