Before you begin:
Note: The installer does not support changing a non-load balancer deployment to a load balancer deployment during an upgrade, or vice versa. To make such a change, you would need to uninstall the server and then reinstall it. See Uninstalling Hybrid Data Pipeline server and Installing the Hybrid Data Pipeline server for details.

After copying the product installation file to a temporary directory, take the following steps to upgrade a non-load balancer deployment with the installer in GUI mode.

  1. From a command-line prompt, navigate to the directory where you saved the product upgrade file. Alternatively, place the product upgrade file directory on your path before proceeding to the next step.

    The product file has the format PROGRESS_DATADIRECT_HDP_SERVER_nn_LINUX_64_INSTALL.bin, where nn is the version of the product.

  2. Make the file an executable using the chmod command. Then, press ENTER. For example:

    chmod +x ./PROGRESS_DATADIRECT_HDP_SERVER_nn_LINUX_64_INSTALL.bin

  3. Run the executable by entering the product file path and pressing ENTER.
    1. Type the file name and path of the product file. For example:

      ./PROGRESS_DATADIRECT_HDP_SERVER_nn_LINUX_64_INSTALL.bin

    2. Press ENTER.
    3. The Introduction window appears. Click Next to continue.
    Note: If the installer cannot continue with a GUI installation, a message is displayed, and the installation continues in console mode.
  4. Enter or choose the directory of the installation you want to upgrade; then, click Next.
  5. Choose the JVM you want to use for the runtime operation of the server.
    Note: By default, the JVM used to run the installer is selected. However, you may choose a different JVM by clicking Choose Java Executable and navigating to the Java executable of your preferred JVM. See Product requirements for details on JVM prerequisites.
  6. Click Upgrade in the pop-up dialog to continue.
  7. Choose whether you want to upgrade with an evaluation or licensed version of the product. Licensed installations require a valid License Key.
    • Evaluation. Select this option to upgrade with an evaluation version that is fully functional for 30 days. Then, click Next.
    • Licensed. Select this option if you purchased a licensed version of the product. Type the license key, including any dashes, and then click Next.
  8. Accept or enter the fully qualified hostname for the Hybrid Data Pipeline server. By default, the installer suggests the name of the current machine. Then, click Next.
    Note:
    • For non-load balancer deployments where the server is receiving queries directly, the hostname should be tied to the DNS for public access. This allows you to update IP addresses in the DNS without having to run an upgrade installation to reconfigure your deployment.
    • If the installer is unable to validate the hostname, you are prompted to reenter the hostname or skip validation. Skipping validation is often desired when performing a silent installation or upgrade.
  9. Select how you want to continue the upgrade.
    • Select Express to persist previously established values. Then, click Next and continue at Step 24.
    • Select Modify settings to change values for the following configurations. Then, click Next.
      • Change the location of the SSL certificate file
      • Enable or disable FIPS
      • Add MySQL Community Edition as a data store or external system database
      • Change the system database you are using
      • Update external system database credentials
      • Enable or disable the On-Premises Connector
      • Change Server Access Ports
      • Change On-Premises Connector Ports
      • Change Server Internal Access Ports
    Note: The key location where the generated key and internal files are stored cannot be modified when performing an upgrade.
  10. Select whether you want to enable FIPS on the Hybrid Data Pipeline server.
    Important: To enable FIPS, your hardware must support secure random, or you must have a secure random daemon installed.
    Important: When you enable FIPS during an upgrade, the environment uses the encryption key for the system database that was generated during the original installation of the server. This encryption key, while generated with a FIPS compliant algorithm, was not generated with a FIPS certified implementation of the algorithm.
  11. Depending on your environment, provide the appropriate SSL certificate information.
    • Select Certificate file to specify a PEM file to be used by the server to establish SSL connections with ODBC and JDBC client applications. Type the full path to the PEM file, or click Choose... to browse to the location of the PEM file. Then, click Next.
      Note: The PEM file must consist of a private key, a public key certificate issued by a certificate authority (CA), and additional certificates that make up the trust chain. See The PEM file for more information.
    • Select Use existing Certificate to use the SSL certificate specified in the previous installation or the self-signed certificate included with the installation. Then, click Next.
      Note: The self-signed certificate may be used in a test environment. However, for production, a PEM file with required information should be specified. See The PEM file for more information.
  12. Select whether you want to use MySQL Community Edition as a data source or external database. Hybrid Data Pipeline does not provide a driver for MySQL Community Edition; however, it does support the MySQL Connector/J driver. If you choose Yes, in the Jar Path field, provide the name and location of the driver's jar file. Click Next to continue.
    Note: For more information on the MySQL Connector/J driver, refer to the MySQL developer website at https://dev.mysql.com/.
  13. Select Kerberos for SQL Server if you plan to authenticate using Kerberos. Click Choose... to type a path or browse to the location of the configuration file krb5.conf.
  14. Select the type of database you want to use to store system information.
    • Select Internal Database (supplied by this install) to use the default internal database. Click Next and continue at Step 22.
    • Select External Database to store the system information in an external database. Then, from the drop down box, choose your database vender. Then, click Next.
      • Select Oracle, and continue at Step 15.
      • Select MySQLCommunity, and continue at Step 16.
      • Select MSSQLServer, and continue at Step 17.
      • Select PostgreSQL, and continue at Step 18.
    Note: Users and data sources created in the internal database are specific to the internal database. They are not migrated to the external database if you subsequently modify the Hybrid Data Pipeline server to use an external database.
  15. Provide the Oracle connection information.
    1. Type the name of the host.
    2. Type the port number.
    3. Select the connection type. Do one of the following:
      • If you connect using the Oracle System Identifier (SID), select Connect using SID, then type the SID.
      • Select Connect using Service Name, then type the database service name that specifies the database that is used for the connection. The service name is a string that is the global database name. The global database name typically comprises the database name and domain name.
    4. Optionally, in the Advanced Options field, specify additional connection parameters and their values to be included the connection url. Values should be entered as a semicolon-separated list of parameter=value. For example, you may enter to following options to configure SSL: 
      encryptionLevel=Required;encryptionTypes=(AES256);
dataIntegrityLevel=Required;dataIntegrityTypes=(SHA1);
encryptionMethod=SSL;keyStore=/common/Oracle/trustStore.jks;
keyStorePassword=secret;serverType=dedicated;authenticationMethod=ntlm;
hostNameInCertificate=oracle;editionName=hybrid
    5. Click Next, and continue at Step 19.
  16. Provide connection information for the MySQL Community Edition database.
    1. Type the name of the host.
    2. Type the port number.
    3. Type the database name.
    4. Optionally, in the Advanced Options field, specify additional connection parameters and their values to be included the connection url. Values should be entered as a ampersand-separated list of parameter=value.
    5. Click Next, and continue at Step 19.
  17. Provide the SQL Server connection information.
    1. Type the name of the host.
    2. Type the port number.
    3. Type the database name.
    4. Type the name of the schema.
    5. Optionally, in the Advanced Options field, specify additional connection parameters and their values to be included in the connection URL. Values should be entered as a semicolon-separated list of parameter=value.
    6. Optionally, in the Advanced Options field, specify the path to the krb5.conf file to use Kerberos authentication method.
    7. Click Next, and continue at Step 19.
  18. Provide the PostgreSQL connection information.
    1. Type the name of the host.
    2. Type the port number.
    3. Type the database name.
    4. Optionally, in the Advanced Options field, specify additional connection parameters and their values to be included in the connection URL. Values should be entered as a semicolon-separated list of parameter=value.
    5. Click Next, and continue at Step 19.
  19. Provide the external database credential information.
    Note:
    • Administrator credentials are required at install time to add and modify required tables in the database schema. Administrator credentials are not used by the product at run time.
    • If the installer is unable to validate, you are prompted to reenter credentials or skip validation. Skipping validation is often desired when performing a silent installation. See Silent installation process for details.
    • In the Admin Username field, type the administrator user name. The administrator user must have certain privileges that are specific to the database vendor. For a list of required privileges, see External system databases.
    • In the Admin Password field, type the password for an external database administrator account.
    • In the Username field, type a user name. The standard user must have certain privileges that are specific to the database vendor. For a list of required privileges, see External system databases.
    • In the Password field, type the user password.
  20. Review the Server Access Ports. A Server Access Port must be available to the end user across the firewall. Best security practices recommend using an HTTPS port.
    Table 1. Server Access Ports
    Name Default Description
    Server Access HTTP Port 8080 HTTP port used for communication to the Hybrid Data Pipeline Web UI and API.
    Server Access HTTPS Port 8443 HTTPS port used for encrypted communication to the Hybrid Data Pipeline Web UI and API.
  21. Select whether you will continue to use the On-Premises Connector.
    • If using the On-Premises Connector, select Enable On-Premises Connector. Click Next and continue to the next step.
    • If not using the On-Premises Connector, leave the check box empty and click Next. Continue at Step 23.
  22. Review the On-Premises Access Ports. The On-Premises Access Port and a Notification Server Port must be available across the firewall. Best security practices recommend using the SSL Notification Server Port.
    Important: If you change any values for On-Premises Access Ports during an upgrade, you will need to reinstall the On-Premises Connector with the updated distribution files in the redist subdirectory. See What to do next.
    Table 2. On-Premises Access Ports
    Name Default Description
    On-Premises Port 40501 Port for communication between the On-Premises Connector and the Hybrid Data Pipeline server.
    Notification Server TCP Port 11280 TCP port for communication from the On-Premises Connector to the Notification Server.
    Notification Server TCP SSL Port 11443 TCP SSL port for encrypted communication from the On-Premises Connector to the Notification Server.
    Message Queue Port 8282 Port for communication between the Notification Server and the Hybrid Data Pipeline server. Required for Linux installation of Hybrid Data Pipeline. Not used in Docker deployments.
  23. Review the Server Internal Ports. A port for the internal API and the Shutdown Port must be opened. Best security practices recommend using the Internal API HTTPS Port.
    Important: As a matter of best practice, the Shutdown Port should not be available outside the firewall of the Hybrid Data Pipeline instance.
    Table 3. Server Internal Ports
    Name Default Description
    Internal API HTTP Port 8190 HTTP port for communication between individual nodes in a cluster deployment.
    Internal API HTTPS Port 8090 HTTPS port for encrypted communication between individual nodes in a cluster deployment.
    Shutdown Port 8005 TCP port that listens for shutdown requests.
  24. Review the summary. If you are satisfied with your choices, press ENTER to upgrade.
  25. After the upgrade has finished, press ENTER to exit the installer.
  26. Verify the upgrade by accessing the Hybrid Data Pipeline user interface from a browser. The login page should appear. For example, type the following.

    https://<myserver>:8443/

    where <myserver> is the fully qualified hostname of the machine where you installed Hybrid Data Pipeline.

    After logging in, administrators can verify product version information by selecting About from the question mark drop-down menu.

Refer to installation log files for a record of any problems that may have occurred during the upgrade. See Deployment logs for details.

What to do next

If you change any of the On-Premises Connector Ports, you will need to reinstall the On-Premises Connector with updated configuration and certificate files. These files are located in the Hybrid Data Pipeline installation directory <install_dir>/redist. These files must be copied to the directory of the On-Premises Connector installer before proceeding with the reinstallation of the On-Premises Connector.

The four configuration and certificate files are:

  • config.properties
  • OnPremise.properties
  • ddcloud.pem
  • ddcloudTrustStore.jks

See Installing the Hybrid Data Pipeline On-Premises Connector for further details.