Hybrid Data Pipeline supports single-node and cluster deployment scenarios. Each deployment scenario includes the same core features: a host or hosts that run the service, a system database to store system and account information, and a key location to store supporting files. The deployment scenario you follow determines, in part, how you configure these core features and the steps you must take to deploy Hybrid Data Pipeline.

The following sections describe general deployment scenarios and workflows. Note that the workflows do not include all deployment configurations. For a complete list of deployment configurations, see Deployment configurations.

  • Single-node deployments
    • Single-node quick deployment. In the quick deployment scenario, a single node receives queries directly. The quick deployment is a simple way to deploy Hybrid Data Pipeline with minimal overhead. The quick deployment uses default settings to get the service up quickly for testing and evaluation purposes.
    • Single-node deployment without load balancer. In the non-load balancer scenario, like the quick deployment scenario, a single node receives queries directly. However, this deployment option has greater flexibility than the quick deployment. It allows you to configure the service in a number of ways, rather than using default settings.
    • Single-node deployment with load balancer. In this scenario, a single node is deployed behind a load balancer that manages queries. This deployment scenario lays the groundwork for increasing availability and scalability.
  • Cluster deployment. In a cluster deployment, multiple nodes are deployed behind a load balancer that manages queries. Multiple nodes allow increased availability and scalability.
Important: Currently, there is no migration path between a deployment that does not use a load balancer and one that does. A deployment that uses a load balancer offers availability and scalability that is not otherwise available.

Single-node quick deployment

The single-node quick deployment is the simplest way to deploy Hybrid Data Pipeline. This deployment provides a running instance of the service on a single host machine. It includes an internal system database and an internal key location, so you do not have to deploy these separately. In addition, a self-signed certificate is used to encrypt communication between the server and client applications.

The single-node quick deployment has a number of limitations. First, it cannot be configured to connect with on-premises data sources. Second, there is no migration path for moving the quick deployment to a cluster deployment for greater availability and scalability. Lastly, the use of a self-signed certificate limits the quick deployment to an evaluation or test environment for security reasons.

Note: Alternatively, Hybrid Data Pipeline may be run as a Docker container on a trial basis. Unlike the single-node quick deployment, the trial deployment allows you to configure the server to connect with on-premises data, and allows you to use a certificate issued by a certificate authority. For details, refer to the Trial Docker Deployment Getting Started.

The following steps describe a single-node quick deployment workflow.

  1. Open the default access ports on the Linux host.
  2. Run the installation program. See Installing and upgrading the Hybrid Data Pipeline server for details.
  3. Select the Typical installation option when prompted.
  4. Install the ODBC and JDBC drivers for ODBC and JDBC connectivity.
    • For ODBC connectivity, refer to Install the ODBC Driver. An ODBC client application must use the ODBC driver to connect with Hybrid Data Pipeline.
    • For JDBC connectivity, refer to Install the JDBC Driver. A JDBC client application must use the JDBC driver to connect with Hybrid Data Pipeline.
    Note: OData connectivity does not require the installation of additional components.
Note: A single-node quick deployment can be upgraded to a single-node deployment without load balancer.

What's next

Single-node deployment without load balancer

In a single-node deployment without a load balancer, the server receives queries directly from client applications. This deployment option gives you the ability to deploy an external database and an external key location. In contrast to the quick deployment, you can specify an SSL certificate to secure communication between the server and client applications. However, it is important to note that you cannot migrate a deployment that does not use a load balancer to one that does. Therefore, a single-node deployment without load balancer may limit your ability to scale at a future time.

The following steps describe the workflow for a single-node deployment without a load balancer.

  1. If you are using an external system database, you must deploy it before installing the server. See System database for details.
    Note: For a Docker deployment, an internal system database is only available for a trial deployment.
  2. If you are using an external key location, you must create it before deploying the server. See Shared files and the key location for details.
    Note: If you are deploying the service as a Docker container, the external key location and persistent volume are the same. For a Docker deployment, the default key location is /hdpshare.
  3. By default, an SSL self-signed certificate is provided during deployment. However, you may provide an SSL certificate during the deployment process. See SSL configuration (non-load balancer) for details.
  4. Open the default access ports on the host machine.
  5. Deploy the service on the host machine. The service can be installed on a Linux host, or deployed as a Docker container. See Installing and upgrading the Hybrid Data Pipeline server for details.
  6. Install supporting components such as the On-Premises Connector, the ODBC driver, and the JDBC driver.
    Note: OData connectivity does not require the installation of additional components.

What's next

Single-node deployment with load balancer

In a single-node deployment with a load balancer, a single node is deployed behind a load balancer that manages requests between the server and client applications. A single-node deployment with a load balancer allows you to scale up the service by adding additional nodes behind the load balancer. In addition, SSL can be configured to secure communications between the load balancer and the server (server-side SSL), as well as between the load balancer and client applications (client-side SSL).

A single-node deployment with a load balancer requires the deployment of a load balancer, an external database, and an external key location.

The following steps describe the workflow for a single-node deployment with a load balancer.

  1. Deploy a load balancer to support the service. Network and cloud load balancers are supported. See Load balancer configuration for details.
  2. Deploy an external system database. See System database for details.
  3. Set up an external key location. See Shared files and the key location for details.
  4. Prepare SSL certificates, and configure the environment for SSL. See SSL configuration (load balancer) for details.
  5. Open the default access ports on the host machine.
  6. Deploy the service on a host machine. The service can be installed on a Linux host, or deployed as a Docker container. See Installing and upgrading the Hybrid Data Pipeline server for details.
  7. Install supporting components such as the On-Premises Connector, the ODBC driver, and the JDBC driver.
    Note: OData connectivity does not require the installation of additional components.

What's next

Cluster deployment

In a cluster deployment, multiple Hybrid Data Pipeline nodes are deployed behind a load balancer that manages requests across nodes. A cluster deployment offers high availability and scalability. In addition to client-side SSL, server-side SSL is supported. Server-side SSL enables you to secure communication between the load balancer and nodes, as well as between the individual nodes.

A cluster deployment requires the deployment of a load balancer, an external database, and an external key location. All cluster deployments begin as single-node deployments that allow for the addition of nodes to scale up and to ensure availability.

The following workflow describes how to deploy a Hybrid Data Pipeline cluster.

  1. Deploy a load balancer to support the service. Network and cloud load balancers are supported. See Load balancer configuration for details.
  2. Deploy an external system database. See System database for details.
  3. Set up an external key location. See Shared files and the key location for details.
  4. Prepare SSL certificates, and configure the environment for SSL. See SSL configuration (load balancer) for details.
  5. Open the default access ports on each host machine.
  6. Deploy the service on a host machine. The service can be installed on a Linux host, or deployed as a Docker container. See Installing and upgrading the Hybrid Data Pipeline server for details.
  7. Deploy the service on any additional nodes.
  8. Install supporting components such as the On-Premises Connector, the ODBC driver, and the JDBC driver.
    Note: OData connectivity does not require the installation of additional components.

What's next