After Setting up an AKS environment, you may take the following steps to deploy Hybrid Data Pipeline to the AKS cluster.

Update the values.yaml file

Take the following steps to update to the values.yaml manifest file.

  1. Download the values.yaml file from the following location.

    https://github.com/progress/hdp-kubernetes/blob/main/charts/hybriddatapipeline/values.yaml

  2. Confirm that all the Secrets have been created and are available with the following command.
    kubectl get secrets --namespace namespace-value

    The response should include the following Secrets at minimum.

    Note: If you are using private Docker image registries for Hybrid Data Pipeline and PostgreSQL images, Secrets for accessing these images should be displayed as well.
    NAME                            TYPE
account-database-secrets        hdp-k8s/database-superuser
hdp-secrets                     hdp-k8s/hdp-and-db-users
tls-cert                        hdp-k8s/tls
  3. Update the following parameters in the values.yaml file to configure your Kubernetes deployment of Hybrid Data Pipeline.
    hdp.eula.accepted
    The parameter for accepting the End User License Agreement (EULA). Refer to End User License Agreement (EULA) to review the agreement.
    hdp.replicaCount
    The number of Hybrid Data Pipeline server pods to be deployed. The default is 2.
    hdp.image.repository
    The fully qualified name of your registry login server and the name of Hybrid Data Pipeline Docker image. For example, mycontainerregistry.azurecr.io/hdp-docker-4.6.2. Note that the repository name does not include the four-digit tag. See Pushing the Docker image to a container registry.
    hdp.image.tag
    The four digit tag used to identify the Docker image in the registry repository. This tag should match the four digit build number of the Docker image. For example, in the Docker image hdp_docker_4.6.2:3113.tar.gz, 3113 is the build number and should be used as the tag. See Pushing the Docker image to a container registry.
    hdp.loadbalancer.hostName
    The FQDN for external access to Hybrid Data Pipeline. For example, the hostname of an Azure DNS record set such as hdp-ingress.eastus.cloudapp.azure.com. See Obtain the load balancer hostname.
    hdp.persistence.keystore.storageClassName
    The name of the StorageClass to be used for the keystore persistent volume. For example, azurefile-csi.
    hdp.persistence.logs.storageClassName
    The name of the StorageClass to be used for the logs persistent volume. For example, azurefile-csi.
    hdp.hdpingressconfiguration.agic.enabled
    The parameter for enabling Azure Application Gateway Ingress (AGIC ingress). Set to true to enable.
    Note: The hdp.hdpingressconfiguration.haproxy.enabled parameter must be set to false. HAProxy ingress is not currently supported.
    hdp.hdpingressconfiguration.tls.enabled
    The parameter for enabling TLS communication with the Hybrid Data Pipeline server. Set to true to enable client-side TLS. See Creating a Kubernetes TLS Secret for more information.
    hdp.hdpingressconfiguration.tls.secretName
    The name of the Kubernetes Secret that contains the TLS certificate and private key. For example, tls-cert. See Creating a Kubernetes TLS Secret for more information.
    postgresql.primary.persistence.storageClass
    The name of the StorageClass to be used for the PostgreSQL persistent volume. For example, azurefile-csi.
Note: Not all parameters are listed here. See the following resources for additional information:

Result: You have configured the Helm chart manifest and may now proceed with installation.

Execute the helm install command

Execute the command helm install to deploy Hybrid Data Pipeline on AKS.

helm install hdp-deploy hdp/hybriddatapipeline -f values.yaml

where:

hdp-deploy
is the Helm chart release name for the deployment. You may specify any name that makes sense to you and helps you track the deployment.
hdp/hybriddatapipeline
is the Helm chart to be installed. hdp is the name of the Helm chart repository, and hybriddatapipeline is the name of the Helm chart.
-f values.yaml
is the flag that points to the values.yaml manifest file. The configurations you make to the manifest file override the default values in the Helm chart.

Sign in to Hybrid Data Pipeline

Important: If you are using your own DNS server rather than the FQDN created in Azure, then you must map the ingress IP address to the DNS server before you can call the Hybrid Data Pipeline service. See Mapping the ingress IP address to the DNS server for details.

You may proceed with signing into Hybrid Data Pipeline through a web browser.

  1. Enter the FQDN associated with the Application Gateway ingress controller in your browser address field. For example:

    hdp-ingress.eastus.cloudapp.azure.com

  2. Sign in to either the d2cadmin or the d2cuser accounts.
    Note: The Base64-encoded passwords for these accounts are the same as the Base64-encoded passwords specified for them in the hdp-secrets.yaml file. See Creating Kubernetes Secrets for required credentials for details.

Next steps

After deploying Hybrid Data Pipeline, you may need to access the shared file location to deploy components, add or update drivers, add plug-ins, and obtain logs. In addition, you may begin creating data sources and enabling OData data sources. See the following topics for details.