For Docker deployments on Linux, the Hybrid Data Pipeline Docker image uses the non-root user hdpuser with the UID 12321 to start the container. Therefore, before a Hybrid Data Pipeline container can be deployed, a superuser with root privileges must create the shared file location, create the hdpuser with UID 12321, and transfer ownership of the shared file location to the hdpuser. The following sections describe different scenarios for setting up and managing the shared file location on Linux.

Set up the shared file location

Take the following steps to set up the shared file location.

  1. Create the shared file location on Linux.
    mkdir /home/users/username/hdpshare
  2. Create a non-root user with the prescribed UID.
    sudo useradd -u 12321 hdpuser
echo "hdpuser:password" | sudo chpasswd
  3. Transfer the ownership of the folder from the superuser to the hdpuser.
    sudo chown -R 12321:100 /home/users/username/hdpshare
  4. Set permissions for the shared file location.
    sudo chmod -R 750 /home/users/username/hdpshare
Important:
  • To secure keys and other information in the shared file location, use permission 750.
  • Use the -R flag to ensure that the hdpuser has ownership of all files in the shared file location and that the correct permissions are applied to all files.
  • Since the ownership of the folder has changed to the hdpuser, you must be logged in as the hdpuser to add drivers and plugins and otherwise access the resources in the shared file location.

Upgrade from a root user image to a non-root user image

Hybrid Data Pipeline versions 4.6.2.2978 and earlier deployed the container with a root user. The capability to deploy with non-root user is only available with later versions of the server. If you are upgrading to the non-root user version of the image, you must create the hdpuser and transfer ownership of the shared file location to the hdpuser.

Take the following steps to upgrade to the non-root user version of the Docker image.

  1. Stop the docker container.
    docker stop hdp-deployment
  2. Create a non-root user with the prescribed UID.
    sudo useradd -u 12321 hdpuser
echo "hdpuser:password" | sudo chpasswd
  3. Transfer the ownership of the folder from the superuser to the hdpuser.
    sudo chown -R 12321:100 /home/users/username/hdpshare
  4. Start the container with the new image. The -v option should point to the same shared file location or mounted volume that was used for persistent storage with the previous container.
    docker run -dt -p 8443:8443 -p 8090:8090 -p 40501:40501 -p 11280:11280 -p 11443:11443 -e "ACCEPT_EULA=true" -e "HDP_ADMIN_PASSWORD=AdminSecret" -e "HDP_USER_PASSWORD=UserSecret" -v /home/users/username/hdpshare:/hdpshare --hostname DockerHost --name ContainerName hdp-docker-version:tag

Set up the shared location for a trial deployment

For a trial deployment or other scenario where the Hybrid Data Pipeline service does not have to be secured, you may consider two alternative ways to deploy the image on Linux. First, you could relax permissions on the shared file location. Second, you could start the Docker container as a root user.

Relax permissions on the shared file location

You may apply 777 permissions to the shared file location. This allows you to skip the steps involved in provisioning the hdpuser but it ensures that the hdpuser has the permissions necessary to access and write to the shared file location.

  1. Create the shared file location on Linux.
    mkdir /home/users/username/hdpshare
  2. Apply the 777 permissions to the shared file location.
    sudo chmod -R 777 /home/users/username/hdpshare

Start the container as a root user

To run the container as a root user, you may add the -u root option to the docker run command as show in the following example.

sudo docker run -dt -p 8443:8443 -e "HDP_EVAL=true" -e "ACCEPT_EULA=true" -e "HDP_ADMIN_PASSWORD=AdminSecret" -e "HDP_USER_PASSWORD=UserSecret" -u root -v /home/users/username/hdpshare:/hdpshare --hostname DockerHost --name ContainerName hdp-docker-version:tag