The method for configuring logs differs, depending on the type of Hybrid Data Pipeline deployment. See the following sections for details.

Note: The location of data source logs is determined by the hdpLoggingDir property in the update.properties file. However, data source logging levels are configured separately. For information on configuring data source logging levels, refer to Data source logging in the Hybrid Data Pipeline User's Guide.

Configuring logging for Docker deployments

Take the following steps before executing the docker run command.

Note: For complete instructions on deploying Hybrid Data Pipeline as a Docker container, see Deploying Hybrid Data Pipeline using Docker.
  1. Obtain the hdplogging.properties file. This file is included in the Docker image product package.
    Note: The hdplogging.properties file is located in the deployment package directory package_dir/hdp-docker-deploy/hdp, where package_dir is the directory into which you unzipped the contents of the deployment package.
  2. Optional. Specify the log location with the hdpLoggingDir property. For example: 
    hdpLoggingDir=USER_HOME/common/shared/hdp/logs
    Note: If you do not specify a location, the logs are stored in the default location local to the node, which is hdp_install_dir/ddcloud/logs. In the hdplogging.properties file, the default location is specified as hdpLoggingDir=ddcloud-home/logs where ddcloud-home indicates the ddcloud directory in the Hybrid Data Pipeline installation directory.
  3. Optional. Configure the logging levels for the Web UI, the data access service, the notification server, and the Apache Tomcat server. See Logging levels for details.
  4. Copy the hdplogging.properties file to the shared file location on the local file system.
    Note: The hdplogging.properties file contains comments that provide detailed guidance on how to update the properties.
  5. Execute the docker run command.
  6. Review the log location to confirm that the directories have been created and log files are being written to this location.

Configuring logging for Linux installations

To configure the log location and set logging levels using the hdplogging.properties file, take the following steps.

  1. Navigate to the keystore directory specified during installation.
    Note: The hdplogging.properties file is located in the keystore directory for Linux installations. For load balancer installations, the keystore directory is shared by all the nodes. For non-load balancer installations, the file path is install_dir/ddcloud/keystore.
  2. Specify the log location with the hdpLoggingDir property in the hdplogging.properties file. For example:
    hdpLoggingDir =/USER_HOME/common/shared/logs
    Note: If you do not specify a location, the logs are stored in the default location local to the node, which is hdp_install_dir/ddcloud/logs. In the hdplogging.properties file, the default location is specified as hdpLoggingDir=ddcloud-home/logs where ddcloud-home indicates the ddcloud directory in the Hybrid Data Pipeline installation directory.
  3. Optional. Configure the logging levels for the Web UI, the data access service, the notification server, and the Apache Tomcat server. See Logging levels for details.
  4. Restart the Hybrid Data Pipeline service. For cluster deployments, restart the service on each node in the cluster.
    1. Run the stop service script.
      ./install_dir/ddcloud/stop.sh
      Note: Shutting down Hybrid Data Pipeline can take a few minutes. Wait until you see the Shutdown complete message displayed on the console before taking any additional actions.
    2. Run the start service script.
      ./install_dir/ddcloud/start.sh
  5. Review the log location to confirm that the directories have been created and log files are being written to this location.