System logs are generated for each node running Hybrid Data Pipeline. System logs are also generated for the On-Premises Connector, but these are limited to the operations of the On-Premises Connector. On-Premises Connector system logs must be retrieved manually from the On-Premises Connector installation directory:

opc_install_dir\OPDAS\server\logs\

System logs for the Hybrid Data Pipeline server are written to the installation directory of each node running the service. As described in the following sections, system logs for the Hybrid Data Pipeline server may be retrieved using the Web UI, the Nodes API, or the getlogs.sh script.

Note: The user must have the Administrator (12) permission to retrieve system logs using the Web UI or the Nodes API.

Obtain system logs with the Web UI

Take the following steps to retrieve system logs using the Web UI.

  1. Navigate to the System Configurations view by clicking the configurations icon . Then, select the Diagnostics tab.
  2. From the Select Node dropdown, select the node or nodes from which you are retrieving system logs.
  3. Optional. Select a date range.
  4. Click Download Log.

Result: A tar.gz file for each node you selected will be downloaded to your browser's download directory.

Obtain system logs with the Nodes API

You may use the Nodes API to retrieve system logs as shown in the following steps.

  1. Retrieve the IDs for the nodes in your deployment by making the following request.
    https://hdp.example.com/api/admin/system/nodes

    Response

    {
  "nodes": [
    {
      "url": "https://node1.example.com",
      "id": 1
    },
    {
      "url": "https://node2.example.com",
      "id": 2
    }
    {
      "url": "https://node3.example.com",
      "id": 3
    }
  ]
}
  2. Retrieve the system logs for the node by making the following request, where {id} is the ID of the node for which you are retrieving logs.
    https://hdp.example.com/api/admin/system/nodes/{id}/logs

    Response

    A tar.gz file, containing the system logs of the specified node, is downloaded to your machine. The name of the file will have the format hdp_logs.datetimestamp.tar.gz.

    Note: If allowed by your browser, you may enter the URL in the address field. You will be prompted for credentials. After you enter your credentials, the logs will be downloaded. If you are using an API tool, you may need to select a download option to ensure that the logs are downloaded. For example, if you are using the Postman GUI client, you must select Send and Download from the Send dropdown.
  3. In a cluster deployment, repeat Steps 1 and 2 for each node as needed.

Use the getlogs.sh script

Take the following steps to bundle the system logs associated with a node using the getlogs.sh script.

  1. From the host machine or Docker shell, navigate to install_dir/ddcloud.
  2. Run the getlogs.sh script. Optionally, you may include start_date and end_date arguments to specify a date range. The dates must be in UTC yyyy-mm-dd format. For example:
    getlogs.sh 2024-02-27 2024-03-04

    Result

    The script will generate a tar.gz file. The name of the file will have the format hdp_logs.datetimestamp.tar.gz.

  3. In a cluster deployment, repeat Steps 1 and 2 for each node as needed.