Hybrid Data Pipeline supports row limit throttling which allows you to set the maximum number of rows that can be fetched for a single query. Row limit throttling may be configured with the MaxFetchRows limit. The MaxFetchRows limit can be applied at four levels in the following manner.

  • Data source. When applied to a data source, the limit applies to queries made to the data source. A limit applied at the data source level overrides the limit set at the other levels.
  • User. When applied to a user account, the limit applies to queries made by that user. A limit applied at the user level overrides limits set at the tenant and system levels.
  • Tenant. When applied to a tenant, the limit applies to queries made by any user in the tenant. A limit applied at the tenant level overrides a limit set at the system level.
  • System. When applied at the system level, the limit applies to queries made by any user in the Hybrid Data Pipeline system.

To configure row limit throttling, the administrator must have either the Administrator (12) or the Limits (27) permission.

System level configuration

Row limit throttling can be configured at the system level either with the Web UI or with the Limits API. For details on using the Web UI, see Manage Limits view.

The following POST creates a system-level limit of 1000 rows. The number 1 is the ID of the MaxFetchRows limit. The payload passes 1000 as the value for this limit.

POST https://<myserver>:<port>/api/admin/limits/system/1
    {
      "value": 1000
    }

Tenant configuration

Row limit throttling can be configured at the tenant level with either the Web UI or with the Limits API. When using the Web UI, you can enable row limit throttling through either the Manage Tenants view or the Manage Limits view.

The following POST sets a limit of 1500 rows on the specified tenant. The number 32 is the ID of the tenant, and the number 1 is the ID of the MaxFetchRows limit. The payload passes 1500 as the value for this limit.

POST https://<myserver>:<port>/api/admin/limits/tenants/32/1
    {
      "value": 1500
    }

User account configuration

Row limit throttling can be configured at the user level either with the Web UI or with the Limits API. For details on using the Web UI, see Manage Users view.

The following POST sets a limit of 2000 rows on the specified user account. The number 86 is the ID of the user account, and the number 1 is the ID of the MaxFetchRows limit. The payload passes 2000 as the value for this limit.

POST https://<myserver>:<port>/api/admin/limits/users/86/1
    {
      "value": 2000
    }

Data source configuration

Row limit throttling can only be configured at the data sources level using the Limits API. The following POST sets a limit of 2500 rows on the specified data source. The number 86 is the ID of the user account; the number 14 is the ID of the data source that is owned by the user account; and the number 1 is the ID of the MaxFetchRows limit. The payload passes 2500 as the value for this limit.

PUT https://<myserver>:<port>/api/admin/limits/users/86/datasources/14/1
    {
      "value": 2500
    }