Data source options
- Last Updated: November 19, 2026
- 8 minute read
- Documentation
When you create or edit a data source, you configure options that tell Progress Data Platform how to connect to the external system and how to import its content. This topic describes the options for each type of data source, their default settings, and when to change them. For the procedures to create and edit a data source, see Manage data sources.
All data sources also have the following options:
- Title: A name that identifies the data source.
- (Optional) Description: A description of the data source.
Required options are marked with an asterisk in Data Platform. Secret values, such as passwords, keys, and tokens, are hidden after you save the data source.
This topic covers the following:
- Amazon S3
- Azure Storage
- Google Cloud Storage
- External Database
- MarkLogic
- Import options for file-based data sources
Amazon S3
Use an Amazon S3 data source to import files from an Amazon S3 bucket. To authenticate, provide an Access Key ID and Secret Access Key, or select Anonymous for a public bucket.
Amazon S3 data sources also have file type, read, and import options. See Import options for file-based data sources.
The following table describes the connection options for Amazon S3 data sources:
| Option | Description | Default |
|---|---|---|
| Bucket | Required. Specifies the name of the S3 bucket that contains your files. | None |
| Prefix | Specifies the folder path within the bucket to import from. Set a prefix to import only the files in a specific folder rather than the entire bucket. For example, reports/2026. |
None (imports entire bucket) |
| Region | The AWS region of the bucket. Set the region when the bucket is not in the default region for your AWS account. For example, us-east-1. |
None |
| Access Key ID | Specifies the AWS access key ID used to authenticate. This value is required to access a private bucket, together with Secret Access Key. | None |
| Secret Access Key | Specifies the AWS secret access key that pairs with the Access Key ID. | None |
| Session Token | Specifies the AWS session token for temporary security credentials. Provide a token only when your access key is temporary, such as a key issued by AWS Security Token Service. | None |
| Endpoint URL | Specifies a custom S3 endpoint. Set an endpoint only when you connect to an S3-compatible storage service other than Amazon S3. | None (Amazon S3) |
| Anonymous | Specifies whether to connect without credentials. Select this option to read from a public bucket. Valid values:
|
Cleared |
Azure Storage
Use an Azure Storage data source to import files from an Azure Blob Storage or Azure Data Lake Storage Gen2 container. To authenticate, provide one of the following: Access Key, Sas Token, or Shared Key.
Azure Storage data sources also have file type, read, and import options. See Import options for file-based data sources.
The following table describes the connection options for Azure Storage data sources:
| Option | Description | Default |
|---|---|---|
| Storage Account | Required. Specifies the name of the Azure storage account. | None |
| Storage Type | Specifies the type of Azure storage that holds your files. Select the type that matches your storage account:
|
DATA_LAKE |
| Container | Required. Specifies the name of the container that holds your files. | None |
| Access Key | Specifies the access key for the storage account. Use an access key to grant access to the entire storage account. | None |
| Sas Token | Specifies a shared access signature (SAS) token that is used to grant limited, time-bound access instead of full account access. | None |
| Shared Key | Specifies the shared key for the storage account. | None |
| Prefix | Specifies the folder path within the container to import from. Set a prefix to import only the files in a specific folder rather than the entire container. For example, reports/2026. |
None (imports entire container) |
| Endpoint URL | Specifies a custom storage service endpoint. Set an endpoint only when you connect to on-premises or emulated Azure storage. | None (Azure) |
Google Cloud Storage
Use a Google Cloud Storage data source to import files from a Google Cloud Storage bucket. To authenticate, provide a Service Account Key JSON, or select Anonymous for a public bucket.
Google Cloud Storage data sources also have file type, read, and import options. See Import options for file-based data sources.
The following table describes the connection options for Google Cloud Storage data sources:
| Option | Description | Default |
|---|---|---|
| Bucket | Required. Specifies the name of the Google Cloud Storage bucket that contains your files. | None |
| Prefix | Specifies the folder path within the bucket to import from. Set a prefix to import only the files in a specific folder rather than the entire bucket. For example, reports/2026. |
None (imports entire bucket) |
| Project | Specifies the ID of the Google Cloud project that owns the bucket. Set the project ID when your credentials have access to more than one project. | None |
| Service Account Key JSON | Specifies the key for a Google Cloud service account that can read the bucket. When setting this option, provide the entire contents of the service account key file, including the opening and closing braces. | None |
| Endpoint URL | A custom Google Cloud Storage endpoint. Set an endpoint only when you connect to a Google Cloud Storage emulator. | None (Google Cloud Storage) |
| Anonymous | Specifies whether to connect without credentials. Select this option to read from a public bucket. Valid values:
|
Cleared |
External Database
Use an External Database data source to import records from a table in a relational database through a JDBC connection. Each row in the table becomes a separate resource.
The following table describes the options for External Database data sources:
| Option | Description | Default |
|---|---|---|
| Driver | Specifies the JDBC driver for your database. When you select a driver, Data Platform fills in a URL template for that driver. Valid values: The drivers that are available to your account. |
The first available driver |
| URL | Required. Specifies the JDBC connection URL for the database. Replace the host, port, and database name in the template with the values for your database. For example, jdbc:datadirect:postgresql://dbhost:5432;DatabaseName=sales. |
A template for the selected driver |
| User | Required. Specifies the database user name. | None |
| Password | Required. Specifies the password for the database user. | None |
| Keytab | Specifies the path to a Kerberos keytab file. Set this option only when your database uses Kerberos authentication. | None |
| Principal | Specifies the Kerberos principal to authenticate as. Set this option only when your database uses Kerberos authentication. | None |
| Refresh Krb5 Config | Specifies whether to reload the Kerberos configuration before connecting. Select this option if the Kerberos configuration changed since the last connection. Valid values:
|
Cleared |
| Table | Required. Specifies the table to import. Enter a table name, or enter a SQL query in parentheses with an alias to import a filtered or joined set of rows. For example, employees or (SELECT * FROM employees WHERE active = true) AS t. |
None |
| Limit | Specifies the maximum number of rows to import. This can be used to limit the size of an import when testing a new data source before importing the entire table. | None (imports all rows) |
| URI Template | Specifies the template that builds each resource's identifier from column values. For example, /{employee_id}.json. You can use this to employ a template that uses a unique column so that rerunning an import updates existing resources instead of creating duplicates. |
None (Data Platform proposes a template based on the table's columns) |
| JSON Root Name | Specifies the name of a root property that wraps the fields of each imported row. For example, employee. Set a root name to give resources from different tables a distinct structure. |
The table name, when Table is a table name |
MarkLogic
Use a MarkLogic data source to copy documents from a MarkLogic database. The authentication options that are exposed depend on the Authentication Type that you select.
The following table describes the options for MarkLogic data sources:
| Option | Description | Default |
|---|---|---|
| Host | Required. Specifies the host name of the MarkLogic server. | None |
| Port | Required. Specifies the port of the MarkLogic app server to connect to. For example, 8000. |
None |
| Authentication Type | Specifies the authentication method that the MarkLogic app server uses. Select the method that matches your app server configuration:
|
DIGEST |
| Username | Specifies the MarkLogic user name. This option is exposed and required when Authentication Type is DIGEST or BASIC. | None |
| Password | Specifies the password for the MarkLogic user. This option is exposed and required when Authentication Type is DIGEST or BASIC. | None |
| PDC API Key | Specifies the Progress Data Cloud API key. This option is exposed and required when Authentication Type is CLOUD. | None |
| Kerberos Principal | Specifies the Kerberos principal to authenticate as. This option is exposed and required when Authentication Type is KERBEROS. | None |
| OAuth Token | Specifies the OAuth access token. This option is exposed and required when Authentication Type is OAUTH. | None |
| SAML Token | Specifies the SAML token. This option is exposed and required when Authentication Type is SAML. | None |
| Base Path | Specifies the path prefix for requests to the MarkLogic server. This option is exposed when Authentication Type is set to CLOUD. Required to connect to a Progress Data Cloud endpoint. | None |
| Database | Specifies the MarkLogic database to copy documents from. Set a database to read from a database other than the app server's default database. | None (the app server's database) |
| Connection Type | Specifies how Data Platform connects to the hosts in the MarkLogic cluster. Choose from:
|
None (GATEWAY behavior) |
| Disable Gzipped Responses | Specifies whether MarkLogic compresses its responses. Select this option to improve performance when responses are very small. Choose from:
|
Cleared |
| Collection | Specifies the MarkLogic collection to copy. Configure this option to copy only a subset of documents. | None (copies all documents) |
Import options for file-based data sources
Amazon S3, Azure Storage, and Google Cloud Storage data sources share options that control how files are read and imported.
File type and read options
The File Type option specifies how files are split into resources, and the read options that are exposed depend on the selected file type. The default file type is Generic, which imports each file as one resource. For descriptions of the file types and their read options, see File types and read options.
Import options
The following table describes the import options for file-based data sources:
| Option | Description | Default |
|---|---|---|
| Limit | The maximum number of records to import. Set a limit to test a new data source with a small sample before you import all of your files. | None (all records) |
| URI Template | A template that builds each resource's identifier from field values. For example, /{order_id}.json. Set a template that uses a unique field so that rerunning an import updates existing resources instead of creating duplicates. This option is exposed when File Type is set to Delimited Text, Avro, Parquet, or ORC. |
None (identifiers are based on the file path) |
| JSON Root Name | The name of a root property that wraps the fields of each imported record. For example, order. Set a root name to give records from different sources a distinct structure. This option is exposed when File Type is set to Delimited Text, Avro, Parquet, or ORC. |
None |