Powered by Zoomin Software. For more details please contactZoomin

Progress Data Platform Help

Ingest data

  • Last Updated: November 19, 2026
  • 3 minute read
    • Documentation

To work with your content in Progress Data Platform, you first ingest it into a dataset. Each item that you ingest becomes a resource in the dataset. After a resource is ingested, Data Platform processes it using the dataset's enrichment settings so that it can be searched, explored, and and consumed by agents and tools.

Ways to ingest data

You can ingest data in two ways:

Method Use it when Where
Upload files You have files on your computer that you want to add once, or occasionally. From a dataset's Actions panel.
Pipelines Your content lives in an external system, such as cloud storage or a database, or you want to import new content on a schedule. From a dataset's Pipelines tab, using a configured data source.

Both methods read files using the same file types and read options, and both record each run as an execution that you can review.

How ingestion works

  1. You choose a destination dataset. Content is always ingested into a dataset in a project.
  2. You provide your content. Either upload files from your computer, or use a configured data source in a pipeline to import content from an external system.
  3. Data Platform reads the content. Data Platform reads each file or record using the selected file type and read options. Depending on the file type, one file can become one resource or many resources. For example, each row in a delimited text file becomes a separate resource.
  4. Data Platform records an execution. Each upload batch or pipeline run is recorded as an execution with a status and counts of successful, skipped, and failed resources.
  5. Data Platform enriches the resources. New resources are processed using the dataset's enrichment settings. While processing is in progress, a resource's status is Pending. When processing finishes, its status changes to Ready or Failed.

Note: Enrichment settings apply when content is ingested. If you change a dataset's enrichment settings later, reprocess the existing resources to apply the new settings. See Enrichments for more information.

Before you ingest data

  • Create a project to host your dataset. See Create projects.
  • Review the dataset's Enrichments tab and enable the enrichments that you need. For example, enable Chunking and Vector Embeddings to support semantic search and questions about your content. See Enrichments for more information.
  • Confirm the file type of your content and whether it is compressed. See File types and read options for more information.

Ingest your data

The steps to ingest data depend on whether you upload files from your computer or import content from an external system with a pipeline.

To ingest data:

  1. Create a dataset to hold your content, and select the enrichments to apply to it. See Create datasets and Enrichments.
  2. Add content to the dataset by using one of the following methods:
    • Upload files: Upload files from your computer directly to the dataset. Select the file type and read options that match your files. See Upload files and File types and read options.
    • Import with a pipeline: Import content from an external system, once or on a schedule. Create and run a pipeline that imports content from the data source into the dataset. See Import data with pipelines.
  3. Review the execution for each upload batch or pipeline run to confirm that your content was ingested, and to investigate any failed resources. See Review ingestion executions.

After your resources are processed, you can search and inspect them. See Explore your resources.

In this section

Alert