DataDirect Bulk Load

The driver supports DataDirect Bulk Load, a feature that allows your application to send large numbers of rows of data to a database.

The driver sends the data to the database in a continuous stream instead of numerous smaller database packets. Similar to batch operations, using bulk load improves performance because far fewer network round trips are required. Bulk load bypasses the data parsing usually done by the database, providing an additional performance gain over batch operations.

DataDirect Bulk Load requires a licensed installation of the driver. If the driver is installed with an evaluation license, the bulk load feature is available for prototyping with your application, but with limited scope. Contact your sales representative or Progress DataDirect SupportLink for further information.

Because a bulk load operation may bypass data integrity checks, your application must ensure that the data it is transferring does not violate integrity constraints in the database. For example, suppose you are bulk loading data into a database table and some of that data duplicates data stored as a primary key, which must be unique. The driver will not throw an exception to alert you to the error; your application must provide its own data integrity checks.

Bulk load operations are accomplished by exporting the results of a query from a database into a comma-separated value (CSV) file, a bulk load data file. The driver then loads the data from bulk load data file into a different database. The file can be used by any DataDirect Connect Series for ODBC drivers. In addition, the bulk load data file is supported by other DataDirect Connect product lines that feature bulk loading, for example, a DataDirect Connect for ADO.NET data provider that supports bulk load.

Suppose that you had customer data on a SAP ASE server and need to export it to an Oracle server. The driver would perform the following steps:

  1. Application using SAP ASE Wire Protocol driver sends query to and receives results from SAP ASE server.
  2. Driver exports results to bulk load data file.
  3. Driver retrieves results from bulk load data file.
  4. Driver bulk loads results on Oracle server.

For SAP ASE, some additional database configuration is required when the destination table for a bulk load operation does not have an index defined. If you are using a destination table that does not have an index defined, you can ask the database operator to execute the following commands:

use master
sp_dboption test, "select into/bulkcopy/pllsort", true 

This option is required to perform operations that do not keep a complete record of the transaction in the log. For more information, refer to the SAP ASE documentation.

Alternatively, you can define an index on the destination table.

Failure to properly configure the database results in errors such as the following:

"You cannot run the non-logged version of bulk copy in this database. Please check with the DBO."