Introduction to MarkLogic Content Pump

MarkLogic Content Pump (mlcp) is a command line tool for getting data into and out of a MarkLogic Server database. Using mlcp, you can import documents and metadata to a database, export documents and metadata from a database, or copy documents and metadata from one database to another. For example:

• Import content into a MarkLogic Server database from flat files, compressed ZIP and GZIP files, or mlcp database archives.

• Create documents from flat files, delimited text files, aggregate XML files, and line-delimited JSON files. For details, see Importing Content into MarkLogic Server.

• Import mixed content types from a directory, using the file suffix and MIME type mappings to determine document type. Unrecognized/missing suffixes are imported as binary documents. For details, see How mlcp Determines Document Type.

• Export the contents of a MarkLogic Server database to flat files, a compressed ZIP file, or an mlcp database archive. For details, see Exporting Content from MarkLogic Server.

• Copy content and metadata from one MarkLogic Server database to another. For details, see Copying Content between Databases.

• Import or copy content into a MarkLogic Server database, applying a custom server-side transformation before inserting each document. For details, see Transforming Content During Ingestion.

• Extract documents from an archived forest to flat files or a compressed file using Direct Access. For details, see Using Direct Access to Extract or Copy Documents.

• Import documents from an archived forest into a live database using Direct Access. For details, see Importing Documents from a Forest into a Database.

The mlcp tool operates in local mode meaning that mlcp drives all its work on the host where it is invoked. Resources such as import and input data and export destination must be reachable from that host. All communication with MarkLogic Server is through that host.

In local mode, throughput is limited by resources such as memory and network bandwidth available to the host running mlcp.

You can use mlcp even when a load balancer sits between the client host and the MarkLogic host. The mlcp tool is compatible with AWS Elastic Load Balancer (ELB) and other load balancers.

Starting in 11.1.0, mlcp supports connecting to MarkLogic Server through a reverse proxy using path-based routing. For details, see mlcp Reverse Proxy Support Using Path-Based Routing.