Network Topologies

This topic summarizes the main network topologies you can use for migration jobs, from an external Flux runner to private connectivity. Choose the simplest option that meets your throughput, security, and operational requirements.

The network topology determines where migration jobs run, how credentials are handled, and which endpoints must be reachable.

The available topologies are summarized below:

External Flux runner to PDC

Use an operator-controlled VM, build agent, or migration host outside PDC to run Flux jobs against the source and target MarkLogic endpoints.

• Best for: pilots, one-off migrations, repeatable scripted imports, and teams that already operate migration workers.
• Connectivity: the runner needs outbound access to the source system and to the PDC MarkLogic endpoint over TLS.
• Security notes: store credentials in a vault or CI/CD secret store; avoid embedding API keys in command history or logs.
• Operational notes: size the runner for network throughput and parallelism; keep logs and Flux job outputs for auditability.

Backup/Restore through cloud object storage

Use cloud object storage as the exchange point for native MarkLogic backups and journal archives. This is usually the preferred topology for large production migrations where minimal downtime is required.

• Best for: large datasets, high write rates, predictable point-in-time recovery, and short cutover windows.
• Connectivity: the source cluster must be able to write backups and journals to a PDC-accessible storage location.
• Security notes: encrypt backups at rest, restrict bucket/container access, rotate temporary credentials, and validate restore permissions before the cutover window.
• Operational notes: rehearse restore and journal replay; measure transfer, restore, and replay duration; maintain retention until business acceptance.

Private connectivity for regulated or high-throughput environments

Use private network connectivity when public endpoint access is not acceptable or when predictable throughput is required.

• Best for: regulated workloads, high-volume production migrations, strict network segmentation, or environments with limited public egress.
• Connectivity: validate DNS resolution, routing, firewall rules, TLS certificates, and source/target port reachability before running migration jobs.
• Security notes: document approved CIDR ranges, certificate trust chains, and identity boundaries.
• Operational notes: perform bandwidth tests, monitor packet loss and latency, and align job parallelism with the available link capacity.

Pre-flight network checklist

• Source system can reach the selected transfer path: PDC endpoint, or cloud object storage.
• DNS, TLS certificate chains, firewall rules, proxy settings, and allow-lists have been validated from the actual migration runtime.
• Required API keys, service users, IAM roles, storage policies, and MarkLogic privileges are least-privilege and time-bound where possible.
• Throughput test completed with representative batch sizes and thread counts.
• Logs, metrics, and audit records are captured for every migration job and retained until acceptance.