Operational Runbooks

Use these runbooks to move data to PDC in a predictable way. They show the main steps for Flux copy, backup and restore, cutover, and validation.

Runbook: Flux Copy with Top-off

Use this runbook to copy data incrementally with Flux, then perform a final top-off before cutover. It helps minimize downtime while ensuring that recent changes are included in the target system.

1. Start the PDC service.
2. Deploy the configuration and security settings with ml-gradle.
3. Run the first Flux copy job.
4. Copy data by collection or directory.
5. Use --batch-size and --thread-count values that fit your data and environment.
6. If the source is S3 or Azure, use the Flux import commands with cloud credentials.
7. If the source is another MarkLogic system, use Flux copy.
8. Run follow-up sync jobs until only a small amount of data is left to move.
9. Watch the success counts and sample data checks to confirm the copy is still on track.
10. Schedule cutover, freeze writes, and run a top-off Flux job to pick up late changes.
11. Use a last-modified filter, such as a CTS range or timestamp field, for the top-off job.
12. Switch traffic to the PDC endpoints.
13. Run validation and monitor the system after cutover.

Runbook: Backup and Restore with Journal Archiving

Use this runbook to restore a full backup to PDC and apply journal updates until cutover. It helps reduce downtime while keeping the target system current with source changes.

1. Turn on journal archiving on the source system.
2. Set the retention policy and the shipping path.
3. Take a full backup.
4. Transfer the backup to storage accessible by PDC.
5. Restore the backup in the target service.
6. Start the journal shipping process.
7. Rehearse journal replay before cutover.
8. At cutover, pause writes on the source system.
9. Ship the final journals.
10. Replay the journals in the target system.
11. Switch traffic to the PDC endpoints.
12. Confirm that the transactions and applications are working correctly.
13. Resume normal operations.