AWS configuration variables

On startup, MarkLogic is customizable by a set of environment variables. This applies to all configurations from single nodes managed externally to large distributed clusters using the full Cluster Management features.

These variables can be specified using any method that guarantees the values are present and consistent in the environment, regardless of what method is used to start the server and when the server is started. The variables related to Managed Cluster support also need to be configured properly on a per-instance basis. A simple and reliable method that allows reuse of the same AMI for all instances and doesn't require customizing the AMI itself is to pass the values as EC2 “User Data.” An alternative is to place the variable assignments in /etc/marklogic.conf either during the initial boot or built into a custom AMI dedicated for each equivalent node in the cluster.

When using CloudFormation, the AWS::CloudFormation::Init resource (and the helper cfn-init commands) are recommended for deployment and configuration. For details, see AWS::CloudFormation::Init.

If not using CloudFormation, the lower-level cloud-init service can be used directly. For details, see Run commands when you launch an EC2 instance with user data input.

Other methods can be used to configure the environment as well, but must be carefully considered and tested due to differences in how the system configures the global root environment during boot, run-level changes and manual service operations (start/stop/restart).

Depending on the deployment tools used to initialize the system and the process and ordering of RPM installation, system configuration and startup, different methods of configuration may be needed to make sure the system is configured correctly before the first launch of MarkLogic on that instance, and that all instances in the group have consistent configuration.

The sample Cloud Formation templates implement an architecture and strategy that is well defined and tested. They are a good model to follow as a design pattern regardless of the tools used for implementation.

The following environment variables are recognized on startup of MarkLogic, or are automatically set from several configuration locations. Some values must be the same across all nodes in a cluster and some may vary for each instance. The sample templates and reference architecture use the Auto Scaling Group (ASG) Launch Configuration for initializing instance variables. One ASG per zone is used so that each zone can have different configurations, but within each zone (or ASG) the same values can be used.

• MARKLOGIC_EC2_HOST — If set to 0 then all EC2 all of AWS related features are disabled. MarkLogic will not access instance metadata by any means and the rest of the variables below are unused. By default, this variable is set to 1 (enable).

  This is useful for when you want to manage MarkLogic externally.

• MARKLOGIC_MANAGED_NODE — Controls the managed cluster feature. If set to 0 (disabled), MarkLogic will not automatically mount volumes, report instance status to DynamoDB, or automatically join a cluster. By default, this flag is set to 1 (enable).

  If you only want to use the IAM role, set MARKLOGIC_EC2_HOST=1 and MARKLOGIC_MANAGED_NODE=0.

• MARKLOGIC_BOOT_WAIT — If set, then the value is a number in seconds (default = 30) as the maximum time to wait for the initial data volume ($MARKLOGIC_EBS, default /dev/sdf) to come online. This is only used when MARKLOGIC_EBS_VOLUME is not specified and MarkLogic is waiting for a volume to be attached manually or from an external process.

  If the timeout is reached without a volume attached then startup aborts.

• MARKLOGIC_LICENSE_KEY — A license key to use for this MarkLogic instance. This license key is only valid for a Bring Your Own License (BYOL) AMI or a user-created AMI.

  Note: A License key is not necessary to enable standard features.

• MARKLOGIC_LICENSEE — The Licensee corresponding to MARKLOGIC_LICENSE_KEY.

• MARKLOGIC_AWS_ACCESS_KEY — An AWS Access Key to be used when accessing the AWS Key Management Services (KWS) and the Simple Storage Service (S3). For details, see Configure AWS Credentials.

• MARKLOGIC_AWS_SECRET_KEY — An AWS Secret Key to be used when accessing the AWS Key Management Services (KWS) and the Simple Storage Service (S3). This variable must be explicitly set by the export keyword. For details, see Configure AWS Credentials.

• MARKLOGIC_AWS_SESSION_TOKEN — An optional AWS session token to be used when accessing the AWS Key Management Services (KWS) and the Simple Storage Service (S3). This variable must be explicitly set by the export keyword. For details, see Configure AWS Credentials.

• MARKLOGIC_CLUSTER_NAME — The MarkLogic cluster name used to auto-configure instances and clusters. For SimpleDB this corresponds to the "Domain" used for simpleDB (V8.0.3 and prior). For DynamoDB, this corresponds to the DynamoDB table name (V8.0.4+). This cluster name is required for any of the managed cluster features, including a single node cluster.

• MARKLOGIC_CLUSTER_MASTER — Must be set and equal to "1" for exactly one node in the cluster. The master node will create the initial databases and become the cluster bootstrap host.

  Can be set to 1 for multiple nodes named the same ending in "#" (See MARKLOGIC_NODE_NAME) in which case only the resolved name that ends in "1" will take on the role of cluster master.

• MARKLOGIC_NODE_NAME — A distinct name of a node within a cluster. Required if MARKLOGIC_CLUSTER_NAME is specified. May end in a "#". If the node name ends with a "#" such as "MyNode-#" this is taken as a variable node name. For more information see the discussion of /sbin/service in Deployment and Startup.

• MARKLOGIC_ADMIN_USERNAME — The MarkLogic Administrator username used for initial installations.

• MARKLOGIC_ADMIN_PASSWORD — The MarkLogic Administrator password used for initial installations.

  EC2 user data is not an AWS 'secure location' and cannot be cleared while the instance is running. Variables set in EC2 user data are evaluated as string literals, unlike values in /etc/marklogic.conf, which are parsed as shell 'source' so are always 'plain text' (or base64 encoded).

  The recommended location for configuration variables is /etc/marklogic.conf. For examples of using a secure store for MarkLogic credentials, see Configuration Security Considerations .

• MARKLOGIC_EBS_VOLUME — The volume specification for the primary EBS volume. This volume will be attached to the logical device /dev/sdf, a filesystem is created, if needed, and mounted on /var/opt/MarkLogic. The format for this value is of the form volspec[,volspec ...] where volspec is one of:

• vol-xxxx Attach to an existing EBS volume

• snap-xxxx An AWS snapshot which will be used to create a volume.

• An integer from 1 to 1024 which indicates the size of the volume in GB. A fresh volume will be created.

• A volume specification string in the format compatible with the V1 EC2 CLI tools. This format is currently only supported by using EC2 user data or /etc/marklogic.conf.

• [snapshot-id]:[volume-size]:[delete-on-termination]:[volume-type[:iops]]

  Where:

Examples:

:20::gp2:true - a 20 GB volume with encryption and D storage type

snap-abcde:200::: - Create volume from snapshot "snap-abcde" and change the size to 200GB. Default gp2 volume type.

:1000::io1:2000: - A 1000 GB PIOP volume with 2000 PIOP