Between MarkLogic 12 and MarkLogic 11

Automatic start when installing on Linux

There is a current difference in expected behavior when installing MarkLogic 12.0.0 on RHEL 8 or RHEL 9. While our documented start-up steps instruct you to run a start command following an install, currently, MarkLogic 12.0.0 automatically starts following the installation on RHEL 8 or RHEL 9. This is a known issue that will be fixed in a future patch release.

The BM25 lengths file will be generated on startup

MarkLogic 12.0 adds BM25 as a new search scoring method. To support this feature, a new file is generated that contains the length of each document in each stand of each forest. After you upgrade, when MarkLogic Server is first started and the forests are mounted, MarkLogic Server automatically generates the required Lengths file.

On startup, when mounting a forest, if the Lengths file does not exist, then it is generated by reading the compressed tree file for each stand. Depending on the I/O bandwidth available and the size of the forest, this process can cause a noticeable delay before the database becomes available. Messages like these will be written to the error log when a Lengths file needs to be generated:

Info: Creating /space/Forests/Forests/MyForest/0000005c/Lengths

New minimum CPU requirements

As of 12.0, MarkLogic Server requires CPUs that support the AVX2 instruction set.

Minimum Intel CPUs:

• Haswell (Q2 2013) or newer Core processor
• Tiger Lake (Q3 2020) or newer Celeron or Pentium processor

Minimum AMD CPUs:

• Excavator (Q2 2015) or newer processor

Operating system support changes

MarkLogic 12.0 no longer supports the following:

• Red Hat Enterprise Linux 7
• macOS with Intel CPUs (use Docker instead)
• Windows Servers with GPUs
• GPUs on RHEL 9 and AL2023 (this was dropped in 11.3 on RHEL 9 and AL2023 as well)

JavaScript main modules require export default to return anything

Due to the upgrade of the embedded Google V8 JavaScript engine, JavaScript modules that are invoked as main programs (for example, through an eval or as an HTTP endpoint) behave differently in MarkLogic 12.0. This type of JavaScript module files (.mjs) now need to use export default to explicitly return a value. These examples illustrate the new behavior:

Previously:

const result = {
    "hello": "world",
    "yourParam": external.myParam,
    "theDoc": external.doc
}
result

Now:

const result = {
    "hello": "world",
    "yourParam": external.myParam,
    "theDoc": external.doc
}
export default result

To retain backwards compatibility, a combination of both returns can be used:

const result = {
    "hello": "world",
    "yourParam": external.myParam,
    "theDoc": external.doc
}
result
export default result

What is considered a JavaScript “module”

These are all executed as JavaScript modules in MarkLogic Server:

• JavaScript files ending in .mjs
• JavaScript files ending in .sjs but that are called from another .mjs JavaScript file
• Any code that uses JavaScript module functionality (directly or indirectly) like import, or export. This includes ad hoc code run through Query Console.
• JavaScript code executed through xdmp.eval("import….; const result =…;… ; export default result; ….;")

Running in Query Console

When running JavaScript code in Query Console that is interpreted as a JavaScript module rather than a script, the last line needs to export the result in order for Query Console to display it.

For example,

'use strict';
import * as optic from '/MarkLogic/optic.mjs';
const result = "my result"
result

will not return the my result value when run in Query Console. This is because the import line causes it to switch to run as a module.

To see the content of the result variable, add the export default result line as the last line:

'use strict';
import * as optic from '/MarkLogic/optic.mjs';
const result = "my result"
export default result

const isModule = (this === undefined) ? true : false;

Known issue with the JavaScript debugger

Due to the Google V8 engine upgrade in MarkLogic 12.0, the JavaScript debugger may hang and need to be restarted. This is a known issue that will be resolved in a future release.

Document conversion functions and CPF pipelines have been removed

As part of our ongoing commitment to security and maintainability, the legacy document conversion capability has been removed in MarkLogic 12.0. The conversion feature supported the ability to convert PDF and Microsoft Office 2007 document formats to renderable XML.

These functions are no longer supported:

• xdmp:pdf-convert()
• xdmp:excel-convert()
• xdmp:powerpoint-convert()
• xdmp:word-convert()

These functions are still in the MarkLogic 12.0 codebase but, when invoked, throw the XDMP-CVTNOTSUPPORTED error.

When upgrading to MarkLogic 12.0, if the default Content Processing Framework (CPF) conversion has been installed, then the MS Office Conversion and PDF Conversion (all variations) will stop working. The pipelines remain installed, but any new content that matches the configured domains fails to process.

Once you upgrade, uninstall the incompatible CPF pipelines to prevent future errors.

Upgrading when XDQP SSL enabled and database replication is configured

When you upgrade to MarkLogic 12.0, if XDQP SSL is enabled and the cluster is running in FIPS mode (the default), then you must use a slightly different upgrade process than the one at Upgrading Clusters Configured with Database Replication. This different process is needed because versions prior to 12.0 support only TLS 1.2 (without EMS), but MarkLogic 12.0 runs in FIPS 140-3, which requires TLS 1.3 or TLS 1.2 with EMS.

To make it possible to upgrade, MarkLogic 12.0 runs in FIPS 140-2 mode until the upgrade is committed. This strategy allows hosts in the same cluster or in external clusters to connect with TLS 1.2 without EMS and continue to communicate with MarkLogic 12.0 hosts. Once the upgrade in a cluster is committed, the MarkLogic 12.0 hosts switch to FIPS 140-3 mode, now requiring TLS 1.3 or TLS 1.2 with EMS.

Here is the new upgrade procedure:

1. Upgrade the replica cluster, but do NOT commit the upgrade. The effective version remains the previous version of the cluster and remains running in FIPS 140-2 mode.

2. Upgrade the primary cluster, but do NOT commit the upgrade. The effective version remains the previous version of the cluster and remains running in FIPS 140-2 mode.

3. On the replica cluster, commit the upgrade. This commit triggers all cluster hosts to restart and switch to FIPS 140-3 mode.

4. On the Master cluster, commit the upgrade. This commit triggers all cluster hosts to restart and switch to FIPS 140-3 mode.

LDAP MD5 bind is no longer supported

Due to the upgrade of OpenSSL in MarkLogic 12.0, the MD5 algorithm is no longer available to use as a bind method for LDAP authentication. If there are any existing external security configurations that use MD5 for the LDAP bind method, then switch LDAP Bind Method to simple or external.

If any external security configurations are present, then MarkLogic 12.0 logs a message like this one:

2025-04-29 09:39:58.729 Info: XDMP-LDAPMD5REMOVED: External Security demo_ldap is using MD5 as the LDAP bind method. As of MarkLogic 12.0, MD5 bind is no longer available for security reasons. Please switch to use simple or external bind method.

Updated stemming dictionaries

MarkLogic 12.0 includes an update to the dictionaries that are used for stemmed searches. This includes updates to thousands of words across English and 12 other languages as well as stemming changes like the handling of s and z in British English. For databases that have stemmed searches enabled, this update to the dictionaries may cause different behavior when running cts:search() (with the stemmed option explicitly or implicitly set) queries against databases with stemmed searches set to basic or advanced, as well as direct calls to cts:stem(). This behavior affects the content in these languages: English, French, Italian, Spanish, German, Russian, Dutch, Portuguese, Korean, Swedish, Nynorsk, Bokmal, Arabic, and Persian.

This table illustrates the behavior for words that have been updated.

To ensure that content is indexed using the new dictionaries and that searches behave as expected, a forced reindex is the most effective approach. Due to the CPU- and I/O-intensive nature of reindexing a large dataset, there may be circumstances where a full reindex is not possible. In these cases, a more selective reindex may be possible by only reindexing the content that contains words that have been changed in the dictionary update. Contact support for information on this approach if a full reindex is not possible.

This table summarizes words and their corresponding stem compared to previous versions of MarkLogic Server:

¹ Delivered but not being used. Words can be tokenized to no clitics.

² Removing duplicates. yo/ye are for letter e to stem ё.

Switching between materialized and virtual views

TDE views from systems before MarkLogic 12.0 are also known as materialized views. In MarkLogic 12.0, views and columns can be toggled between materialized and virtual by changing view-virtual and virtual between true and false.

These rules apply when switching from non-virtual views or columns to virtual views or columns:

• Switching from materialized to virtual:
  • For TDEs created in MarkLogic 12.0: Reindexing is not required. Resources are reclaimed as merges happen in the system.
  • For TDEs created before MarkLogic 12.0: Reindexing is required only the first time it is switched. To mitigate, create new views that are virtual, use Query-Based Views, and join the existing views with new virtual views or columns.
• Switching from virtual to materialized: Reindexing is required before the toggled view or column becomes available.

TDE context paths for sub-templates are no longer allowed to start with a forward slash

Prior to MarkLogic 12.0, context paths for sub-templates within a TDE template could start with a forward slash (/), and it was stripped off when processing the template. Because the / in XPath actually indicates that the root of the document should be selected, it is not a valid path to use as a sub-template in a TDE template. Therefore, in MarkLogic 12.0, paths of this form in a sub-template context are no longer allowed and will throw an error when validating the template.

TDEs handle empty element values as the empty string

Before MarkLogic 12.0, if an XPath in a TDE selected an XML element and the element existed in the document but the value was an empty string, then it was considered as a null value rather than as the empty string. In MarkLogic 12.0, this value is considered as an empty string. If the element is not present in the document, then it still results in a null value.

So, if a database has XML documents with empty element values and TDE templates match those documents, then previously null columns will start to have empty string values as new documents are written to the database or existing documents are updated or reindexed.

TDE templates with sub-templates that have the same context path are not supported

Before MarkLogic 12.0, TDE templates that contained a sub-template with the exact columns and column settings as the parent template would be inserted even though the template would not work correctly. In MarkLogic 12.0, these templates are considered invalid. So, they will not pass the validation check and will fail to be inserted.

Optic fromSearch() now locks documents in update transaction

Prior to MarkLogic 12.0, documents selected by the Optic API fromSearch() or fromSearchDocs() functions did not lock if run in an update transaction. In MarkLogic 12.0, read locks are taken against these documents.

The default healthcheck app server threads setting is increased to 32

The default value for the threads setting of the HealthCheck 7997 app server has been raised from 8 to 32. The existing configuration is respected during upgrade.

Private keys need to be at least 2048 bits

With the upgrade of OpenSSL, certificate private keys now need to be least 2048 bits. Certificates will fail to import if they were generated using a private key with fewer than 2048 bits.

Existing certificates generated with a private key with less than 2048 bits will no longer work. Replace any such certificates before upgrading to ensure the cluster does not become unexpectedly unavailable.

The MLCMD component is no longer part of the MarkLogic Server RPM

Before MarkLogic 12.0, the MarkLogic Server RPM included a component called MLCMD. This component manages a number of aspects when MarkLogic Server is run as a “Managed Cluster” in AWS (see Overview of MarkLogic Server on AWS). In MarkLogic 12.0, the MLCMD component is no longer included in the main RPM. It is now packaged and shipped only with the MarkLogic Server AMIs published on the AWS Marketplace.

Separating MLCMD allows for quicker releases of fixes and updates of supported instance types for the MarkLogic Server AMIs.

The Entity Services functions are no longer supported

The Entity Services functions were deprecated in MarkLogic 10.0-5 and, other than es:model-validate(), are no longer supported as of MarkLogic 12.0. This table lists the functions that are no longer supported.

Certain sec:create-external-security function signature have been removed

These function signatures for sec:create-external-security() have been removed as of MarkLogic 12.0:

create-external-security(
  $external-security-name as xs:string,
  $description as xs:string,
  $authentication as xs:string,
  $cache-timeout as xs:unsignedInt,
  $authorization as xs:string,
  $ldap-server-uri as xs:string,
  $ldap-base as xs:string,
  $ldap-attribute as xs:string,
  $ldap-default-user as xs:string,
  $ldap-password as xs:string
) as xs:unsignedLong

create-external-security(
  $external-security-name as xs:string,
  $description as xs:string,
  $authentication as xs:string,
  $cache-timeout as xs:unsignedInt,
  $authorization as xs:string,
  $ldap-server-uri as xs:string,
  $ldap-base as xs:string,
  $ldap-attribute as xs:string,
  $ldap-default-user as xs:string,
  $ldap-password as xs:string,
  $ldap-bind-method as xs:string
) as xs:unsignedLong

create-external-security(
  $external-security-name as xs:string,
  $description as xs:string,
  $authentication as xs:string,
  $cache-timeout as xs:unsignedInt,
  $authorization as xs:string,
  $ldap as element(sec:ldap-server)?,
  $saml as element(sec:saml-server)?,
  $client-certificate-authorities as xs:unsignedLong*,
  $require-client-certificate as xs:boolean
) as xs:unsignedLong

See sec:create-external-security() and sec.createExternalSecurity() for the supported function signatures.

mlcp no longer depends on the MarkLogic build of Apache Commons CSV

MarkLogic 12.0 addresses a longstanding issue with the Apache Commons CSV library. Previously, because the maintainers of Apache Commons CSV did not publish a fix required by mlcp, a separate fork was maintained and hosted internally. Build files had to be configured to pull this dependency from the MarkLogic custom Maven repository.

The fix for tracking byte positions correctly has been accepted by the Commons CSV maintainers and is now available as of version 1.13.0. With this update, mlcp now resolves the commons-csv dependency from Maven Central, eliminating the need to manually configure or pull from the MarkLogic repo. This change preserves the same functionality while offering a smoother, more standard dependency management experience.

HTTP HEAD and POST requests against an HTTPS app server result in connection errors

Previously, HTTP HEAD and HTTP POST calls against app servers configured to use HTTPS resulted in a response with an HTTP 403 error. In MarkLogic 12.0, the same requests against an app server configured to use HTTP results in connection errors because the client is trying to connect to an HTTPS server with HTTP.

Invalid values for path range indexes against JSON arrays throw errors instead of silently failing

Before MarkLogic 12.0, path range indexes of type string that select a JSON array node directly (instead of selecting the values in the array) do not put values into the range index because the array cannot be converted into a string. This condition silently failed with no messages in the logs even if invalid values is set to reject.

In MarkLogic 12.0, a path range index of scalar type string that selects an array node directly with invalid values set to reject throws an error rather than silently ignoring it. Code that previously succeed when inserting JSON documents with arrays now fails if one or more path range indexes are configured to use a path that selects an array node directly.

For example, given this document,

{
  "directory": {
    "name": "myDir",
    "files": [
      "file1",
      "file2"
    ]
  }
}

a path range index of

/directory/node("files")

or

/directory/array-node("files")

with invalid values set to reject would have not caused an error in versions before MarkLogic 12.0 but will now.

New REST API content transform parameters

REST API content transforms now support these context parameters:

• collections
• permissions
• quality
• temporalCollection (temporal-collection in XQuery)
• metadata

If you have code that sets these parameters in the content transform context with the assumption that they are ignored, then you must change this code, because MarkLogic 12.0 no longer ignores them, so your code may behave differently.

See Working With Content Transformations.

v1/documents URI handling made consistent

In MarkLogic 12.0, HTTP PUT calls to the v1/documents REST API now accepts URIs with spaces. Before, these calls failed with invalid URIs errors. MarkLogic 12.0 now has consistent behavior for URI handling between the HTTP PUT and POST methods.

xs:numeric is now compliant with the W3C specification

MarkLogic 12.0 updates the behavior of the xs:numeric type so that it follows the W3C specification. When taking an integer, float, or double as the input, xs.numeric no longer modifies the input and returns it directly.

This change also affects JavaScript code that uses xs.numeric and tries to get the type of the result: