Issues fixed in MarkLogic Server 9.0-5

Having a protected path defined breaks search:suggest to work on a suggestion source based on a field

Having a protected path defined (any path, even a path does not apply to documents being searched) breaks (no result return) search:suggest to work on a suggestion source based on a field. It does not break search:suggest working on a suggestion source based on an element.

sem:database-nodes returns no nodes for unmanaged triples in JSON documents

If a triples was indexed because it is found in a JSON document of the proscribed form, that node will not be detected when running sem:database-nodes on the indexed triple.

TDE variable shadowing breaks variable evaluation.

Variable evaluation fails when the TDE child template has a variable with the same name as its parent TDE template.

cts:aggregate functions do not support a timezone=TZ option

cts:aggregate functions such as cts:min and cts:max do not support a timezone=TZ option for dateTime objects in the same way that cts:value-tuples does.

SPARQL ORDER BY DESC performance doesn't scale

SELECT * WHERE { ?S ?P ?O } ORDER BY DESC(?O) LIMIT 10 takes longer to execute, depending on graph size

op:join-doc loads elements protected by element level security for users that do not have permission to view them

In some cases, the result from op:join-doc doesn't have elements properly concealed based on element level security for unprivileged users.

500 error thrown for invalid input to REST API extension

The REST API supports resource service extensions so developers can expose new functionality on the /v1/resources endpoint. The extension can throw the RESTAPI-SRVEXERR error during execution to provide errors to the client. The resource service extension cannot handle an error thrown before or after the resource service extension executes. In particular, if the client sends an invalid JSON document to a resource service extension, the resource service extension cannot handle the error.

Unable to rename MarkLogic nodes on EC2

MarkLogic nodes on EC2 can't be renamed through Admin-UI or Admin-API.

xdmp:host-status might show incorrect converter version

If the version of MarkLogicConverters installed along with MarkLogic Server is different from the version of MarkLogic Server, xdmp:host-status always shows server's version as converters-version.

XSLT current-group() not resolved properly

current-group() is not returning the correct sequence if there is another element preceding it.

exporting an Optic query with document joins from XQuery

An Optic query can be exported for later execution by POST to the /v1/rows endpoint of the REST API. If the query is constructed in XQuery containing an op:join-doc() or op:join-doc-uri() operation, the import generated an invalid serialisation that failed with a 500 error in the REST API.

Improve robustness of Encryption

If an encrypted multi-forest database backup is taken by multiple hosts to a shared file location, some forests in the backup cannot be decrypted because different hosts used different encryption keys, but only one of the keys is saved in the shared BackupKEKLabel file. Now it allows all the hosts use the same key.

SPARQL filters using stemmed queries fail to return results

A SPARQL query with a filter of fn:contains with a stemmed query is not match values with language strings for the language given in the query.

Backup purge removes backup from other databases

Each database should use a different backup directory. In other words, different databases should not share the same backup directory. Backup purge will remove the old backup regardless of which database the backup is for. The behavior in the description is expected.

Poor Chinese tokenization

Even when advanced language support is licensed, Chinese tokenization for both traditional and simplified Chinese (zh, zh-Hant) produces some poor results, splitting some words into two and combining other words together.

cts:geopatial-region-query throws error XDMP-BADLINESTRING

cts:geopatial-region-query throws XDMP-BADLINESTRING when matching certain polygons.

Missing permissions in mlcp JavaScript transform.

Some user-specified permissions are missing in mlcp JavaScript transform.

MarkLogic Server is vulnerable to the "Billion Laughs" (XML Bomb) vulnerability

Loading the malicious file will capture CPU and memory resources. The query will not properly time out and cannot be canceled.

cts.referenceParse() does not handle cts.geospatialPathReference

cts.referenceParse() throws XDMP-RANGEINDEXNODE when it is passed cts.geospatialPathReference

Allow entries in the spelling dictionary to be longer than 64 characters

In some cases, a dictionary may need to hold specific words that exceed 64 characters in length. Currently, this is not allowed, and words longer than 64 characters will trigger a schema error.

Incorrect application of use-db-config option on cts:distinctive-terms and cts:similar-query

The use-db-config option is not being correctly applied on cts:distinctive-terms and cts:similar-query, leading to inappropriate results.

Query by Example including a query with ampersand in value throws XDMP-ENTITYREF

Evaluating a Query by Example with value comparisons which includes a query with ampersand in text value throws XDMP-ENTITYREF - Invalid entity reference

Extra spaces are introduced when some pdf file is filtered using xdmp:document-filter

When using xdmp:document-filter parsing PDF files in some cases extra spaces are added.

Ttriples, Encryption and RAM

Many triples on encrypted databases uses large amounts of RAM

No option for destination collection or directory for OpenXML pipeline

The OpenXML is missing support for destination collection or directory for generated files.

Confusing error message from mlodbcinst during odbcdriver installation

When "odbcinst" is not installed, installation of odbcdriver throws error that is confusing. /opt/MarkLogic/bin/mlodbcinst: line 5: which:: command not found odbcinst: command not found Install unixODBC in /usr/local or /usr

Unable to install OpsDirector on cluster with no account called admin

If the user called 'admin', with role 'admin' does not exist, installation of OpsDirector fails with an error. This problem has been resolved in Ops Director 1.1-1 by selecting a user with the admin role irrespective of the name of that user.

boolean-node constructor has "true" value unexpectedly

The boolean-node constructor should behave consistently with number-node. The original version is unexpected.

Query console not showing error on result tab

When using xdmp.httpGet in a query, if the response returns a SVC-SOCCONN error, it is not rendered in the Result tab.

Admin UI is missing a force bulk replication in database replication

There is no option to stop and restart a database replication temporarily nor you can force a bulk replication after a cluster disconnection to bring the master and replica back in sync quickly.

SEC-PRIV returned from REST request on ssl enabled App-Services

When ssl is enabled on the App Services (port 8000) app server and a REST call is made to that port, SEC-PRIV error is returned for a rest-reader user.

cannot pause or suspend database replication from Admin UI

There is no option to stop and restart a database replication temporarily nor you can force a bulk replication after a cluster disconnection to bring the master and replica back in sync quickly.

EBS volumes not mounted to data directory on C5 and M5 instances

Because C5 and M5 instances expose NVMe device directly, which is different from other instance families, the EBS volumes attached are not correctly mounted to MarkLogic data directory.

Documented privilege (xdmp-filesystem-file-get-time) missing from MarkLogic Server

xdmp-filesystem-file-get-time is not installed with a clean install of MarkLogic Server, so if you run xdmp:filesystem-file-get-time with a user with the "filesystem-access" role, you'll hit an exception citing the missing privilege.

SQL left joins sometimes return incorrect results

In some cases where LEFT JOIN clauses are used, the SQL optimiser can chose an algorithm that returns incorrect results when the join condition is against a nullable column.

Query by Example with null value in JSON property throws XDMP-AS

Query by Example with null value in JSON property throws an XDMP-AS error - Invalid coercion: () as xs:string

Segmentation faults in position handling code in query processing

If a database is configured with phrase arounds, and phrase throughs or element word query throughs, and these elements may nest in the actual data, stored positions may be incorrect, leading to segmentation faults in position handling code, or incorrect results.

Creating a REST API server after renaming the Schemas database causes error

The /v1/rest-apis endpoint on port 8002 creates the REST API server and, potentially, content and modules databases for the server. Previously, if the Schemas database had been renamed, the request returned an error. Now, the request succeeds. You should use the Admin UI, the Admin API, or the REST Management API to specify the schemas database for the content database after the request finishes.

MarkLogic doesn’t have proper support for AppServer error handler in JavaScript

MarkLogic doesn’t have proper support for AppServer error handler in JavaScript

Incorrect PDF document-filter results

xdmp:document-filter() result for some pdf files are garbled

Segmentation fault in X509_subject_name_cmp() from lib/libcrypto.so.1.0.0

In some cases a segmentation fault (in X509_subject_name_cmp() from lib/libcrypto.so.1.0.0) has been observed in situations where multiple users are performing ldap authentication with simple bind and ssl configured

Wrong value reported for XDQP transfer rates

Values reported for XDQP transfer rates are about 10x too high.

SPARQL UPDATE [INSERT/DELETE] clause locks the default graph when a WITH clause is not used

In a situation where an insert or a delete is evaluated by SPARQL UPDATE, if the graph is not specified within the statement (using a WITH clause), the default graph is locked until that update has completed.

New database's encryption key not saved

When creating a new database while on external kms, if you entered a key ID to encrypt the database, it would not be saved.

triple deduplication for Optic queries in XQuery

The Optic op:from-triples() accessor takes a options parameter, which can specify deduplication of triples. An XQuery call passed the deduplication option without also passing the fragment id argument, the option was ignored and duplicate triples were returned.

Host memory-process-swap-size and memory-file-size incorrect

Host metrics memory-process-swap-size and memory-file-size report wrong data. This effects error logging, xdmp.hostStatus and host metrics in Meters database.

cts:near-query-options returning incorrect values when query is serialised as XML then back to a cts:query

In some cases, a call to cts:near-query-options returns the "distance" value in addition to the options sequence. This has been observed when the query is serialised as an XML element and then the cts:near-query-options is called on the transformed content.

Management API ignores 'X-Error-Accept'

Management API port 8002 ignores the 'X-Error-Accept' option. Will return message REST-REQUIREDPARAM: (err:FOER0000) Required parameter: uri

Some wildcard queries producing incorrect results with limited wildcard indexing enabled

On a database with limited support for wildcard indexes (trailing wildcards only, no word lexicons), simple one and two character wildcard queries (e.g. "a*") produce query plans that are missing annotations and produce incorrect results.

ADMIN-INVALIDCONFIG host is missing host-mode

Host rename results in ADMIN-INVALIDCONFIG Invalid configuration: host is missing host-mode

Windows jemalloc purge decay

Environment variable JE_MALLOC_CONF was not set to purge:decay during MarkLogic installation on Windows.

Module cache performance improvement

Performance improvement when there is an update or delete to a modules database.

xdmp:document-filter() fails to filter word-art text from specific Microsoft Excel file formats (.xls and .xlsx)

In cases where Excel spreadsheets contain "Word Art" objects, calls to xdmp:document-filter will fail to filter word-art text from files with an .xls file extension, while for the .xlsx file format xdmp:document-filter filters word-art text successfully.

Default permission not applied to documents imported with mlcp transform

With XQuery transform modules, default permissions or collections are not applied to documents imported with mlcp transformation. With JavaScript transform modules, default permissions are not applied to documents imported with mlcp transformation.

Quotes around boundary string in multipart/form-data header produce incorrect results

When MarkLogic receives a multipart/form-data payload with a Content-Type header that wraps a boundary token in quotes, it fails to decode the payload into parts.

apply values options to aggregates in Search API

In the Search API, the values-options element can specify options to apply to lexicon lookups. Under certain circumstances some of these options (timezone definition) are ignored and result in wrong results.

Unable to use EC2 instance's IAM role when MARKLOGIC_EC2_HOST is disabled

Setting MARKLOGIC_EC2_HOST to 0 in /etc/marklogic.conf, disables all AWS related feature, including using EC2 instances' IAM role to do S3 backup and restore. With the current changes, MARKLOGIC_EC2_HOST is still the global switch for "EC2 awareness". If this flag is set to 0 (disabled), all of AWS related feature will not be used. MarkLogic will not access instance metadata by any means. By default, this flag is set to enable (1). A new environment variable MARKLOGIC_MANAGED_NODE is introduced. This variable is to control managed cluster feature. If disabled, MarkLogic will NOT automatically mount volumes, report instance status to DynamoDB or automatically join a cluster. By default, this flag is set to enable (1). In order to leverage the functionality of IAM role without Managed Cluster, the variables should be set to MARKLOGIC_EC2_HOST=1 and MARKLOGIC_MANAGED_NODE=0. Since MARKLOGIC_EC2_HOST is enabled by default the only mandatory change is to use MARKLOGIC_MANAGED_NODE=0.

Database files are not encrypted with the database level key

The database files are encrypted with the cluster level key and not the database level key

Track and limit registered query memory

Track registered query memory as memory-registry-size in host-status and as "registry=" in the "Memory" log message. Limit registered query memory to 1/8 of physical memory by aging out old entries. This can be increased to 1/4 of physical memory by enabling the "Registry Size Double" trace event.

Host memory-process-swap-size and memory-file-size incorrect

Host metrics memory-process-swap-size and memory-file-size report wrong data. This effects error logging, xdmp.hostStatus and host metrics in Meters database.

JSON Query By Example (QBE) fails to match

A JSON QBE specifying a query for a property or element name having an underscore fails to match.

Wrong values are reported for cache hits/misses sometimes.

Wrong values are reported for cache hits/misses sometimes in xdmp.serverStatus, xdmp.forestStatus and in performance metrics in the Meters database for forest, database and app servers.

Documented privilege (xdmp-filesystem-file-get-time) missing from MarkLogic Server

xdmp-filesystem-file-get-time is not installed with a clean install of MarkLogic Server, so if you run xdmp:filesystem-file-get-time with a user with the "filesystem-access" role, you'll hit an exception citing the missing privilege.

missing AppServer Javascript "error" property

There is no "errors" property in the context of the app server handler in javascript (whereas it exists in xquery context)

Configuration Manager - Showing invalid difference count between current and imported configurations

In some cases, when comparing an imported package with an existing one using configuration manager it shows the total difference as 1 but when we expand it we can't see any difference in the details view. This is attributed to a miscalculation in the diff count.

cts:geospatial-region-query times out

cts:geospatial-region-query using "crosses" operation times out in wgs84 coordinates when matching a linestring with a linestring

The sem:isIRI() function raises an error when passed the empty sequence

An error is returned if an empty sequence is passed to see:isIRI().

/admin/v1 Endpoints Unavailable in Admin Server of New Groups

When a new group is created (either in the Admin UI or via the Mgmt REST API), then the newly create Admin server in that group does not have the “url rewriter” value set to “rewriter.xqy”, resulting in the /admin/v1 endpoints being unavailable (return 404).

xdmp.filesystemFile cannot read encrypted file on remote host.

xdmp.filesystemFile cannot read an encrypted file on a remote host, giving error "ReadFile File is not in UTF-8".

fn:contains throws SVC-UTF8SEQ

With three-character-searches on, fn:contains throws SVC-UTF8SEQ when dealing with certain high-codepoint characters.

cts:geospatial-region-query throws error XDMP-BADPOLYGON

cts:geospatial-region-query throws XDMP-BADPOLYGON for certain polygons

Setting the environment-ui/ui-message element to empty of cluster-ui-settings.xml causes Query Console XDMP-OBJCONSTRUCTCHILDSEQ error

When attempting to set an environment without a pop-up message, for example: <env-ui:environment-ui xml:lang="zxx" xmlns:env-ui="http://marklogic.com/environment-ui"> <env-ui:ui-active>{fn:true()}</env-ui:ui-active> <env-ui:ui-label>banner_text</env-ui:ui-label> <env-ui:ui-header-color>#FFA500</env-ui:ui-header-color> <env-ui:ui-header-text-color>#FFFFFF</env-ui:ui-header-text-color> <env-ui:ui-message></env-ui:ui-message> </env-ui:environment-ui> Reloading the Query Console results in a 500 XDMP-OBJCONSTRUCTCHILDSEQ error.

mlcp can't connect with AppServer requiring TLSv1.2.

mlcp can't connect with AppServer requiring TLSv1.2 as it defaults to SSLv3.

TDE: A TDE nullable column should not error out on a JSON null node

When a TDE column is nullable, it should not error out on a JSON null node. The original code throws an error on a JSON null node.

There is no option to disable external system entities from being processed by XML parser

If an XML document contains a system entity with a URI that references the local file system, and the MarkLogic process has permissions to read that file, loading the XML document will cause the entity to be dereferenced and included. There is no option to disable system entities.

Subtraction of xs:time values on Windows incorrect

Subtraction of untimezoned xs:time values on Windows may give incorrect answers when the time value is within the timezone offset after midnight. Example: xs:time("24:00:00")-xs:time("23:59:59")

MarkLogic instances are not restarted when configuration is updated.

If a configuration change is made on hosts in a particular group, other hosts will not restart if they are in the same cluster but in different groups. These other hosts will subsequently not pick up the configuration change initiated on the original host.

CA certificates missing

Required CA certificates may be missing depending on the bootstrap installation method used when installing the MarkLogic server

term options for the default query in the Search API

The query options for the Search API can specify options for terms that aren't tagged with a constraint. Such options were not used if the query options also specified a default query for such terms. This returns unexpected search results.

Duplicate configuration keys for certain configurations of Geospatial Region Indexes

For certain configurations of Geospatial Region Indexes duplicate configuration keys are generated. Note: Upgrade to ML 9.0-5 may cause the server to reindex if it has Geospatial Region Indexes configured.

Specifying a transform_module with a MLCP command using output_permissions removes the "read permission

When mlcp is invoked specifying a -transform_module and -output_permissions the "read" permission is removed.

LDAP authentication/authorization can use case sensitivity on LDAP DN

MarkLogic matches User/Group external name with DN with case-sensitive comparison which can fail in some circumstances.

Encrypted forests become un-encrypted when migrated to another host in the cluster.

During migration of a forest from one node to another in a cluster an encrypted forests become un-encrypted on the destination host in the cluster.

Documented privilege (xdmp-filesystem-file-get-time) missing from MarkLogic Server

xdmp-filesystem-file-get-time is not installed with a clean install of MarkLogic Server, so if you run xdmp:filesystem-file-get-time with a user with the "filesystem-access" role, you'll hit an exception citing the missing privilege.

JSON serialization of a cts:query in a REST search response

The Search API provides a return-query flag that provides the executed cts:query in the search response. When serializing the search response to JSON, the REST API didn't emit the cts:query in the standard JSON serialization for the query. As a result, the value of the "query" property in the search response could not be passed to the cts:query() constructor to construct an executable cts:query.

cts:geospatial-region-query returns incorrect results

cts:geospatial-region-query returns incorrect results when matching certain linestrings with points within tolerance of its boundary points

cts:range-query incorrectly applies collation option or default collation

cts:range-query should use the collation associated with the reference and not the query options (or the default collation). In some cases the query may fail to return results.

Union queries surrounded by a group by operator doesn't return the correct results

When the group by operator surrounds the union clause, the distinct operation as part of the union clause only acts on the columns present in the group by clause

SQL queries with sub-query in HAVING clause are slow

SQL queries with sub-query in HAVING clause are slow

Incorrect fragment-count in reindex-protected-paths element of forest-count api

The fragment-count in reindex-protected-paths element of forest-count api does not work correctly. It should show the total number of fragments to be reindexed, while the original version always show 0.

Loading bi-temporal documents from the past

Loading bi-temporal documents from the past using system-time results in incorrect system time range

MarkLogic ODBC connector causes "out of memory while reading tuples" with Microsoft Access 2016 (32-bit) on large datasets

When the 32-bit version of Microsoft Access is used on 32-bit Windows to connect (over ODBC) to MarkLogic, the process of linking larger tables causes Microsoft Access to throw an OOM exception.