Issues fixed in MarkLogic Server 9.0-1

In-Memory Triple Index larger than configured

The In-Memory Triple Index consumes twice as much memory than configured.

Embedded triple in JSON does not allow extra metadata

The embedded triple in the following JSON document is not indexed due to the extra metadata ("foo":"bar"): { "my": "data", "triple":{ "subject": "http://example.org/ns/dir/js/", "predicate": "http://xmlns.com/foaf/0.1/firstname/", "object": { "value" : "50", "datatype" : "http://www.w3.org/2001/XMLSchema#integer"}, "foo":"bar" } }

MHT file xdmp:document-filter missing from, subject, and date metadata

In some circumstances, output of xdmp:document-filter on MHT files might not include from, subject, and date metadata

Queries against any fragment in the Search API

Previous, the Search API could specify content documents or property fragments but not either as the query target in <facet-option>, <suggestion-option>, <geo-facet-option>, and <values-option>. Now, these options can specify "any" to match either content documents or property fragments.

Incorrect co-occurrences when proximity=0

If documents have nested elements with the same name, and empty elements, and those elements are targets in a co-occurrence, co-occurrences with proximity=0 may fail to report some co-occurrences that it should.

Content not returned for deployed Application Builder applications

The Application Builder deploys applications that can provide content. The relative path of the uris for such requests starts with /content/ Previous, requests for the content of deployed applications were not retrieving the content. Now, such requests retrieve the content.

mlcmd leave-cluster cleanup of “update-hosts” crontab and corresponding entry in “SDB”

“update-hosts” crontab entry and the corresponding entry in “SDB” does not get cleaned up for a host in a cluster after running mlcmd leave-cluster command on that particular host.

Query Console xdmp.arrayValues with a null inside an array arg throws TYPEERROR

JS Query ======= xdmp.arrayValues(["a", null, false]); Result ===== [javascript] TypeError: Cannot read property 'toLowerCase' of undefined

Reverse query false positive against a near query nested as only child of parent near query

If a query consists of a near query containing a single subquery, itself a near query, a reverse query may produce a false positive match to this query when run unfiltered.

mlcp -fastload does not work if a forest is failed over

If a forest is failed over (i.e. a configured replica forest is an acting master), then an MLCP job that uses the -fastload option will fail to complete. MLCP error returned will likely be ADMIN-DATABASEFORESTNOTATTACHED

MLCP throwing NullPointerException and ArrayIndexOutOfBoundsException when importing documents from a forest backup.

MLCP throwing NullPointerException and ArrayIndexOutOfBoundsException when importing documents from a forest backup.

Relevance trace incorrect for cts:or-query with "synonym" option

The relevance trace information does not match with the actual score of the document.

extraction from documents in a single response

In the Search API, query options can specify XPaths for JSON properties or XML elements to extract from documents. Previously, the content extracted from each document was always returned as a separate document. Thus, in the REST API, extracted documents could only be retrieved as a multipart/mixed response. Now, in addition, the content extracted from a document can be returned as part of a single search response. The extracted content appears in the result for the document similar to snippets.

Performance: Value queries always use positions, even when unnecessary

When values positions are enabled, value queries will perform position-sensitive index resolution always, regardless of whether positions are needed, slowing down index resolution. This does not effect the results.

HTTP specifications conformance of Multipart content

The generation of Multipart content by the /v1/eval and /v1/invoke endpoints may generate non-compliant multipart bodies. The parsing of multipart content by xdmp:multipart-decode() may not correctly parse all compliant multipart bodies.

Field-range-index values from custom-tokenization fields don't ignore "remove" characters during searches

Field-range-index values from custom-tokenization fields don't ignore "remove" characters during searches, causing search results possibly with false positives and false negatives.

mlcp COPY ArrayIndexOutOfBounds

mlcp COPY fails with ArrayIndexOutOfBounds exception when database has no forest

Using Kerberos Authentication on port 8000 may fail repeatedly with authentication errors

When using Kerberos Authentication on port 8000, for example accessing Query Console in Internet Explorer in a Windows Domain authentication with Active Directory for authentication can repeatedly present the Authorization dialog in the browser.

XDMP-TOOMANYPOSITIONS thrown when too-many-positions-error not set

Under some circumstances, XDMP-TOOMANYPOSITIONS error is thrown even though the option too-many-positions-error has not been set. This may occur in queries involving synonym queries, cached range queries, or registered queries.

REST API servers no longer enable last modified by default

MarkLogic can maintain the last modified date for a document in the properties document. A POST request to /v1/rest-apis on port 8002 creates a REST API server. Previously, such requests enabled automatic maintenance of the last modified date by default. That approach, however, creates a properties fragment for each document, which is more expensive and may not be desired. Now, the request does not enable the last modified date by default for new REST API servers. (Existing servers are unaffected.) When desired, use the Admin UI or API to enable the last modified date explicitly for new REST API servers.

ingestion of quads over REST is too slow on /v1/graphs

There is significant improvement in the speed of quads ingestion via PUT or POST on /v1/graphs.

multiple collections parameters have AND instead of OR

Note: This bug was a previous ambiguity, and fixing it has led to a change in behavior from 8 to 9. There is a docs bug for updating documentation https://bugtrack.marklogic.com/44305 I'm not sure how it should be documented here.

While importing a signed certificate, the file contained the signed certificate can file can have multiple (chained) certificates.

It is now possible to import a signed certificate signed by an intermediate certificate and the imported file can contain the signed certificate can have the root certificate and all the intermediate certificates.

Multi-request transactions in REST API

Requests in the REST API can take a parameter to execute in an open transaction, operating on documents added in the transaction and ignoring documents deleted in the transaction. Previously, the alerting, suggest, and graph explore endpoints did not support the txid parameter. These endpoints now support the parameter.

mlcp StringIndexOutOfBoundsException

StringIndexOutOfBoundsException when mlcp extracts documents with properties

Nested elements may cause element queries containing nested triple queries to produce false negatives

Under some conditions documents with nested elements with the same name may fail to match complex element queries that contain triple queries as well as other element-position-sensitive subqueries (such as element-value queries). Example: cts:element-query(xs:QName("parent"), cts:and-query(( cts:triple-range-query((), sem:iri("pred1"), "example", "=", ()), cts:element-value-query(xs:QName("child1"), "value1"), cts:element-value-query(xs:QName("child2"), "value2") )) )

ldap group name with special characters does not mapped to any roles

For external authorization, ldap group name with special characters does not mapped to any roles

Monitoring History shows zero for "anon" memory use on Linux.

The Monitoring History page incorrectly reports "0" for anonymous ('anon') memory use on linux.

MLCP duplicate documents from same row of a CSV file

When -split_input and -generate_uri are used together to ingest delimited_text files with MLCP, duplicate documents are ingested from same row of a CSV file.

MarkLogic throws XDMP-NOTXN when the transaction id actually exists.

MarkLogic throws XDMP-NOTXN when the transaction id actually exists. This happens in a multi-statement transaction session, where a previous update statement executed through xdmp:eval is rolled back.

Packaging databases with multiple sub-databases

Attempting to export a database in Configuration Manager when that database had more than one sub-database would fail. That problem has been corrected. Note that packaging does not attempt to configure sub-databases on package import.

fn:analyze-string gives incorrect results, even with the pedantic flag enabled

fn:analyze-string will pick the longest matching alternative, not the first matching alternative. This is the default behavior (for performance). Adding the 'p' (pedantic) flag should alter this behavior, but it does not.

xdmp:document-filter results garbled characters

- Document-filter() for message/rfc822 email document garbles characters on To: and CC: headers for non unicode documents - Multi Byte characters being extracted by xdmp:document-filter from XML file gets garbled

REST: Creating a database without denoting a schema database

To create a database without a schema database omit the schema-database property.

Omitting "Fields" on package import caused an import problem

When importing a package, unchecking a property is supposed to cause it to be ignored during import. In the special case of the "Fields" property, unchecking it had the effect of deleting all fields on the database. This error has been corrected.

Create forests without waiting for them to mount

To create a forest without waiting for acknowledge it has mounted use the REST Management API by POST on manage/v2/forests?wait-for-forest-to-mount=false

mlcp leaks resource if killed through ctrl-C.

mlcp leaks resource if killed through ctrl-C. Now if running in local mode, Ctrl+C kills the job and cleans up all resources. If running in distributed mode, Ctrl+C prints a WARN message with the current job state.

SPARQL union server crash

Use of some SPARQL union clauses or alternative property path operators ("|") may cause MarkLogic Server to crash.

mlcp copy/export doesn't copy/export property fragments correctly

Using mlcp to copy/export doesn't always copy/export permissions, collections, and quality on property fragments

XCC does not return ELB cookies

XCC does not return cookies managed by AWS Elastic Load Balancer so that session affinity doesn't work.

"Undefined namespace prefix" defining a path range index

"Undefined namespace prefix" error when defining a path range index on 'node("foo:bar")'. With 'node("foo:bar")', "foo" is still seen as a namespace prefix. It should be treated as part of a plain string instead.

incorrect JSON reading preferred matches or elements in query options

The REST API can persist query options in the modules database for the application server. Previously, when reading persisted query options that specified preferred matches or preferred elements for transform-results as JSON, only a single JSON property or XML element was returned even if the query options had multiple elements or properties. Now, query options with multiple JSON properties or XML elements in preferred matches or preferred elements can be correctly as JSON.

Highlighting whole document queries in positional context may fail

Highlighting involving a query that involves a query that references the whole document (such as cts:collection-query) in an intrinsically positional context (such as cts:near-query or cts:not-in-query) may fail to show some matches.

MLCP ingest of ntriples may result in many deleted fragments

MLCP ingest of ntriples creates many deleted fragments in one forest

Intermittent 2-way SSL on XCC timeout

Intermittent and infrequent XCC connection hang / timeout when using 2-way SSL. MarkLogic Server is waiting on Client and it appears the client is waiting on on the server. Problem occurs immediately following the SSL accept being completed on the server side.

Review logs from all hosts

Review and search logs from all hosts from REST Management API manage/v2/logs endpoint.

xdmp:user-last-login takes user id but returns current user

xdmp:user-last-login has changed to take no parameter. It returns information for the current user.

admin:database-reorder-forests performance

admin:database-reorder-forests can take up to a few minutes when the database has hundreds of forests.

Replica forest 'async replicating' post flash-backup

If document inserts or updates attempted while master forest is in flash backup mode, the replica forest may get stuck in 'async replicating' state. Restarting forest will not resolve forest state and instead takes the forest into 'wait replication' state.

Hung Query with update or delete transaction, xdmp:set-session-field() and triggers

A race condition may occur when an update or delete transaction contains a call to xdmp:set-session-field() and there are content processing triggers involved in the transaction. This may result in a hung query, requiring MarkLogic Server to be restarted in order to clear. For example: CPF configured on /space/workspace, the following will lead to a race condition let $doc-uri := "/space/workspace/To-Kill-a-Mockingbird.xml" let $_ := xdmp:set-session-field("last-login-updated", fn:true()) return (xdmp:document-delete($doc-uri), fn:concat($doc-uri," deleted"))

mlcp reports Exceptions on disabled meta data

mlcp copy/export with the following options '-copy_properties="false" -copy_collections="false" -copy_permissions="false" -copy_quality="false"' (disable all meta data) results in NullPointerException.

Document URIs with semicolons for bulk write in the REST API

The REST API supports bulk write requests providing documents as a multipart/mixed payload. Each part specifies the database URI for the document to be written using a Content-Disposition header. Previously, the REST API threw an error when the URI contained a semicolon character. Now, the document uri can contain a semicolon in a quoted Content-Disposition header.

MLCP error message for importing RDF

Improved error message for MLCP importing RDF with syntax error

Performance: Rooted paths with a wildcard followed by an attribute step result in excess filtering for fragmented documents

A path of the form /*/@attr will filter all fragments, not just the fragment containing the root element.

Deleting a user with admin role fails with SEC-LASTADMIN

Delete a user assigned the admin role indirectly fails with the error: "SEC-LASTADMIN: (err:FOER0000) Cannot delete last admin user". This problem happens when there is only one admin user and a secondary user that has inherited the admin role.

Server crashes when executing JavaScript using let in case blocks.

MarkLogic Server crashes when executing JavaScript program using let keyword in case blocks.

Segmentation fault running some SPARQL query with EXISTS clause

Segmentation fault running some SPARQL query with EXISTS clause

Memory management performance

Memory management performance improvement for highly concurrent systems

Complex reverse queries during heavy insert load may cause server hang

This bug may occur when running an and- or or-query that involves multiple reverse-queries, while simultaneously attempting to insert, update, or reindex documents that contain serialized queries. In this situation, it is possible that the server will hang and neither the reverse-query nor the insert will succeed. Once in this state the reverse index is deadlocked and no inserts or reverse queries can proceed. The server must be restarted to resolve the deadlock.

default forests for modules database of REST API

The modules database typically only contains enough resources to require a single forest. Previously, the /v1/rest-apis endpoint on port 8002 initialized a modules database with one forest on each host. Now, the endpoint initializes the modules database with a single forest on the host receiving the request. If it is necessary to create additional forests for the modules database, use the Admin UI or API.

rest-extension-user role required to read query options

The rest-extension-user role is required to execute extensions in the modules database. Previously, a user also had to have the rest-extension-user role or a role inheriting the rest-extension-user role (such as the rest-reader role) to read query options persisted in the modules database using the REST API including when querying with persisted query options . Now, a role that has the rest-reader privilege is sufficient to read query options (consistent with other rest-reader operations).

Extracting json doc with Japanese content using mlcp

Exception occurs while extracting json doc with Japanese content using mlcp

mlcp leaks resource if killed through ctrl-C.

mlcp leaks resource if killed through ctrl-C. Now if running in local mode, Ctrl+C kills the job and cleans up all resources. If running in distributed mode, Ctrl+C prints a WARN message with the current job state.

Embedded triples in JSON not reindexed

Embedded triples in a JSON document are not reindexed if "triple index" was turned on after the insert.

cts:element-word-query with more than two element names may miss results

A cts:element-word-query with more than two element element names may incorrectly miss results.

Importing signed Certificate not matching private key of template should generate Error on Admin GUI

Importing signed Certificate in incorrect template, which does not match the existing private keys, does Not Generate any Error on Admin GUI and fails silently.

Monitoring cache status using REST Management API host status view

Retrieve cache status information using the REST Management API host status view (manage/v2/hosts/{id|name}?view=status).

Connection timeout, mlcp resources leak

mlcp leaks resources when connection times out and thread_count_per_split > 1.

Update does not rebalance documents correctly with respect to range assignment policies

With the range assignment policy, when the partition key in a document is updated, the rebalancer doesn't move the document based on the new value of the key until the deleted fragment is merged away.

Metadata text from .eml garbled by xdmp:document-filter

Metadata text from .eml file gets garbled by xdmp:document-filter

application/x-turtle content type on turtle ingest over REST

Cannot use application/x-turtle content type on turtle ingest over REST

Wildcard search does not take into account field specific index settings.

Wildcard search only considers database index settings and ignores field specific index settings such as three-character indexes and trailing wildcard indexes.

xdmp.validate in JavaScript does not return anything

xdmp.validate should return a report of any validation errors, but it does not.

.eml characters garbled by xdmp:document-filter

In some circumstances, xdmp:document-filter will produce incorrect results for .eml files