XQuery Dialects in MarkLogic Server

The XQuery specification is a formal recommendation from the W3C XQuery Working Group. MarkLogic 10 implements the W3C XQuery 1.0 Recommendation (http://www.w3.org/TR/xquery/). To maximize compatibility with MarkLogic Server and to offer strict XQuery compliance to those who desire it, as well as to include extensions to the language to make it easier to build applications, MarkLogic Server supports two dialects of XQuery. This chapter describes these dialects, and includes the following sections:

• Overview of the XQuery Dialects
• Rules For Combining the Dialects
• Using a Non-Default Dialect in XSLT (xdmp:dialect)
• Strategies For Migrating Code to Enhanced Dialect

Overview of the XQuery Dialects

MarkLogic Server supports the following dialects of XQuery:

• MarkLogic Server Enhanced (XQuery 1.0-ml)
• Strict (XQuery 1.0)

You can use library modules from different dialects together, as described in Rules For Combining the Dialects. Each dialect has a different set of pre-defined namespaces, as described in Predefined Namespace Prefixes for Each Dialectpage/topics/namespaces.html#id_21104).

MarkLogic Server Enhanced (XQuery 1.0-ml)

For a module to use the MarkLogic Server enhanced dialect, use the following for the XQuery version declaration on the first line of the XQuery module:

xquery version "1.0-ml";

Note the semi-colon at the end of the declaration, which is required in 1.0-ml. The enhanced dialect has the XQuery 1.0 syntax and also includes various extensions to the language such as try/catch. This dialect is the default for new App Servers, and is considered the preferred dialect for new applications. For more details on the enhanced 1.0-ml dialect, see MarkLogic Server Enhanced XQuery Languagepage/topics/enhanced.html#).

Strict (XQuery 1.0)

For a module to use the MarkLogic Server strict dialect (1.0), use the following for the XQuery version declaration on the first line of the XQuery module:

xquery version "1.0";

Note the semi-colon at the end of the declaration, which is required in 1.0. The strict mode is for compatibility with other XQuery 1.0 processors; if you write a library in 1.0, you can use it with MarkLogic Server and you can also use it with other conforming processors. Similarly, you can use modules that are written in standard XQuery with MarkLogic Server.

To use the MarkLogic Server built-in functions in 1.0, you must bind a prefix (for example, xdmp) to the namespace for the MarkLogic Server functions; there is no need to import a library for these built-in functions, but you do need to bind the namespace to a prefix. To use the xdmp functions in 1.0, add prolog entries for the namespace bindings you are using in your query, as in the following example:

xquery version "1.0";
declare namespace xdmp = "http://marklogic.com/xdmp";

xdmp:version()

Rules For Combining the Dialects

MarkLogic Server has a very flexible way of combining the different XQuery dialects. You can import a library module written in any of the three dialects into any main or library module. For example, you might find an open source standards-compliant module that you found on the internet which is written in the strict XQuery 1.0 dialect. You can then import this module into any MarkLogic Server XQuery program, regardless of dialect, and then use those functions in your code.

When writing modules of different dialects, the best practice is to always use the XQuery version declaration as the first line, indicating which dialect the module is written in. That way, if the module is written in a different dialect than the default dialect for the App Server or the program, it will still work correctly (for details, see Inheriting the Default XQuery Version From the App Server).

Using a Non-Default Dialect in XSLT (xdmp:dialect)

You can use the xdmp:dialect attribute to specify which dialect expressions are evaluated in an XSLT stylesheet. For details, see xdmp:dialect Attributepage/topics/xslt.html#id_35081).

Strategies For Migrating Code to Enhanced Dialect

If you are writing new XQuery code, the best practice is to use the 1.0-ml dialect. If you are updating code that was written in previous versions of MarkLogic Server, migrate that code to 1.0-ml. This section describes things to think about when migrating your application code and includes the following parts:

• When To Migrate XQuery Code
• XQuery Changes from 0.9-ml to 1.0-ml
• Inheriting the Default XQuery Version From the App Server
• Porting 0.9-ml XQuery Code to Enhanced 1.0-ml

When To Migrate XQuery Code

XQuery 0.9-ml has been deprecated and will be removed in a future release. If you have code written in XQuery dialect 0.9-ml, you need to migrate your XQuery code to either enhanced XQuery 1.0-ml or strict XQuery 1.0. The differences between the dialects are mostly syntax changes in the prolog, but there are also some other differences that might cause subtle changes in behavior. For details on the differences between the XQuery dialects in 0.9-ml and 1.0-ml, see XQuery Changes from 0.9-ml to 1.0-ml. When you migrate XQuery code to 1.0-ml (or to 1.0), there are several ways you can go about it:

• Migrate an entire application all at once. This method gets everything over with at once, and therefore focuses the effort. If you have a relatively small amount of code to migrate, it might make sense to just go ahead and migrate it all at once.

• Migrate one module at a time. This method allows you to spread the migration work over a number of small tasks instead of one large task, and further allows you to test each module independently after migration. This technique is very flexible, as you can do a little bit at a time. A good first step for this one-by-one approach is to start by adding an XQuery 0.9-ml declaration to the first line of each XQuery file. Then, as you migrate a module, you can change the declaration to 1.0-ml and make any needed syntax changes to that module.

XQuery Changes from 0.9-ml to 1.0-ml

While MarkLogic Server currently allows 0.9-ml code to run without changes, the dialect is deprecated and will be removed in a future release. The new XQuery dialect 1.0-ml has enhancements. Because you can mix modules in the old dialect with modules in the new, you can perform your migration one module at a time. This section highlights the major syntax and semantic changes between the XQuery used in 0.9-ml and enhanced XQuery dialect 1.0-ml. The changes fall into the following categories:

• Syntax Changes from XQuery 0.9-ml to XQuery 1.0-ml
• Semantic Changes from XQuery 0.9-ml to XQuery 1.0-ml
• Changes to Built-in Definitions

Syntax Changes from XQuery 0.9-ml to XQuery 1.0-ml

The following syntax has changed from XQuery 0.9-ml to XQuery 1.0-ml:

• Semi-colons ( ; ) are now required at the end of each prolog declaration.

• Prolog declarations that previously used define now use declare.
• Variable declaration syntax is slightly different, and now uses the := syntax (for details and an example, see Declaring Variablespage/topics/langoverview.html#id_52597)).

Semantic Changes from XQuery 0.9-ml to XQuery 1.0-ml

The following semantic changes have occurred from XQuery 0.9-ml to XQuery 1.0-ml:

• If you are modifying a main module and it has function declarations that are used in the same module, they must be declared in a namespace. Library module declarations now require the namespace keyword and a prefix for the namespace, for example:

  Copy

  module namespace my = "my-namespace";
  

  Important:
  
  Functions in a main module must be put into the local: namespace.

• Function declarations that return the empty sequence now require the empty sequence to be specified as follows:

  Copy

  empty-sequence()
  

  The 0.9-ml dialect had you specify empty() for the empty sequence.

• Some of the effective boolean value rules have changed. Notably, the following returns true in 0.9-ml and returns false in 1.0-ml (and throws an exception in 1.0):

  Copy

  (: returns true in 0.9-ml, false in 1.0-ml, and
     throws XDMP-EFFBOOLVALUE in 1.0 :)
  fn:boolean((fn:false(), fn:false()))
  

  This change might affect applications that have if/then/else statements where the if test returns a sequence of boolean values. In these cases, you might see the if statement evaluating to false in cases where it previously evaluated to true, causing the else statement to be evaluated instead of the then statement.

• String functions treat empty arguments as equivalent. For example, the following statements, while not equivalent, are treated the same:

  Copy

  substring((), 1);
  substring("",1)
  

  In the first line, () is parsed identically to "" in the second line.

  The functions affected by this are:

  fn:starts-with	fn:ends-with	fn:contains	fn:substring
  fn:substring-before	fn:substring-after	fn:string-length	fn:upper-case
  fn:lower-case	fn:normalize-space	fn:normalize-unicode	fn:translate
  fn:tokenize	fn:replace	fn:matches

• Constraints on timezone min and max offsets are now enforced.

• Range expressions in which the lower bound is greater than the upper bound are no longer treated as a range. For example, [5 to 1] is now evaluated as ().

• The namespace used for durations now uses the xs namespace prefix; previously it used the xdt prefix. Any code you have that uses the xdt namespace prefix will require a change to the xs prefix. For example, if you have code that uses xdt:dayTimeDuration, change it to xs:dayTimeDuration.

• element() tests in 0.9-ml are equivalent to schema-element() test in 1.0 and 1.0-ml. Any code you have with element() tests might not match some elements that previously matched. For example, substitution elements previously would match the base element name, but will now only match with schema-element() test in 1.0 and 1.0-ml.

• XQuery 1.0-ml has has function mapping enabled by default.

• Some changes to the XQuery standard functions. For example, there are subtle changes to fn:avg and fn:sum, fn:error has a different signature, and fn:node-kind does not exist in 1.0 and 1.0-ml (it is replaced by xdmp:node-kind).

Changes to Built-in Definitions

The following built-in functions have changed:

• fn:translate: The second and third arguments can no longer be empty.

• fn:error: The signature has changed. For details, see: https://www.w3.org/TR/xpath-functions/#func-error

• fn:resolve-uri: XQuery 1.0-ml has implemented this function differently. It is more likely to throw an error when it has trouble resolving a relative URI reference.

• The following function names have changed:

  XQuery 0.9-ml Function	XQuery 1.0-ml Function
  fn:get-local-name-from-QName	fn:local-name-from-QName
  fn:input	fn:doc
  fn:expanded-QName	fn:QName
  fn:node-kind	xdmp:node-kind
  fn:get-minutes-from-dayTimeDuration	fn:minutes-from-duration
  fn:get-month-from-date	fn:month-from-date
  fn:get-seconds-from-dateTime	fn:seconds-from-dateTime
  fn:get-year-from-dateTime	fn:year-from-dateTime
  fn:get-day-from-date	fn:day-from-date
  fn:get-seconds-from-dayTimeDuration	fn:seconds-from-dayTimeDuration
  fn:get-year-from-date	fn:year-from-date
  fn:get-days-from-duration	fn:days-from-duration
  fn:get-timezone-from-date	fn:timezone-from-date
  fn:get-namespace-uri-from-QName	fn:namespace-uri-from-QName
  fn:get-seconds-from-time	fn:seconds-from-time
  fn:get-minutes-from-time	fn:minutes-from-time
  fn:get-timezone-from-time	fn:timezone-from-time
  fn:get-hours-from-dateTime	fn:hours-from-dateTime
  fn:get-months-from-yearMonthDuration	fn:months-from-duration
  fn:get-day-from-dateTime	fn:day-from-dateTime
  fn:get-month-from-dateTime	fn:month-from-dateTime
  fn:get-hours-from-dayTimeDuration	fn:hours-from-duration
  fn:get-minutes-from-dateTime	fn:minutes-from-dateTime
  fn:get-years-from-yearMonthDuration	fn:years-from-duration
  fn:get-hours-from-time	fn:hours-from-time
  fn:get-timezone-from-dateTime	fn:timezone-from-dateTime
  fn:get-namespace-uri-for-prefix	fn:namespace-uri-for-prefix
  fn:get-in-scope-namespaces	fn:in-scope-prefixes

• The following functions are no longer supported:

  • fn:subtract-dateTimes-yielding-dayTimeDuration Use the minus sign operator (-).
  • fn:distinct-nodes You can write an XQuery loop to replace this function or use a similar function from the FunctX library functx:distinct-nodes. Read more at http://www.xqueryfunctions.com/xq/functx_distinct-nodes.html.
  • fn:subtract-dateTimes-yielding-yearMonthDurationUse the minus sign (-) operator to get xs:dayTimeDuration.
  • fn:string-pad You must now write an XQuery loop to replace this function.
  • fn:context-item Use . or fn:current() as appropriate.

Inheriting the Default XQuery Version From the App Server

Each App Server has a setting for the default XQuery version. Any requests against that App Server that do not have explicitly specify an XQuery version declaration are treated as the default XQuery version value. Because of the way a request inherits it default XQuery version from the App Server environment, requests without an explicit declaration can be treated differently by different App Servers (if the App Servers have different default XQuery values). Therefore, it is best practice to specify the XQuery version in each module.

The task server does not allow you to specify a default XQuery version, and if there is no explicit version declaration in the XQuery code evaluated on the task server, the default XQuery version is determined as follows:

• If you run an xdmp:spawn call, the default XQuery version is 1.0-ml.

• If a trigger action module is executed on the task server (for example, as the result of an update on a document that has a post-commit update trigger), then the default XQuery version is the default XQuery version for the App Server that triggered the update (as specified in the configuration for the App Server).

This makes it especially important to use XQuery version declarations in modules used by CPF or modules called from triggers. For details on CPF, see the Content Processing Framework Guide.

To ensure your code is always evaluated in the dialect in which you have written it, regardless of the context in which it is run, the best practice is to begin each XQuery module with a XQuery version declaration. For the syntax of the version declaration, see XQuery Version Declarationpage/topics/langoverview.html#id_95538).

Specifying the XQuery Dialect in the Prolog

You specify the dialect for an XQuery module with a version declaration. The version declaration is optional, and comes before the prolog in an XQuery module. It is best practice to put the XQuery version declaration in your code as the first line in the module, as having it there ensures it will work as expected in any environment. For example, to specify 1.0-ml as the XQuery version, begin your XQuery module with the following:

xquery version "1.0-ml";

Porting 0.9-ml XQuery Code to Enhanced 1.0-ml

In most cases, porting any XQuery code used in 0.9-ml to the 1.0-ml dialect will be straightforward. The bulk of the differences are syntax changes in the prolog. As stated earlier, you do not need to port all of your code at one time. A sensible approach is to migrate your code one XQuery module at a time. This section outlines the basic steps to follow when migrating your XQuery code.

The following are some basic steps to take when migrating 0.9-ml XQuery code to 1.0-ml:

1. Add XQuery version declarations to all of your existing modules. For code written in 0.9-ml, the declarations will be as follows:

   Copy

   xquery version "0.9-ml"
   

   Important:
   
   If you use the xquery version "0.9-ml" declaration, you will get a warning message. MarkLogic recommends that you move to either XQuery dialect 1.0-ml or 1.0 as soon as possible.

2. Review the code for any incompatibilities.

3. For each module you migrate, change the version number string in the XQuery version declaration to 1.0-ml and add a semi-colon to the line so it appears as follows:

   Copy

   xquery version "1.0-ml";
   

4. Change all of the prolog declarations to the 1.0 syntax (change define to declare, add semi-colons, and so on, as described in XQuery Changes from 0.9-ml to 1.0-ml). For the prolog syntax, see XQuery Prologpage/topics/langoverview.html#id_48211), the W3C specification (http://www.w3.org/TR/xquery/#id-grammar), or a third-party XQuery book.

5. If you are modifying a main module and it has function declarations that are used in the same module, they must be declared in a namespace. The preferred way to put functions local to a main module is to prefix those functions definitions and function calls with the local: prefix, which is predefined.

6. If you have any durations that use the xdt namespace prefix, change the prefix to xs (for example, change xdt:dayTimeDuration to xs:dayTimeDuration).

7. If you are modifying a library module that is defined with the fn namespace URI, you must change the namespace URI of that module; you cannot use the URI bound to the fn namespace prefix as the URI for a library module in 1.0 or 1.0-ml. If you do change the namespace URI of a library module, you must also change the URI in any import module statements in other modules that call the library.

8. Test the module and correct any syntax errors that occur.

9. After getting the module to run, test your code to make sure it behaves as it did before. Pay particular attention to parts of your code that might rely on boolean values that take boolean values of sequences, as those behave differently in 0.9-ml and 1.0-ml (see XQuery Changes from 0.9-ml to 1.0-ml). Check for any changes due to function mapping, which is described in Function Mappingpage/topics/enhanced.html#id_55459).

10. Repeat this process for other modules you want to migrate.