Endpoints and Request Monitoring

The Request Monitoring feature enables you to configure logging of information related to requests, including metrics collected during request execution. This feature lets you enable logging of internal preset metrics for requests on specific endpoints. You can also log custom request data by calling the provided Request Logging APIs. This logged information may help you evaluate server performance.

This chapter provides an overview of creating endpoint declarations and using them to monitor requests on MarkLogic Server. This chapter includes the following sections:

• Monitoring Requests
• App Server Request Monitoring
• XDBC Server Request Monitoring
• Task Server Monitoring
• Creating Endpoint Declarations
• Request Cancelling
• Query Console Monitoring
• Request Monitoring APIs

Monitoring Requests

Using the Request Monitoring feature, you can switch on logging of internal preset metrics for requests on specific endpoints, or you can choose to log additional custom request data by calling the request logging APIs. The custom request data might contain a query plan, traces, or whatever information you want to collect and log for a request. This logged information may help you spot offending requests and evaluate request history.

App Server Request Monitoring

You can trigger request logging for an App Server through one or more of the following options:

• For targeted endpoints (main modules), by switching on monitoring in their endpoint declaration.

• For all requests on the App Server, by using a special server declaration.

• By calling request logging APIs in modules.

To switch on monitoring for an endpoint, you must add a monitoring section to the App Server Endpoint Declaration file and specify which metrics will be logged. Request monitoring is switched off by default for all metrics.

XDBC Server Request Monitoring

XDBC Server enables XCC and XDBC applications to communicate with MarkLogic Server. You can configure request monitoring for XDBC requests for specific endpoints and globally for the XDBC Server. There are two types of XDBC requests where request monitoring are enabled:

• XDBC Invoke Requests
• XDBC Eval Requests

XDBC Invoke Requests

You can enable request monitoring at both the endpoint level and at the server level for XDBC invoke request. For a specific endpoint, you must add a monitoring section to the XDBC Server Endpoint Declaration file and specify which metrics will be logged. To configure monitoring on a global level, you must add a default.api file in the modules root directory for the XDBC Server. For the Task Server, as there is no port number, the output request log file will be logged into a new type of log file called TaskServer_RequestLog.txt

XDBC Eval Requests

For XDBC eval requests, request monitoring is only available at the server level, as there are no real endpoints. To separate the monitoring configuration between the invoke and eval request, and to add more control over the monitoring of the eval request, you can add an eval.api file to the module root in addition to the default.api file. The eval.api file has the same format as the default.api file, which contains only a monitoring section, but the settings in eval.api override those in default.api.

Task Server Monitoring

The Task Server processes requests that have been spawned from xdmp:spawn() or from a post-commit trigger action. Since each task is handled as a module, you can configure request monitoring at either an endpoint level or a global server level.

To configure request monitoring for a specific endpoint, when scheduling the task, add a monitoring section to the Task Server Endpoint Declaration file (*.api) in the task's Task Root, and specify which metrics are to be logged.

For monitoring on a global server level (monitoring all tasks with the same task root), add a monitoring section to the default.api file in the Task Root for the Task Server.

The configuration in a specific endpoint declaration file overrides the settings in the global server default.api.

Creating Endpoint Declarations

An Endpoint Declaration is a JSON file with the extension .api that resides in the module database or file directory of an HTTP server. The App Server uses the declarations in this file to dispatch requests to corresponding main modules. The declarations in this file also determine which requests are to be logged.

The Endpoint Declaration File

The name, parameters, and return value for each endpoint are declared in the *.api file. The *.api file contains a JSON data structure with the following properties:

The following is a list of meters that can be logged with the parameters that control them:

The following parameters may be included in a *.api file:

Constraints on Meters

To control the number of meters that are logged, you can put the following constraints on meters:

The declaration format of a constraint is:

meter_name : {"operator":value, "operator":value, ...}

For example:

"constraints": {
  "tripleCacheHits" : { "ge":1 }
}

In this example, tripleCacheHits is logged only if the the value of tripleCacheHits is >= 1.

Meters with zero or empty values are not normally logged. This is done to minimize the size of the Request Log file. To log a zero or empty value, use the following code:

"constraints": {
  "meter_name" : { "ge":0 }
}

The default constraint value on any meter is:

gt:0

Controlling Request Logging Using Thresholds

You can add thresholds to specify that a request is logged only when the threshold conditions are satisfied. To declare a threshold, create a thresholds section in your *.api file as follows:

{
  "monitoring":{
   "thresholds" : {
    "elapsedTime": { "gt": 1.0 }
   }
  }
 }

To disable threshold checks on custom meters so the meters can always be logged, set the boolean flag excludeCustom to true in the thresholds section:

{
  "monitoring":{
   "thresholds" : {
    "excludeCustom": true,
     "elapsedTime": { "gt": 1.0 }
   }
  }
 }

In this example, the general meters for the request are logged only when the total runtime of the request is greater than one second:

{
  "monitoring":{
    "thresholds" : {
      "elapsedTime": { "gt": 1.0 }
    },
   "general":{
     "enabled": true
    }
  }
}

The following operators are allowed in thresholds:

Enabling Request Monitoring

Request monitoring is enabled by default on the following default MarkLogic application servers:

• The default App-Services application server, normally Port 8000:

  • For all requests related to the Query Console.
  • For requests with runtime exceeding one second that are not related to the Query Console. This mainly covers the REST Client API.

• The default Manage application server, normally Port 8002, for requests running longer than one second. This covers the Configuration Manager and Monitoring Dashboard.

If request monitoring is not already enabled, you can enable request monitoring by calling any server-side JavaScript (*.sjs) or XQuery (*.xqy) functions in files that reside in the modules directory as declared on the HTTP server via the Admin interface, and you can create other *.api files in the same directory or, if using the file system, in the same subdirectory as the JavaScript or XQuery file being called. For more information on configuring HTTP servers and the modules directory, see the HTTP Servers chapter in the Administrator's Guide.

The following example enables request monitoring:

{
"monitoring":{
  "general":{
    "enabled":true
  },
  "custom":true
}
}

The following example logs tripleCacheHits if the the value of tripleCacheHits is >= 0:

"constraints": {
     "tripleCacheHits" : { "ge":0 }
    }

The following code fragment logs up to 10 documents, 10 fragments, and 4 hosts:

"documents": 10,
"fragments" : 10,
"hosts" : 4

The following example is called countdocs.api. The function getCount is defined in a file called countdocs.sjs that resides in the same directory. The general object under the monitoring section of the file has enable and custom set to true; this enables request logging.

{
  "functionName":"getCount",
  "params":[
    {
      "name":"collection",
      "datatype":"string",
      "multiple":false,
      "nullable":false
    },
    {
      "name":"method",
      "datatype":"int",
      "multiple":false,
      "nullable":false
    }

  ],
  "return":{
    "datatype":"string",
    "nullable":false,
    "multiple":false
  },

  "monitoring":{
    "general": {
      "enabled": true,
      "constraints": {
        "tripleCacheHits" : { "ge":1 }
      }
    },
    "custom": true ,
    "documents": 10,
    "fragments" : 0,
    "hosts" : 4
  }
}

The Default Declaration File

You can configure request monitoring globally for all requests on an App Server by adding a default.api file where the modules root is configured for the App Server (either to the modules database or to the file system). The default.api is a JSON file that only contains a monitoring section.

The following is a sample default.api file:

{
"monitoring":{
    "general":  {
      "enabled": true,
      "constraints": {
        "tripleCacheHits" : { "ge":0 }
      }
    },
    "custom": true,
    "documents": 10,
    "fragments" : 10,
    "hosts" : 5  }
}

When you make a request to an endpoint and do not specify an App Server Endpoint Declaration file, the default.api file in the module database or file directory is used if it exists. If a module has a *.api file associated with it, the monitoring settings in the *.api file for that module are used instead of those in the default.api file.

If you want to save the default.api file, use JavaScript:

declareUpdate();
xdmp.documentInsert("/default.api",{ 
 "monitoring":{
  "general": {
   "enabled": true
   },
  "custom": true,
  "documents": 10,
  "fragments" : 10,
  "hosts" : 5 }
}); 

Request Logs

The logs for MarkLogic Server containing information about the requests you have chosen to log are stored in the Logs directory under the MarkLogic Server data directory (typically c:\Program Files\MarkLogic\Data\Logs on Windows, /var/opt/MarkLogic/Logs on UNIX-based systems) in the file port_no_RequestLog.txt. Each RequestLog.txt file will contain the meters from multiple monitored endpoints that:

• are configured on an App Server with some monitoring switched on.

• have calls to xdmp:request-log-put in their module.

The collected data is logged in JSON format at the rate of one line per request information. Even if monitoring is completely switched off on an endpoint, calls to xdmp:request-log-put(key,value) during a request will result in data being logged for all the (key,value) pairs that have been stored during the request.

The following is an example of a request log where different meters are logged for two endpoints:

{"time":"2018-11-09T14:07:14-08:00", "url":"/booleanApiDecl.sjs?booleanArg=true", "user":"admin", "result":10, "elapsedTime":0.007145, "requests":1, "dbProgramCacheMisses":1, "dbMainModuleSequenceCacheMisses":1, "dbLibraryModuleCacheMisses":0}
{"time":"2018-11-09T14:10:18-08:00", "url":"/booleanApiDecl.xqy?booleanArg=true", "user":"admin", "result":10, "elapsedTime":0.001603, "requests":1, "dbProgramCacheMisses":1, "dbMainModuleSequenceCacheMisses":1, "dbLibraryModuleCacheMisses":0}

Request Cancelling

This section describes the procedure to setup and enable request cancelling for an endpoint, a main module, or globally on an App Server or an XDBC Server. Request cancelling is disabled by default for all meters. You can enable request cancelling by adding a limits section to the monitoring section in the endpoint declaration, as in the following example:

{
  "monitoring": {
    "limits" : {
      "lockCount" : 100,
      "readSize" : 1000000 
    }
  }
}

The following limits are available:

You can update the limits configuration while the server is running without having to restart the server.

Query Console Monitoring

By default, request monitoring is enabled for all requests related to Query Console normally running on port 8000 of the App-Services application server. Internally, the Query Console uses an .api configuration file equivalent to:

{
  "monitoring": {
   "general": {
    "enabled": true
   }
  }
}

Starting with MarkLogic 10.0-8, request cancellation (and monitoring) can be configured globally by users for Query Console requests by inserting a qconsole.api document at the root of the App-Services server's Modules database. Since, the root of the App-Services server is normally configured to / , the full URI for the qconsole.api document would be /qconsole.api.

The configuration in the qconsole.api replaces the defaults for both request monitoring and cancellation. No attempt is made to merge the default request monitoring configuration with a request cancellation configuration set by the user. In order to keep monitoring enabled for requests on the query console, the user must set monitoring/general/enabled to true in qconsole.api.

Configuration example:

To enable request cancellation with elapsed time limit of 3 seconds and to keep request monitoring enabled, the following qconsole.api document should be inserted into the Modules database with a URI of /qconsole.api:

{
 "monitoring": {
  "general": {
   "enabled": true
  },
  "limits" : {
   "elapsedTime" : 3
  }
 }
}

Request Monitoring APIs

The Request Monitoring APIs are XQuery and JavaScript functions enable you to log additional information in request logs. The following are the request monitoring functions available:

• xdmp:request-log-put

• xdmp:request-log-get

• xdmp:request-log-delete

• xdmp:set-request-limit

• xdmp.requestLogPut

• xdmp.requestLogGet

• xdmp.requestLogDelete

• xdmp:request-status

Whatever is logged with xdmp:request-log-put displays in the log files unless the custom flag is set to false. When the custom flag is set to false, all custom logging is muted.

For more details about syntax and usage of these functions, see the AppServer functions in the MarkLogic XQuery and XSLT Function Reference and the MarkLogic JavaScript Reference Guide.