Overview

Corticon provides analytics capabilities that let you capture the details of rule execution, thus enabling auditing of individual decision service executions and analysis of decision service behavior. Corticon uses an extension point to where you specify an optional Analytics Handler to perform the persistence of rule trace analytics. Corticon provides two reference examples of an Analytics Handler that persist data to the Corticon log or to a SQL database. You can use these examples as-is, or customize them to meet your needs.

With analytics, you can answer questions about processed rules, such as:

  • A claim was denied. What rules triggered during the processing of that claim and what actions did they take?

  • What was the frequency of execution for each rule over the past month for a decision service?

Corticon analytics is designed to give you the flexibility to store analytic data in your preferred database or format. This allows you to use it for standalone analysis of your decision services, or to integrate it with other data for custom analysis.Are there any rules in a decision service that have not triggered over the past 90 days?

About the Analytics Handler

An Analytics Handler is a Java class packaged in a JAR file that performs the persistence of rule trace analytics data. When configured to use an Analytics Handler, Corticon calls the Handler after every decision service execution. The Handler can persist the rule trace data for the execution how it chooses. The Handler is executed in a background thread. This allows the extra processing to not block the original execution thread from returning to the client, thereby not introducing latency in the return to the caller.

Levels of analytics handling

Analytics can be as simple as a JAR attached to a project plus two brms.properties lines that enable a Ruletest run with Rule Trace to add result to the server log. That is the process that will be presented to acquaint you with a successful analytics process.

Building on that sample, you will see how you can:
  • Create a custom analytics handler and control the data in your analytics handler
  • Decide what to persist
  • Channel the results into a database

How to Activate an Analytics Handler

Once you have added CcAnalyticsHandlerSamples.jar or CcAnalyticsHandler.jar to a project, running rule tests in Corticon Studio or packaging rules for deployment, will include the Analytics Handler as part of the decision service. This is the same process used for adding custom service callouts or extended operators to a rule project.

Note: Alternatively, the Corticon classpath can be modified to include the Analytics Handler and any dependent JARs. If taking this approach, it must be done for both Corticon Studio and Corticon Server.

The presence of an Analytics Handler on the Corticon classpath does not automatically cause it to be used. You must also set properties in your brms.properties file for both Corticon Studio and Corticon Server, identifying the handler to be used and enabling it.

Properties:

com.corticon.server.analytics.handler.enabled=<true or false (defaults to false)>
com.corticon.server.analytics.handler.class=<fully qualified Handler class>

Example:

com.corticon.server.analytics.handler.enabled=true
com.corticon.server.analytics.handler.class=com.corticon.analytics.samples.CcPersistToLogger
Note: When the brms.properties file is updated, and Studio or Server need to be restarted before properties take effect.

Set up and run the Sample Analytics Handler in Corticon Studio

  1. Connect to the repository at https://github.com/corticon/server-analytics/tree/main
  2. On the repository's page, click Code, and then click Download ZIP.
  3. Expand the downloaded ZIP file into a staging area on your local machine.
  4. In Corticon Studio, choose File > New > Rule Project, and give it your preferred name, such as AnalyticsHandlerSample.
  5. In the file system, navigate to server-analytics-main\PersistenceSamples\Sample\Rule Assets\CHR Calculations. Drag all the files there into the root of the Studio project you created.
  6. In Corticon Studio:
    1. Right-click on the project, and then choose Properties.
    2. Choose Corticon Extensions.
    3. Choose Add. Navigate to server-analytics-main/PersistenceSamples/Sample/PersistToLog/jar folder, and then choose CcPersistToLogger.jar. Click Open.
    4. Click Apply and Close.


    Note: It is good practice to now Upgrade Rule Assets, and then Validate Project.
  7. In the Studio's work directory, edit the brms.properties file to add two lines: 
    com.corticon.server.analytics.handler.enabled=true
com.corticon.server.analytics.handler.class=com.corticon.analytics.samples.CcPersistToLogger
  8. Restart Studio.
  9. In Studio, open the project file Tester.ert, and then choose Run Test with Rule Trace.
  10. Examine the log in the Studio Work directory.
Note: The required actions for this basic example were adding the JAR file to the project, adding the analytics.handler lines to the brms.properties file, and then running a Ruletest with Rule Trace. Try it with one of your projects.

How to Create a Custom Analytics Handler in the Development Environment

To implement a custom Analytics Handler, you need to navigate to the staging area's PersistenceSamples/Development/Dependent Jars folder, and then add CcAnalyticsHandler.jar to your development classpath.

Corticon Analytics has dependency on two third party libs. The libraries and their version are JSON v20120521 and JDOM2 v2.0.6.1. Once you obtain them, put them on your classpath.

Implementing your Custom Analytics Handler

Once your Development Environment is properly setup, you will have access to the com.corticon.analytics.CcAnalyticsHandler class that your custom handler will extend.



By extending from this class, you will need to implement the abstract method processAnalytics(). This is the method that the Corticon Server will call inside your custom Analytics Handler after each execution. This is where all your custom post-processing code will be written.



Methods Available to your Custom Analytics Handler

The following inherited public methods can be called by your custom handler. All of the data will be pre-populated by Corticon after each decision service execution.

//  Decision Service Name
public String getDecisionServiceName();
			
//  Major Version Number for the deployed Decision Service
public int getMajorVersionNumber();
			
//  Minor Version Number for the deployed Decision Service
public int getMinorVersionNumber();
			
// Input payload - org.json.JSONObject or org.jdom.Document depending on payload type
public Object getInput();
			
//  Output results - org.json.JSONObject or org.jdom.Document depending on payload type
public Object getOutput();

//  List of the CcAnalyticsHandlerRuleCatalog - Object representations of super set of Rules inside the Decision Service
public List<AnalyticsHandlerRuleCatalog> getRuleCatalog();
				
//  List of the CcAnalyticsHandlerRuleMessage - Object representation of all the Rule Messages posted during this execution
public List<AnalyticsHandlerRuleMessage> getRuleMessges();
					
// List of the Metrics
public List<AnalyticsHandlerMetric> getEntityChanges();
public List<AnalyticsHandlerMetric> getAttributeChanges();
public List<AnalyticsHandlerMetric> getAssociationChanges();
								
// Execution Start Times
public long getExecutionStartTime();
								
//  Execution Total Time
public long getExecutionTotalTime();
The API for these classes listed are defined in Corticon Server’s Javadoc.

Source code for the Analytics Handler

You can review and adapt the analytics handlers to suit your needs:
  • Persist to database: Navigate in your staging area to Sample/PersistToDatabase/src/com/corticon/analytics/samples, and then edit CcPersistToDatabase.java to refine your database design.
  • Persist to log: Navigate in your staging area to Sample/PersistTolog/src/com/corticon/analytics/samples, and then edit CcPersistToLogger.java to refine your logger design.

How to channel the analytics results into a database

You can add the results of each execution into a database. You need to add the handler JAR to the project, and update brms.properties on both the Studio and any Servers that will host the decision services:
com.corticon.server.analytics.handler.enabled=true
com.corticon.server.analytics.handler.class=com.corticon.analytics.samples.CcPersistToDatabase
Once the brms.properties file is updated, the Studio or Server needs to be restarted before the properties take effect.

Set up the database

The sample resources at your staging area's Sample\PersistToDatabase\Database Scripts location providesSQL Server/CreateTablesWithForeignKeys.sql to create SQL Server tables to support the sample.

What should an Analytics Handler persist?

What you choose to persist in a custom Analytics Handler is dependent on your specific needs. Following are recommended scenarios:

  • To enable auditing of rule executions, you want to persist details about each individual execution. This typically includes getEntityChanges, getAttributeChanges, and getAssociationChanges. If this is too much data you can choose to persist just a subset of the data, for example only specific attribute changes.

  • To enable analysis of the before and after payloads, you want to persist getInput, getOutput, and possibly getRuleMessges. The format of this data will be either XML or JSON, depending on the input payload.

  • To enable analysis of execution times, you want to persist when the execution started (getExecutionStartTime) and how long the execution took to execute (getExecutionTotalTime).

  • To enable analysis of rule coverage, specifically including rules not executed, you need to persist getRuleCatalog. This will give you the definition of all rules in a decision service. You only need to persist this information once per decision service.

  • In all cases you will also want to persist getDecisionServiceName, getMajorVersionNumberand getMinorVersionNumber. This will form the catalog of the decision services deployed. If storing data in a normalized SQL database, this data would likely form the key for a decision service and be used as a foreign key in other tables to tie them to the decision service.

Queries for the persisted data

The sample resources at your staging area's Sample\PersistToDatabase\Database Queries location provide the format of several queries to add to the database tables to find:

  1. Which Rules fired for each execution?
  2. Which Rules didn't fire for each execution?
  3. Which Rules fired for this Decision Service regardless of execution?
  4. Which Rules never fired for this Decision Service for all executions?
  5. Compare how many times a Rule fired in an execution to how many executions were fired?
  6. Which executions and their input payloads didn't fire a specific Rule?
  7. Which executions and their input payloads didn't fire a specific Rulesheet?

You can tune what is persisted, and how that data is queried and presented to suit your needs.