Batch processing in Corticon

Corticon Decision Services can process very large datasets efficiently by using the Corticon Advanced Data Connector and batch processing capabilities in the Corticon Web Console. Corticon Studio includes the Batch Processing sample which provides a completed project demonstrating batch processing. This tutorial dives deeper, providing you step-by-step instructions for taking an ADC enabled decision service and using it for batch processing.

Use the sample project

The project's Ruleflow illustrates the steps:

The Ruleflow operates on a set of input patient IDs. This is essential to understanding batch processing. Corticon Server’s batch processing capability will be used to retrieve “batches” of patient IDs from the database, and then ADC will efficiently retrieve the information for these patients, apply rules, and then store the results back into the database. The efficiency is realized in the minimizing of the number of database queries: retrieving the information for 1000 patients with one query is much more efficient than a thousand separate queries.

This tutorial focuses only on taking this sample and using it with batch processing. For details on ADC, see Getting Started with ADC and Advanced ADC Topics.

Install the sample

1. In Corticon Studio, select Help -> Samples.
2. Scroll to the Advanced section on the left-side of the screen. Click ADC Database Connectivity, and then click Open.
3. Choose the ADC Database Connectivity option, and then click OK.

The ADC Database Connectivity project is imported, and displayed in the Project Explorer window.

Set up the database

1. Create a database named PatientRecords in a database Corticon can access. You will need write access to this database to complete the tutorial.
2. Expand the ADC Database Connectivity project, and then locate the adc and patient SQL scripts for the appropriate database product.
3. Execute the scripts in the database management tool's editor. The scripts create ADC, Batch, Patient and Treatment tables in the PatientRecords database, and then update the tables with data.

Configure ADC for your database

1. Test your database connection in Studio by opening the project's Vocabulary, Medical.ecore, and then updating the connectivity tabs for Query and Patient Data.

   

2. In each tab, select the appropriate Database Server, and then enter its URL, and authentication values.
3. Click CONNECTION Test to validate the connection.
4. On the Vocabulary menu, select Datasource Configuration File > Export. Save the configuration file as PatientApprovalConfiguration.xml at a location that it will be accessible to the Corticon Server installation that will be servicing the batch processing.

Test the ruleflow

1. Open the sample's Ruletest, ProcedureApproval.ert.
2. Execute the No Input Data Testsheet. You should see the following output:

   
   
   

Package the decision service

1. In the Project Explorer, right-click on the ADC Database Connectivity project.
2. Choose Package and Deploy Decision Services, and then choose Package and save for later deployment. Click Next.
3. Select the ProcedureApproval ruleflow.
4. Choose a directory that will be accessible by the Server, and then click Finish.

Your Decision Service is now ready to deploy.

Deploy the Decision Service

1. Start Corticon Server.
2. Connect to Corticon Web Console from your browser using the default credentials Admin/Admin. The typical address for the Web Console is http://localhost:8850/corticon. The Web Console opens

   

3. Click Manage Decision Services, and then click Add Decision Service.
4. Select Add a single Decision Service, and then click OK.
5. In the Add Decision Service dialog, enter:
   1. Name: MyADC_Sample
   2. Description: Deploys ADC Database Connectivity sample project’s Decision Service.
   3. EDS File: navigate to the ProcedureApproval.eds file you created, and then select it.
   4. Servers: Select the server location; typically, local server.
   5. The completed Decision Service tab looks like this:

      
      
      

   6. Click the Database tab, and then choose the Datasource Configuration File you exported from Corticon Studio.
   7. Click Save and Deploy. The completed Datasource tab looks like this:

      
      
      

The Web Console displays the Decision Service.

Create a batch configuration

1. Select Batch Configuration on the left side of the Web Console.
2. Click New Batch Configuration.
3. In the New Batch Configuration dialog, enter:
   1. Enter MyADC_Batch for the Batch Configuration Name.
   2. Choose Select Decision Service, and then chooseMyADC_Sample 1.0.
   3. Choose the Datasource Patient Data.
   4. Choose the Query AllPatients.
   5. The Batch Configuration looks like this:

      
      
      

      Click Save.

The Batch Configuration panel displays the new configuration.

Run a batch job

1. On the Batch Configuration: MyADC_Batch page, click Execute to run the batch process.
2. After execution, the Process Count shows as 10 on the Batch Configuration: myADC_Batch page:

   
   
   

3. To see that the batch processing ran as expected, look in the Treatment table of the PatientRecords Database for values in the Approved column:

   
   
   

Run a Parameterized Batch Job

You can set your batch configuration with query parameters for more control. To try it out:

1. Select the MyADC_Sample 1.0 Decision Service.
2. Choose the Patient Data datasource.
3. Select PatientsByRegion for the query, which will add fields under Query Parameters.
4. Enter NE for the region value, and then click Save.
5. Execute the MyADC_Batch_regional job. Only the four patients from the NE region are processed in the Treatment table of the PatientRecords database.

Run a Large Batch Test

The scripts from the Batch Rule Processing sample can be used to run large amounts of data through a Decision Service at one time. To test this out, a batch utility script generates patient records into the database. The example adds a thousand more records.

1. In the Corticon Studio's Project Explorer window, locate the Batch Rule Processing sample's SQL for your database, choose the generate_patients script, and then open it in a database management tool.
2. Execute the script in the database. The database table expands to 1000 records.
3. In the Web Console, choose the Batch Configuration MyADC_Batch, and then click Execute. While you have 1000 patients, only 171 of them are in the NE region.
4. Edit the Batch configuration to set the Query to AllPatients. Save it and then click Execute. All 1000 records are processed.
5. When the job is done, open the Decision Service panel in the Web Console to see the updated counters and charts: