This chapter describes the specific implementation details for various functional requirements.

Load Balancing Configuration

Generally load balancing of Semaphore occurs for only two server components: Classification Server and Semantic Enhancement Server. These components can simply be installed on as many servers as required (within the licence agreement you have with Progress for use of the software).

The Classification Serve publish mechanism will support publishing to multiple installations of Classification Server so this is not a problem (you simply alter the publisher configuration file to send the rulebases generated from the publication process to all of the Classification Servers you have installed, see the details below).

A general configuration looks something like this:

Knowledge Model Management → publish process → [Load Balanced Servers]

The types of load balancing supported by Semaphore include:

• Round Robin
• NLB
• Windows NLB

Implementing the Semaphore loading balancing infrastructure consists of the following:

1. Install load balancer - Ensure the software and/or hardware is in place to be able perform load balancing.
2. Install Classification Server/Semantic Enhancement Server on all required servers (in the load balanced environment) - Each server should be individually addressable to allow the Publisher to publish to each separate instance
3. Configure Publisher to publish the model to all relevant Classification Server/Semantic Enhancement Server instances - This is discussed below
4. Publish the model using the updated configuration file
5. Configure the load balancer to use the Classification Server/Semantic Enhancement Server instances - As per normal load balancing configuration

Failover Configuration

The configuration of a failover environment is much the same as is done with the load balancing configuration. Implementing the Semaphore failover infrastructure consists of the following:

1. Install Classification Server/Semantic Enhancement Server on the required servers (in the failover environment) - Each server should be individually addressable to allow the Publisher to publish to each separate instance
2. Configure Publisher to publish the model to all relevant Classification Server/Semantic Enhancement Server instances - This is discussed below
3. Publish the model using the updated configuration file
4. Configure the failover software to use the relevant Classification Server/Semantic Enhancement Server instances - As per normal failover configuration

Deploying Rulebase sets to multiple Classification Server instances

Having multiple instances of Classification Server running on separate infrastructure allows you to dramatically improve classification performance, however, managing these instances can be problematic as you need to ensure the rulebase sets they use are kept in sync with one another. To address this there are two approaches recommended by Progress for deploying a published set of rulebases to multiple instances (installations) of Classification Server:

Each of these approaches are described below.

Publishing to multiple Classification Server instances

This approach is to simply update the model-specific publisher configuration to publish to each and every Classification Server instance. It is implemented as follows:

1. Download the publisher configuration for the relevant model from Semaphore Studio (Knowledge Model Management).

2. Duplicate the “rulebasePublisher” section of the downloaded configuration file (e.g. “Semaphore-Publisher.xml”) as many times as required changing the “id” value for each of the new copies as well as changing the “classificationServerHost” and “classificationServerPort” values, as appropriate, for example to publish both to “localhost” and “server.domainname.com” I might have:

   Copy

   <bean id="rulebasePublisher" class="com.smartlogic.publisher.pak.PakFilePublisher">
       <property name="classificationServerHost" value="localhost" />
       <property name="classificationServerPort" value="5058" />
       <property name="publishSetName" value="${model.name}" />
   </bean>
   <bean id="rulebasePublisherSecond" class="com.smartlogic.publisher.pak.PakFilePublisher">
       <property name="classificationServerHost" value="server.domainname.com" />
       <property name="classificationServerPort" value="5058" />
       <property name="publishSetName" value="${model.name}" />
   </bean>
   
   

3. Update the “outputProcessors” property of the “myAllConcepts” section (this is used by default but if you have created any other “configurationSets” update each of those) of the configuration file to reference the new sections created in the previous step. With our example this would change from:

   Copy

   <bean id="myAllConcepts" parent="AllResources">
           <property name="outputProcessors">
               <list>
               ...
               <ref bean="rulebasePublisher" />
           </property>
   </bean>
   
   

   To:

   Copy

   <bean id="myAllConcepts" parent="AllResources">
           <property name="outputProcessors">
               <list>
               ...
               <ref bean="rulebasePublisher" />
               <ref bean="rulebasePublisherSecond" />
           </property>
   </bean>
   
   

4. Save the configuration file changes. 5. Upload the new configuration file in Semaphore Studio (Knowledge Model Management).

After completing these steps, the next time you publish the model using this configuration file Publisher will publish the rulebases to all configured Classification Server instances.

Configuring Classification Server to share published rulenets

This approach makes use of a shared folder to store the compiled rulenets (the complete set of rulebases published to Classification Server compiled for immediate loading into the Classification Server memory space) which is created by a single instance of Classification Server with all others simply loading the rulenet when it is ready. This approach can be implemented as follows:

1. Shut down all instances of Classification Server (if any are running).

2. Configure a network share containing the contents of the “rulebases” subfolder of the Classification Server installation, for example, using “SMB” in Windows or “NFS” in Linux. This share will need to be mounted/made available (permanently) for all Classification Server installations.

3. Copy the files from a Classification Server instance “rulebases” folder that contains the current set of rulebases you are using into this network share.

4. Edit the “<Classification Server installation folder>/config/config.xml” file on all instances to change the “acquisition” bean, “rulebasedir” setting to point to the network share you have created, for example in Linux I might have:

   Copy

   ...
       <bean id="acquisition" class="acquisition">
       <property name="tmp_dir" value="../tmp"/>
       <property name="rulebasedir" value="/mnt/share"/>
   ...
   
   

   Note:For simplicity you may want to simply mount the network share over top of the existing “/var/opt/semaphore/CS/rulebases” folder which would mean no modification to the Classification Server configuration.

   In Windows I might have (for the network share mounted as drive “E:”):

   Copy

   ...
       <bean id="acquisition" class="acquisition">
       <property name="tmp_dir" value="../tmp"/>
       <property name="rulebasedir" value="E:/"/>
   ...
   
   

5. Start all Classification Server instances back up.

6. Download the publisher configuration for the relevant model from Semaphore Studio (Knowledge Model Management).

7. Update the “rulebasePublisher” section of the configuration file (e.g. “Semaphore-Publisher.xml”) to change the “classificationServerHost” and “classificationServerPort” values, as appropriate, to use one of the Classification Server instances. It does not matter which node you choose as the other nodes will automatically pick up the compiled rulenet once the node that has been published to saves it.

8. Upload the updated configuration for the relevant model from Semaphore Studio (Knowledge Model Management).

9. Repeat steps 6, 7 and 8 for all models required to be published.

Publishing to multiple Semantic Enhancement Server instances

This approach is to simply update the model-specific publisher configuration to publish to each and every Semantic Enhancement Server instance. It is implemented as follows:

1. Download the publisher configuration for the relevant model from Semaphore Studio (Knowledge Model Management).

2. Duplicate the “SolrWriterTemplate” section (<bean>) of the downloaded configuration file (e.g. “Semaphore-Publishers.xml”) as many times as required changing the URLs appropriately in the same “outputProcessors” section, for example to publish both to “localhost” and “server.domainname.com” I might have:

   Copy

   ...
   <bean id="myAllConcepts" parent="AllResources">
       <property name="outputProcessors">
           <bean parent="SolrWriterTemplate">
               <property name="indexName" value="${model.name}" />
               <property name="solrURL" value="http://localhost:8983/solr"/>
               <property name="sesModelsVersionsURLs">
                   <list>
                       <value>http://localhost:8983/ses/modelversions</value>
                   </list>
               </property>
               <property name="zkHost" value="localhost:9983"/>
           </bean>
           <bean parent="SolrWriterTemplate">
               <property name="indexName" value="${model.name}" />
               <property name="solrURL" value="http://server.domainname.com:8983/solr"/>
               <property name="sesModelsVersionsURLs">
                   <list>
                       <value>http://server.domainname.com:8983/ses/modelversions</value>
                   </list>
               </property>
               <property name="zkHost" value="server.domainname.com:9983"/>
           </bean>
   ...
   
   

3. Save the configuration file changes.

4. Upload the new configuration file into Semaphore Studio (Knowledge Model Management) for the relevant model.

Note the requirement here to have the various SES ports (8983 and 9983) accessible to the machine performing the publish process.

After completing these steps, the next time you publish the model using this configuration file Publisher will publish the model to all configured Semantic Enhancement Server instances.