Optic Java API for Relational Operations

The MarkLogic Optic API is implemented in JavaScript, XQuery, REST, and Java. A general overview and the JavaScript and XQuery implementations of the Optic API is described in Optic API for Multi-Model Data Access in the Application Developer's Guide. This chapter describes the Java Client implementation of the Optic API, which is very similar in structure to the JavaScript version of the Optic API.

This chapter has the following main sections:

• Overview
• Getting Started
• Java Packages
• Structure of the Java Optic API
• Examples
• New Optic operations in Java Client 7.2.0

Overview

The Optic Java Client API provides classes to support building a plan on the client, executing the plan on the server, and processing the response on the client.

On the server, the Optic API can yield a row that satisfies any of several common use cases:

• A traditional flat list of atomic values with names and XML Schema atomic datatypes.

• A dynamic JSON or XML document with substructure and leaf atomic values or mixed text.

• An envelope with out-of-band metadata properties and relations for a list of documents.

The second use case can take advantage of document joins or constructor expressions. The third use case can take advantage of document joins.

On the client, the Optic Java Client API can consume a set of rows in one of the following ways:

• As a single CSV, JSON, or XML payload with all of the rows in the set.

• By iterating over each row with a Java map key-value interface, a pre-defined Plain Old Java Object (POJO) tree structure, or a JSON or XML document structure.

A structured value in a format alien to the response format is encoded as a string. In particular, when getting a CSV payload, a JSON or XML column value is encoded as a string. Similarly, when getting a JSON payload or row, an XML value is encoded as a string (and vice versa).

Getting Started

The Optic Java Client communicates with a REST App Server on MarkLogic.

1. Download the MarkLogic Java Client API to your client system and configure, as described in Getting Started.

2. You can use the preconfigured REST App Server at port 8000, as described in Choose a REST API Instance, however it is generally better that you create your own REST App Server. You can use the POST:/v1/rest-apis call to quickly and conveniently create a REST App Server. For example, to create a REST App Server, named Optic, on a server named MLserver, you can simply enter:

   Copy

   curl -X POST --anyauth -u admin:admin -H "Content-Type:application/json" \
   -d '{
      "rest-api": {"name": "Optic"} 
   }' \
   http://MLserver:8002/v1/rest-apis
   

The Optic App Server will be assigned an unused port number and all of the required forests and databases will be created for it. The Optic database created for you will use the default Schemas database. However, you should create a unique schemas database and assign it to the Optic database.

To run the examples described in this chapter, do the following:

1. Follow the steps in Load the Data in the SQL Data Modeling Guide to load the sample documents into the database. Use the database associated with your REST API instance (Optic) rather than the one used in the procedure.
2. Follow the stops in Create Template Views in the SQL Data Modeling Guide to create views and insert the template view documents into the schema database assigned to the Optic database.

Java Packages

The following packages implement the Optic features in the Java API:

See the MarkLogic Java API JavaDoc reference for details.

Structure of the Java Optic API

The Java Optic API is similar to the server-side JavaScript and XQuery implementations of the Optic API described in Optic API for Multi-Model Data Access in the Application Developer's Guide. This chapter describes the Java Client implementation of the Optic API, which is similar in structure.

The Optic API for Multi-Model Data Access chapter in the Application Developer's Guide contains the following main topics of interest to Java Optic developers:

• Objects in an Optic Pipeline
• Data Access Functions
• Kinds of Optic Queries
• Expression Functions For Processing Column Values
• Functions Equivalent to Boolean, Numeric, and String Operators
• Node Constructor Functions
• Best Practices and Performance Considerations
• Optic Execution Plan
• Parameterizing a Plan
• Exporting and Importing a Serialized Optic Query

Values and Expressions

The *Val interfaces represent client-side values typed with server data types. For example, the PlanBuilder.xs.decimal method constructs a client value with an xs.decimal data type.

The *Expr interfaces represent server expressions typed with server data types. For example, the PlanBuilder.fn.formatNumber method constructs a server expression to format the result of a numeric expression as a string expression.

Server expressions executed to produce the boolean expression for a where operation or the expression assigned to a column by the PlanBuilder.as function can take columns as arguments. The function call calculates the result of the expression for each row using the values of the columns in the row. For example, if the first argument to PlanBuilder.fn.formatNumber is a column, the formatted string will be produced for each row with the value of the column. The column must have a value in each row with the data type required in the expression.

The API provides some overloads for typical literal arguments of expression functions as a convenience.

The com.marklogic.client.type package has the marker interfaces for the server data types.

Items and Sequences

Some functions can take multiple values or expressions for a parameter. Such parameters have a sequence data type. A sequence data type can take either a single item of the data type or a sequence of the data type. The API provides constructor functions that take a varargs of items of the appropriate data type and return the sequence.

For instance, PlanBuilder.pattern takes a sequence for the subject, predicate, and object parameters.

The call can pass either one PlanTriplePosition instance (an XsAnyAtomicTypeVal, PlanColumn, or PlanParamExpr object) as the subject or use PlanBuilder.subject to construct a sequence of such objects to pass as the subject.

Atomic Values and Nodes in RowRecord

RowRecord provides the getKind metadata method for discovering the ColumnKind of a column in the current row (ATOMIC_VALUE, CONTENT, or NULL).

For an ATOMIC_VALUE column, the getDatatype metadata method reports the atomic data type.

You can call a get* getter to cast the value to the appropriate primitive or to a *Val type.

For a CONTENT column, the getContentFormat and getContentMimetype metadata methods report the format and mime type. The caller can pass the appropriate handle to the getContent getter to read the JSON, XML, binary, or text content (consistent with the Java API elsewhere).

Examples

The following two examples are based on documents and template views described in the Creating Template Views chapter in the SQL Data Modeling Guide.

List all of the employees in order of ID number.

package Optic;

import com.marklogic.client.DatabaseClient;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.io.StringHandle;
import com.marklogic.client.expression.PlanBuilder;
import com.marklogic.client.expression.PlanBuilder.ModifyPlan;
import com.marklogic.client.row.RowManager;

public class optic4 {
   public static void main(String[] args) {
      DatabaseClient db = DatabaseClientFactory.newClient(
         "localhost", 8000, 
         new DatabaseClientFactory.DigestAuthContext("admin", "admin")
      );

      RowManager rowMgr = db.newRowManager();
      PlanBuilder p = rowMgr.newPlanBuilder();

      ModifyPlan plan = p.fromView("main", "employees")
         .select("EmployeeID", "FirstName", "LastName")
         .orderBy("EmployeeID")
         .offsetLimit(0, 25);

      System.out.println(
         rowMgr.resultDoc(plan, 
         new StringHandle().withMimetype("text/csv")).get()
         );

      return;
   }
}

Return the ID and full name for the employee with an EmployeeID of 3.

package Optic;

import com.marklogic.client.DatabaseClient;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.io.StringHandle;
import com.marklogic.client.expression.PlanBuilder;
import com.marklogic.client.expression.PlanBuilder.ModifyPlan;
import com.marklogic.client.row.RowManager;
import com.marklogic.client.type.XsIntVal;

public class optic {

   public static void main(String[] args) {
      DatabaseClient db = DatabaseClientFactory.newClient(
         "MLserver", 8000, 
         new DatabaseClientFactory.DigestAuthContext("admin", "admin")
      );

      RowManager rowMgr = db.newRowManager();
          PlanBuilder p = rowMgr.newPlanBuilder();
          XsIntVal EmployeeID = p.xs.intVal(3);

      ModifyPlan plan = p.fromView("main", "employees")
          .where(p.eq(p.col("EmployeeID"), EmployeeID))
          .select("EmployeeID", "FirstName", "LastName")
          .orderBy("EmployeeID");

      System.out.println(
          rowMgr.resultDoc(plan, 
          new StringHandle().withMimetype("text/csv")).get()
          );

      return;
      }
}

The following example returns a list of the people who were born in Brooklyn in the form of a table with two columns, person and name. This is executed against the example dataset described in Loading Triples in the Semantics Developer's Guide. This example is the Java equivalent of the last JavaScript example described in fromView Examples in the Optic API for Multi-Model Data Access chapter in the Application Developer's Guide.

package Optic;

import com.marklogic.client.DatabaseClient;
import com.marklogic.client.DatabaseClientFactory;
import com.marklogic.client.DatabaseClientFactory.DigestAuthContext;
import com.marklogic.client.io.StringHandle;
import com.marklogic.client.expression.PlanBuilder;
import com.marklogic.client.row.RowManager;
import com.marklogic.client.type.PlanColumn;

public class optic2 {

    public static void main(String[] args) {
      DatabaseClient db = DatabaseClientFactory.newClient(
          "localhost", 8000, 
          new DigestAuthContext("admin", "admin")
      );
      RowManager rowMgr = db.newRowManager();
      PlanBuilder p = rowMgr.newPlanBuilder();

      PlanBuilder.Prefixer foaf =
          p.prefixer("http://xmlns.com/foaf/0.1");
      PlanBuilder.Prefixer onto =
          p.prefixer("http://dbpedia.org/ontology");
      PlanBuilder.Prefixer resource =
          p.prefixer("http://dbpedia.org/resource");

      PlanColumn person = p.col("person");

      PlanBuilder.QualifiedPlan plan = p.fromTriples(
        p.pattern(person, onto.iri("birthPlace"),
                  resource.iri("Brooklyn")),
        p.pattern(person, foaf.iri("name"), p.col("name"))
      );

      System.out.println(
          rowMgr.resultDoc(plan, 
          new StringHandle().withMimetype("text/csv")).get()
      );

      return;
    }
}

New Optic operations in Java Client 7.2.0

The Java Client API supports these new Optic functions:

• cosine
• shortestPath
• annTopK
• VectorUtil

cosine

The cosine() function returns the cosine of the angle between two vectors. The vectors must be of the same dimension. The cosine function in the Java client provides a client interface to the server function vec.cosine().

Parameters: It takes two vector inputs to calculate the cosine value.

Example:

op.fromView("vectors", "persons")
.bind(op.as("sampleVector", op.vec.vector(twoDimensionalVector)))
.bind(op.as("cosine", op.vec.cosine(op.col("embedding"), op.col("sampleVector"))))
.select(op.col("name"), op.col("summary"), op.col("cosine"));

shortestPath

This method can be used to find the shortest path between two nodes in a given graph. The shortestPath method provides a client interface to the server function shortestPath().

shortestPath takes these parameters:

• start: This column is the input starting subject of the shortest path search. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• end: This column is the input ending object of the shortest path search. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• path: This column is the output column representing the actual shortest path(s) taken from start to end. Values are not returned for this column if this is the empty sequence. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• length: This column is the output column representing the length of the shortest path. A value is not returned for this column if this is the empty sequence. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• weight : This column is an optional weight parameter.

Example:

op.fromTriples(
op.pattern(op.col("player_team"), team.iri("name"), op.col("team_name")),
op.pattern(op.col("player_team"), team.iri("city"), op.col("team_city"))
)
.where(op.eq(op.col("team_name"), op.xs.string("Giants")))
.shortestPath(op.col("team_name"), op.col("team_city"), op.col("path"), op.col("length"));

annTopK

This method searches against vector data, using a query vector, selecting and returning the top K nearest vectors from the column along with data associated with that vector, for example, document, node, or row. The annTopK() method provides a client interface to the server function annTopK().

annTopK takes these parameters:

• inputK: This column is the positive integer k, the number of nearest neighbor results to return.
• vectorColumn: This column is the input table to perform vector distance calculations against. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• queryVector: This column is the query vector to compare with the vectors from the vector-column column.
• distanceColumn: This column is an optional output column that returns the value of the distance metric for that output row. The columns can be named with a string or a column function such as op:col(), op:view-col(), or op:schema-col().
• options: This column is either a sequence of strings or a map containing keys and values for the options to this operator. These are the options:
  • max-distance: This option is a number that determines the maximum distance for a returned result. For cosine distance, the default is float max. Rows with a distance greater than float max are not returned.
  • search-factor: This option can be used to increase or decrease the number of candidate vectors found from the index. Default: 1.0. Higher values result in slower searches that may provide higher results accuracy. Lower values result in faster searches that may give lower accuracy.
  • distance: This option is a cosine value.

Example:

Map<String, Object> options = new HashMap<>();
options.put("distance", "cosine");
PlanBuilder.ModifyPlan plan = op.fromView("vectors", "persons")
.annTopK(10, op.col("embedding"), op.vec.vector(sampleVector), op.col("distance"), options);

VectorUtil

Supports encoding and decoding vectors using the same approach as the vec:base64-encode() and vec:base64-decode() functions supported by MarkLogic Server.

base64Encode: Converts an array of vector float values into an encoded string. Encoding vectors before writing them to documents helps reduce the amount of disk space and memory that the vectors consume.

Example:

VectorUtil.base64Encode(new float[]{3.14f, 1.59f, 2.65f});

base64Decode: Converts an encoded string value to an array of vectors.

Example:

VectorUtil.base64Decode("AAAAAAMAAADD9UhAH4XLP5qZKUA=");