At a high level, a consumer reads and processes streams of records (messages) from an Apache® Kafka® cluster. The first step is to create a consumer builder, for managing the configuration for the consumer. Configuration includes specifying the servers to connect to and the deserializers for reading records. After the configuration options are set, the consumer is ready to be built. Once built, the consumer is ready to receive records. The consumer polls the cluster and requests a record from the stream. When the consumer receives a record, it acknowledges it, and then reads the record body and processes it according to the needs of the application. This cycle repeats with the consumer polling, acknowledging, reading, and processing records. The consumer handles any errors and deletes the consumer object when it no longer needs it.

This figure shows the basic steps of the process:

Progress® OpenEdge® provides two ways to configure consumer properties. The first is generic, where Kafka properties are configured through strings using name-value pairs. The second uses Kafka-specific classes which expose strongly-typed methods for setting properties. The advantage to using strongly-typed methods is that some errors can be caught during compile time, rather than at runtime. You can choose whatever approach works best for your application. The next topics describe both approaches.