Additional configuration settings are required for an Apache Kafka® transactional producer.

transactional.id

Kafka supports the transactional sending of messages when the producer is configured with an application-wide unique identifier known as the transactional.id. This identifier is used to contain stale transactions from previous instances of the application, which may occur following a crash or an outage. Once a producer is configured to be transactional, messages may only be sent from within a transaction.

Kafka property OpenEdge method
transactional.id SetTransactionalId("some unique identifier")

transaction.timeout.ms

Most of the operations of the transactional producer can time out. The timeout value is configured using the Kafka producer option, transaction.timeout.ms. This value is the time in milliseconds to wait before a transaction is aborted. The default value is 60,000 ms.

Kafka property
transaction.timeout.ms

After a time out, some operations can be retried. In such a case, a Progress.Lang.MessagingErrorRetriable is raised. The exception can be caught and the operation retried as appropriate.

Additional consumer-side configuration for the Consume → Produce transactional pattern

Additional configuration options must be set on the consumer side in order to use a Consume → Produce transactional pattern.

enable.auto.commit

Enable.auto.commit must be disabled (set to false) in order to allow the application to control when and how commits occur.

Kafka property OpenEdge method
enable.auto.commit=false SetEnableAutoCommit(false)

isolation.level

The isolation.level should be set to read_committed (the default value), so that only transactional messages which have been committed are visible to consumers. If isolation.level was set to read_uncommitted, then all messages would be accessible, even transactional messages which have been aborted.

Kafka property OpenEdge method
isolation.level=read_committed SetIsolationLevel(IsolationLevel:ReadCommitted)

In this pattern, everyone needs to play by the rules. The application code might be reading messages from another transaction, but the messages should not be read until the transaction is committed. Once committed, the application can produce messages and tie them to the consumed records, thus completing a chain of transactions.