The transactional producer is an additional set of APIs that extend the existing message producer class and provide for transactional control over sending messages to an Apache Kafka® cluster. A transaction is a way to group messages, so that all the messages in the group remain invisible to consumers, until all have been sent successfully and the transaction is completed.

Note: When working with messages and transactions, it is important to understand how, and when, the system will attempt to deliver a message (delivery guarantee). Be sure you understand what Kafka actually guarantees, so that you can write your producer/consumer application correctly to get the desired behavior. You can learn more here:
Sending messages transactionally from ABL code is achieved through calls to the following methods:
  • InitTransactions()
  • BeginTransaction()
  • CommitTransaction()
  • AbortTransaction()
These methods allow you to initialize, begin, commit, and abort a transaction, where one or more messages are sent (using Send()), within the same transaction.

Consume → Produce transactional pattern

The Consume → Produce transactional pattern is a way to code your application in order to provide a delivery guarantee where messages are processed once, and only once. In this pattern, an ABL application produces one or more messages each time it consumes one or more messages. The consumption of messages is scoped to the same transaction as the production of messages by calling one of the following methods:
  • SendOffsetToTransaction()
  • SendOffsetsToTransaction()
These methods send a specified offset, or list of offsets, to the consumer group coordinator, and also mark those offsets as part of the current transaction. The offsets are considered committed only if the transaction is committed successfully.