The Apache® Kafka® implementation of the message producer sends messages asynchronously. This means that the Send() method does not block, but returns immediately. Individual records are added to a queue and are only sent after a configurable amount of time has expired, a configurable number of messages has been reached, or a configurable amount of bytes has been reached. Once the message is sent, the broker that received the message returns a delivery receipt. The delivery receipt indicates a success or failure to record the message, along with a reason in the case of failure. This delivery receipt is sent back to the application asynchronously.

Initially, the delivery receipt has a Completed value of FALSE, and other properties such as Offset have unknown values (?). Once the delivery receipt is received, the AVM updates the response object. The response object has the Completed property set to TRUE, and the Success property set appropriately. For messages that failed to send, or were rejected by the broker, the application can access the ErrorMessage and/or ErrorCode properties to determine the cause of the failure.

Sending messages asynchronously is much more efficient than sending a single message. The Kafka client combines multiple individual records into a single request to the server, thus reducing network traffic. Because messages are sent asynchronously, the application itself is responsible for verifying if a message was sent successfully. The application may choose to force the message to be sent right away by calling the Flush() method and waiting for the response, or the application may send multiple messages before checking for the results.

Note: You may choose not to wait or verify the status of the message. In such cases, the response object is garbage collected and there is no way to verify if the message was sent successfully from the producer client. The broker still sends the delivery report, but the response is not accessible. It is best to preserve a reference to the response object until it is no longer needed.

Message compression

Kafka messages can be sent uncompressed or compressed. Message compression can often reduce bandwidth and storage space on the server. Some compression schemes supported by Kafka are: "none", "gzip", "snappy", "lz4", and "zstd". The default setting is uncompressed ("none"). The following example shows how to enable compression for the producer:

producerBuilder:SetProducerOption("compression.codec", "name-of-supported-compression").