To troubleshoot issues with the Apache® Kafka® integration with Progress® OpenEdge®, it is helpful to enable logging. To enable logging, you must set up logging in both the Kafka client and in the OpenEdge ABL application.

Enable logging in the Kafka client

Kafka has eight logging levels compared to four for OpenEdge. The mapping of logging levels is as follows:

Kafka logging level OpenEdge logging level
0 - EMERGENCY 1 - Errors
1 - ALERT 1 - Errors
2 - CRITICAL 1 - Errors
3 - ERROR 1 - Errors
4 - WARNING 2 - Basic
5 - NOTICE 2 - Basic
6 - INFO 3 - Verbose
7 - DEBUG 4 - Extended

The Kafka client used by OpenEdge does not log much other than at level 7 - debug logging. To enable debug logging in Kafka, you have to set the log_level property to "7". You also have to set the debug property with a comma-separated list of desired debug contexts. Specifying "all" turns on all debug logging. See the Kafka documentation at https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#logging for a list of debug contexts.

To set the logging for a Kafka producer, use the SetProducerOption() method on the ProducerBuilder. The following example enables debug logging in the broker and security areas of the Kafka client.

// Turn on debug logging
pb:SetProducerOption("debug", "broker, security").
pb:SetProducerOption("log_level", "7").

Enable OpenEdge Logging

To enable OpenEdge logging, set the logging level, log entry type, and log file name. The logging level should be set to 4 and the log entry type should be set to MESSAGING. You must set the logginglevel and logentrytype appropriately for Kafka debug logging to appear in the log. When starting a client, specify these startup parameters:
  • -logginglevel 4
  • -logentrytypes MESSAGING
  • -clientlog logfilename

For general OpenEdge logging information, see Logging in OpenEdge in Troubleshoot ABL Applications.

An example of MESSAGING log messages (with headers suppressed) for a Kafka producer is shown:

-- Logging level set to = 4
-- Log entry types activated: MESSAGING
MESSAGING      Kafka Producer rdkafka#producer-1 Configuration
MESSAGING          builtin.features = gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer
MESSAGING          client.id = rdkafka
MESSAGING          client.software.name = librdkafka
...
MESSAGING      Messaging thread started