To configure SSL connections with Apache® Kafka®, you must set the security.protocol property and, optionally, the ssl.ca.location property:
Kafka property Value
security.protocol SSL
ssl.ca.location (optional) server_certificate_authority_file

Where server_certificate_authority_file is a path to the x509 server certificate file in a format supported by openssl.

The librdkafka library uses the openssl library and supports retrieving certificates from the operating system if they are installed, rather than specifying the ssl.ca.location file name explicitly. If the certificate chain for the Kafka broker is installed, then Kafka uses that and there is no need to set the ssl.ca.location file name. In this case, only the security.protocol setting is required to indicate SSL connections should be used.

The next sections describe how to set these properties for a producer builder and for a consumer builder.

Producer

Set the security.protocol property using either the SetProducerOption() method or the strongly-typed method, SetSecurityProtocol():
pb:SetProducerOption("security.protocol", "ssl").
// or
pb:SetSecurityProtocol(SecurityProtocol:SSL).

Set the ssl.ca.location property using the SetProducerOption() method.

pb:SetProducerOption("ssl.ca.location", "server_certificate_authority_file").

Consumer

Set the security.protocol property using either the SetConsumerOption() method or the strongly-typed method, SetSecurityProtocol():
cb:SetConsumerOption("security.protocol", "ssl").
// or
cb:SetSecurityProtocol(SecurityProtocol:SSL).
Set the ssl.ca.location property using the SetConsumerOption() method.
cb:SetConsumerOption("ssl.ca.location", "server_certificate_authority_file").

Troubleshooting errors related to SSL

Errors related to SSL (for example, if ssl.ca.location is not set or if the SSL certificate is not installed) result in timeouts for sending (in the producer) or polling (in the consumer). You can Enable logging on the client to help troubleshoot any issues. Set the Progress® OpenEdge® log entry type to MESSAGING and the logging level to 2 or higher. In addition, set the Kafka "log_level" option to 2 or higher.

The following are example messages logged from the producer when the SSL handshake fails:
4GL MESSAGING      3 - FAIL: [thrd:ssl://vm-kafkabroker1-3:25369/bootstrap]: ssl://vm-kafkabroker1-3:25369/bootstrap: SSL handshake failed: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 4ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)

4GL MESSAGING      3 - ERROR: [thrd:app]: rdkafka#producer-1: ssl://vm-kafkabroker1-3:25369/bootstrap: SSL handshake failed: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 9ms in state SSL_HANDSHAKE)
The following are example messages logged from the consumer when the SSL handshake fails:
4GL MESSAGING      3 - FAIL: [thrd:ssl://vm-kafkabroker1-3:25369/bootstrap]: ssl://vm-kafkabroker1-3:25369/bootstrap: SSL handshake failed: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 4ms in state SSL_HANDSHAKE, 32 identical error(s) suppressed)

4GL MESSAGING      3 - ERROR: [thrd:app]: clientidmstlsbasic60924915#consumer-1: ssl://vm-kafkabroker1-3:25369/bootstrap: SSL handshake failed: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 4ms in state SSL_HANDSHAKE, 32 identical error(s) suppressed)