org.apache.flink.connector.kafka.source

Class KafkaPropertiesUtil

java.lang.Object

org.apache.flink.connector.kafka.source.KafkaPropertiesUtil

@Internal
public class KafkaPropertiesUtil
extends Object

Utility class for modify Kafka properties.

Method Summary

Modifier and Type	Method and Description
static void	copyProperties(Properties from, Properties to)
static void	setClientIdPrefix(Properties properties, String kafkaClusterId) client.id is used for Kafka server side logging, see https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#consumerconfigs_client.id

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

copyProperties

public static void copyProperties(@Nonnull
                                  Properties from,
                                  @Nonnull
                                  Properties to)

setClientIdPrefix

public static void setClientIdPrefix(Properties properties,
                                     String kafkaClusterId)

client.id is used for Kafka server side logging, see https://docs.confluent.io/platform/current/installation/configuration/consumer-configs.html#consumerconfigs_client.id

Set client id prefix to avoid mbean collision warning logs. There are multiple instances of the AdminClient/KafkaConsumer so each instance requires a different client id (i.e. also per cluster).

Flink internally configures the clientId, making this the only way to customize the Kafka client id parameter.

If this is not done, we will encounter warning logs of the form:

WARN org.apache.kafka.common.utils.AppInfoParser [] - Error registering AppInfo mbean javax.management.InstanceAlreadyExistsException: kafka.consumer:type=app-info,id=null-enumerator-consumer

WARN org.apache.kafka.common.utils.AppInfoParser [] - Error registering AppInfo mbean javax.management.InstanceAlreadyExistsException: kafka.admin.client:type=app-info,id=null-enumerator-admin-client