Package org.apache.flink.connector.kafka.sink

Class KafkaRecordSerializationSchemaBuilder<IN>

java.lang.Object
org.apache.flink.connector.kafka.sink.KafkaRecordSerializationSchemaBuilder<IN>
Type Parameters:
IN - type of records to be serialized
@PublicEvolving public class KafkaRecordSerializationSchemaBuilder<IN> extends Object
Builder to construct KafkaRecordSerializationSchema.

This class should give a first entrypoint when trying to serialize elements to ProducerRecord. The following examples show some of the possibilities. 

Simple key-value serialization:
 
 KafkaRecordSerializationSchema.builder()
     .setTopic("topic)
     .setKeySerializationSchema(new SimpleStringSchema())
     .setValueSerializationSchema(new SimpleStringSchema())
     .build()
 
Using Kafka's serialization stack:
 
 KafkaRecordSerializationSchema.builder()
     .setTopic("topic)
     .setKeySerializer(StringSerializer.class)
     .setKafkaValueSerializer(StringSerializer.class)
     .build()
 
With custom partitioner:
 
 KafkaRecordSerializationSchema.builder()
     .setTopic("topic)
     .setPartitioner(MY_FLINK_PARTITIONER)
     .setValueSerializationSchema(StringSerializer.class)
     .build()

The different serialization methods for key and value are mutually exclusive thus i.e. it is not possible to use setKeySerializationSchema(SerializationSchema) and setKafkaKeySerializer(Class) on the same builder instance.

It is necessary to configure exactly one serialization method for the value and a topic.

See Also:

  • Constructor Details

    • KafkaRecordSerializationSchemaBuilder

      public KafkaRecordSerializationSchemaBuilder()

  • Method Details

    • setPartitioner

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setPartitioner(FlinkKafkaPartitioner<? super T> partitioner)
      Deprecated.
      use setPartitioner(KafkaPartitioner)
      Sets a custom partitioner determining the target partition of the target topic.
      Parameters:
      partitioner -
      Returns:
      this

    • setPartitioner

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setPartitioner(KafkaPartitioner<? super T> partitioner)
      Sets a custom partitioner determining the target partition of the target topic.
      Parameters:
      partitioner -
      Returns:
      this

    • setTopic

      public KafkaRecordSerializationSchemaBuilder<IN> setTopic(String topic)
      Sets a fixed topic which used as destination for all records.
      Parameters:
      topic -
      Returns:
      this

    • setTopicSelector

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setTopicSelector(TopicSelector<? super T> topicSelector)
      Sets a topic selector which computes the target topic for every incoming record.
      Parameters:
      topicSelector -
      Returns:
      this

    • setKeySerializationSchema

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setKeySerializationSchema(org.apache.flink.api.common.serialization.SerializationSchema<? super T> keySerializationSchema)
      Sets a SerializationSchema which is used to serialize the incoming element to the key of the ProducerRecord.
      Parameters:
      keySerializationSchema -
      Returns:
      this

    • setKafkaKeySerializer

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setKafkaKeySerializer(Class<? extends org.apache.kafka.common.serialization.Serializer<? super T>> keySerializer)
      Sets Kafka's Serializer to serialize incoming elements to the key of the ProducerRecord.
      Parameters:
      keySerializer -
      Returns:
      this

    • setKafkaKeySerializer

      public <T extends IN, S extends org.apache.kafka.common.serialization.Serializer<? super T>> KafkaRecordSerializationSchemaBuilder<T> setKafkaKeySerializer(Class<S> keySerializer, Map<String,String> configuration)
      Sets a configurable Kafka Serializer and pass a configuration to serialize incoming elements to the key of the ProducerRecord.
      Type Parameters:
      S - type of the used serializer class
      Parameters:
      keySerializer -
      configuration -
      Returns:
      this

    • setValueSerializationSchema

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setValueSerializationSchema(org.apache.flink.api.common.serialization.SerializationSchema<T> valueSerializationSchema)
      Sets a SerializationSchema which is used to serialize the incoming element to the value of the ProducerRecord.
      Parameters:
      valueSerializationSchema -
      Returns:
      this

    • setHeaderProvider

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setHeaderProvider(HeaderProvider<? super T> headerProvider)
      Sets a HeaderProvider which is used to add headers to the ProducerRecord for the current element.
      Parameters:
      headerProvider -
      Returns:
      this

    • setKafkaValueSerializer

      public <T extends IN> KafkaRecordSerializationSchemaBuilder<T> setKafkaValueSerializer(Class<? extends org.apache.kafka.common.serialization.Serializer<? super T>> valueSerializer)
      Sets Kafka's Serializer to serialize incoming elements to the value of the ProducerRecord.
      Parameters:
      valueSerializer -
      Returns:
      this

    • setKafkaValueSerializer

      public <T extends IN, S extends org.apache.kafka.common.serialization.Serializer<? super T>> KafkaRecordSerializationSchemaBuilder<T> setKafkaValueSerializer(Class<S> valueSerializer, Map<String,String> configuration)
      Sets a configurable Kafka Serializer and pass a configuration to serialize incoming elements to the value of the ProducerRecord.
      Type Parameters:
      S - type of the used serializer class
      Parameters:
      valueSerializer -
      configuration -
      Returns:
      this

    • build

      public KafkaRecordSerializationSchema<IN> build()
      Constructs the KafkaRecordSerializationSchemaBuilder with the configured properties.
      Returns:
      KafkaRecordSerializationSchema