Package org.apache.flink.connector.kafka.sink

Class KafkaSinkBuilder<IN>

java.lang.Object
org.apache.flink.connector.kafka.sink.KafkaSinkBuilder<IN>
Type Parameters:
IN - type of the records written to Kafka
@PublicEvolving public class KafkaSinkBuilder<IN> extends Object
Builder to construct KafkaSink.

The following example shows the minimum setup to create a KafkaSink that writes String values to a Kafka topic. 


 KafkaSink<String> sink = KafkaSink
     .<String>builder
     .setBootstrapServers(MY_BOOTSTRAP_SERVERS)
     .setRecordSerializer(MY_RECORD_SERIALIZER)
     .build();

One can also configure different DeliveryGuarantee by using setDeliveryGuarantee(DeliveryGuarantee) but keep in mind when using DeliveryGuarantee.EXACTLY_ONCE one must set the transactionalIdPrefix setTransactionalIdPrefix(String).

See Also:

  • Method Details

    • setDeliveryGuarantee

      public KafkaSinkBuilder<IN> setDeliveryGuarantee(org.apache.flink.connector.base.DeliveryGuarantee deliveryGuarantee)
      Sets the wanted the DeliveryGuarantee. The default delivery guarantee is deliveryGuarantee.
      Parameters:
      deliveryGuarantee -
      Returns:
      KafkaSinkBuilder

    • setDeliverGuarantee

      @Deprecated public KafkaSinkBuilder<IN> setDeliverGuarantee(org.apache.flink.connector.base.DeliveryGuarantee deliveryGuarantee)
      Deprecated.
      Will be removed in future versions. Use setDeliveryGuarantee(org.apache.flink.connector.base.DeliveryGuarantee) instead.
      Sets the wanted the DeliveryGuarantee. The default delivery guarantee is deliveryGuarantee.
      Parameters:
      deliveryGuarantee -
      Returns:
      KafkaSinkBuilder

    • setKafkaProducerConfig

      public KafkaSinkBuilder<IN> setKafkaProducerConfig(Properties props)
      Sets the configuration which used to instantiate all used KafkaProducer.
      Parameters:
      props -
      Returns:
      KafkaSinkBuilder

    • setProperty

      public KafkaSinkBuilder<IN> setProperty(String key, String value)

    • setRecordSerializer

      public KafkaSinkBuilder<IN> setRecordSerializer(KafkaRecordSerializationSchema<IN> recordSerializer)
      Sets the KafkaRecordSerializationSchema that transforms incoming records to ProducerRecords.
      Parameters:
      recordSerializer -
      Returns:
      KafkaSinkBuilder

    • setTransactionalIdPrefix

      public KafkaSinkBuilder<IN> setTransactionalIdPrefix(String transactionalIdPrefix)
      Sets the prefix for all created transactionalIds if DeliveryGuarantee.EXACTLY_ONCE is configured.

      It is mandatory to always set this value with DeliveryGuarantee.EXACTLY_ONCE to prevent corrupted transactions if multiple jobs using the KafkaSink run against the same Kafka Cluster. The default prefix is transactionalIdPrefix.

      The size of the prefix is capped by MAXIMUM_PREFIX_BYTES formatted with UTF-8.

      It is important to keep the prefix stable across application restarts. If the prefix changes it might happen that lingering transactions are not correctly aborted and newly written messages are not immediately consumable until the transactions timeout.

      Parameters:
      transactionalIdPrefix -
      Returns:
      KafkaSinkBuilder

    • setBootstrapServers

      public KafkaSinkBuilder<IN> setBootstrapServers(String bootstrapServers)
      Sets the Kafka bootstrap servers.
      Parameters:
      bootstrapServers - a comma separated list of valid URIs to reach the Kafka broker
      Returns:
      KafkaSinkBuilder

    • build

      public KafkaSink<IN> build()
      Constructs the KafkaSink with the configured properties.
      Returns:
      KafkaSink