org.apache.flink.streaming.connectors.kafka

Class KafkaSink<IN>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.connectors.kafka.KafkaSink<IN>

Type Parameters:

IN - Type of the sink input

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<IN>

Direct Known Subclasses:

KafkaSink

public class KafkaSink<IN>
extends org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

Sink that emits its inputs to a Kafka topic.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
KafkaSink(String brokerList, String topicId, Properties producerConfig, org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema) Creates a KafkaSink for a given topic with custom Producer configuration.
KafkaSink(String brokerList, String topicId, org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema) Creates a KafkaSink for a given topic.
KafkaSink(String brokerList, String topicId, org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema, Class<? extends SerializableKafkaPartitioner> partitioner)
KafkaSink(String brokerList, String topicId, org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema, SerializableKafkaPartitioner partitioner) Creates a KafkaSink for a given topic.

Method Summary

Modifier and Type	Method and Description
void	close()
void	invoke(IN next) Called when new data arrives to the sink, and forwards it to Kafka.
void	open(org.apache.flink.configuration.Configuration configuration) Initializes the connection to Kafka.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KafkaSink

public KafkaSink(String brokerList,
                 String topicId,
                 org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema)

Creates a KafkaSink for a given topic. The sink produces its input to the topic.

Parameters:

brokerList - Addresses of the brokers

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema.

KafkaSink

public KafkaSink(String brokerList,
                 String topicId,
                 Properties producerConfig,
                 org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema)

Creates a KafkaSink for a given topic with custom Producer configuration. If you use this constructor, the broker should be set with the "metadata.broker.list" configuration.

Parameters:

brokerList - Addresses of the brokers

topicId - ID of the Kafka topic.

producerConfig - Configurations of the Kafka producer

serializationSchema - User defined serialization schema.

KafkaSink

public KafkaSink(String brokerList,
                 String topicId,
                 org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema,
                 SerializableKafkaPartitioner partitioner)

Creates a KafkaSink for a given topic. The sink produces its input to the topic.

Parameters:

brokerList -

topicId - ID of the Kafka topic.

serializationSchema - User defined serialization schema.

partitioner - User defined partitioner.

KafkaSink

public KafkaSink(String brokerList,
                 String topicId,
                 org.apache.flink.streaming.util.serialization.SerializationSchema<IN,byte[]> serializationSchema,
                 Class<? extends SerializableKafkaPartitioner> partitioner)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration configuration)

Initializes the connection to Kafka.

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

invoke

public void invoke(IN next)

Called when new data arrives to the sink, and forwards it to Kafka.

Specified by:

invoke in interface org.apache.flink.streaming.api.functions.sink.SinkFunction<IN>

Specified by:

invoke in class org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

Parameters:

next - The incoming data

close

public void close()

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction