Package org.apache.flink.connector.kafka.source.reader.deserializer

Interface KafkaRecordDeserializationSchema<T>

All Superinterfaces:

org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>, Serializable

@PublicEvolving
public interface KafkaRecordDeserializationSchema<T>
extends Serializable, org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>

An interface for the deserialization of Kafka records.

Method Summary

Modifier and Type	Method	Description
void	deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record, org.apache.flink.util.Collector<T> out)	Deserializes the byte message.
static <V> KafkaRecordDeserializationSchema<V>	of(KafkaDeserializationSchema<V> kafkaDeserializationSchema)	Wraps a legacy KafkaDeserializationSchema as the deserializer of the ConsumerRecords.
default void	open(org.apache.flink.api.common.serialization.DeserializationSchema.InitializationContext context)	Initialization method for the schema.
static <V> KafkaRecordDeserializationSchema<V>	valueOnly(Class<? extends org.apache.kafka.common.serialization.Deserializer<V>> valueDeserializerClass)	Wraps a Kafka Deserializer to a KafkaRecordDeserializationSchema.
static <V,D extends org.apache.kafka.common.serialization.Deserializer<V>> KafkaRecordDeserializationSchema<V>	valueOnly(Class<D> valueDeserializerClass, Map<String,String> config)	Wraps a Kafka Deserializer to a KafkaRecordDeserializationSchema.
static <V> KafkaRecordDeserializationSchema<V>	valueOnly(org.apache.flink.api.common.serialization.DeserializationSchema<V> valueDeserializationSchema)	Wraps a DeserializationSchema as the value deserialization schema of the ConsumerRecords.

Methods inherited from interface org.apache.flink.api.java.typeutils.ResultTypeQueryable

getProducedType

Method Detail

open

default void open(org.apache.flink.api.common.serialization.DeserializationSchema.InitializationContext context)
           throws Exception

Initialization method for the schema. It is called before the actual working methods deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[], byte[]>, org.apache.flink.util.Collector<T>) and thus suitable for one time setup work.

The provided DeserializationSchema.InitializationContext can be used to access additional features such as e.g. registering user metrics.

Parameters:

context - Contextual information that can be used during initialization.

Throws:

Exception

deserialize

void deserialize(org.apache.kafka.clients.consumer.ConsumerRecord<byte[],byte[]> record,
                 org.apache.flink.util.Collector<T> out)
          throws IOException

Deserializes the byte message.

Can output multiple records through the Collector. Note that number and size of the produced records should be relatively small. Depending on the source implementation records can be buffered in memory or collecting records might delay emitting checkpoint barrier.

Parameters:

record - The ConsumerRecord to deserialize.

out - The collector to put the resulting messages.

Throws:

IOException

of

static <V> KafkaRecordDeserializationSchema<V> of(KafkaDeserializationSchema<V> kafkaDeserializationSchema)

Wraps a legacy KafkaDeserializationSchema as the deserializer of the ConsumerRecords.

Note that the KafkaDeserializationSchema.isEndOfStream(Object) method will no longer be used to determine the end of the stream.

Type Parameters:

V - the return type of the deserialized record.

Parameters:

kafkaDeserializationSchema - the legacy KafkaDeserializationSchema to use.

Returns:

A KafkaRecordDeserializationSchema that uses the given KafkaDeserializationSchema to deserialize the ConsumerRecords.

valueOnly

static <V> KafkaRecordDeserializationSchema<V> valueOnly(org.apache.flink.api.common.serialization.DeserializationSchema<V> valueDeserializationSchema)

Wraps a DeserializationSchema as the value deserialization schema of the ConsumerRecords. The other fields such as key, headers, timestamp are ignored.

Note that the DeserializationSchema.isEndOfStream(Object) method will no longer be used to determine the end of the stream.

Type Parameters:

V - the type of the deserialized record.

Parameters:

valueDeserializationSchema - the DeserializationSchema used to deserialized the value of a ConsumerRecord.

Returns:

A KafkaRecordDeserializationSchema that uses the given DeserializationSchema to deserialize a ConsumerRecord from its value.

valueOnly

static <V> KafkaRecordDeserializationSchema<V> valueOnly(Class<? extends org.apache.kafka.common.serialization.Deserializer<V>> valueDeserializerClass)

Wraps a Kafka Deserializer to a KafkaRecordDeserializationSchema.

Type Parameters:

V - the value type.

Parameters:

valueDeserializerClass - the deserializer class used to deserialize the value.

Returns:

A KafkaRecordDeserializationSchema that deserialize the value with the given deserializer.

valueOnly

static <V,D extends org.apache.kafka.common.serialization.Deserializer<V>> KafkaRecordDeserializationSchema<V> valueOnly(Class<D> valueDeserializerClass,
                                                                                                                               Map<String,String> config)

Wraps a Kafka Deserializer to a KafkaRecordDeserializationSchema.

Type Parameters:

V - the value type.

D - the type of the deserializer.

Parameters:

valueDeserializerClass - the deserializer class used to deserialize the value.

config - the configuration of the value deserializer. If the deserializer is an implementation of Configurable, the configuring logic will be handled by Configurable.configure(Map) with the given config, otherwise Deserializer.configure(Map, boolean) will be invoked.

Returns:

A KafkaRecordDeserializationSchema that deserialize the value with the given deserializer.