org.apache.flink.connector.base.sink

Class AsyncSinkBase<InputT,RequestEntryT extends Serializable>

java.lang.Object

org.apache.flink.connector.base.sink.AsyncSinkBase<InputT,RequestEntryT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.connector.sink2.Sink<InputT>, org.apache.flink.api.connector.sink2.StatefulSink<InputT,BufferedRequestState<RequestEntryT>>, org.apache.flink.api.connector.sink2.SupportsWriterState<InputT,BufferedRequestState<RequestEntryT>>

@PublicEvolving
public abstract class AsyncSinkBase<InputT,RequestEntryT extends Serializable>
extends Object
implements org.apache.flink.api.connector.sink2.StatefulSink<InputT,BufferedRequestState<RequestEntryT>>

A generic sink for destinations that provide an async client to persist data.

The design of the sink focuses on extensibility and a broad support of destinations. The core of the sink is kept generic and free of any connector specific dependencies. The sink is designed to participate in checkpointing to provide at-least once semantics, but it is limited to destinations that provide a client that supports async requests.

Limitations:

• The sink is designed for destinations that provide an async client. Destinations that cannot ingest events in an async fashion cannot be supported by the sink.
• The sink usually persist InputTs in the order they are added to the sink, but reorderings may occur, eg, when RequestEntryTs need to be retried.
• We are not considering support for exactly-once semantics at this point.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.StatefulSink

org.apache.flink.api.connector.sink2.StatefulSink.StatefulSinkWriter<InputT,WriterStateT>, org.apache.flink.api.connector.sink2.StatefulSink.WithCompatibleState

Nested classes/interfaces inherited from interface org.apache.flink.api.connector.sink2.Sink

org.apache.flink.api.connector.sink2.Sink.InitContext, org.apache.flink.api.connector.sink2.Sink.InitContextWrapper

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter, int maxBatchSize, int maxInFlightRequests, int maxBufferedRequests, long maxBatchSizeInBytes, long maxTimeInBufferMS, long maxRecordSizeInBytes)

Method Summary

Modifier and Type	Method and Description
protected ElementConverter<InputT,RequestEntryT>	getElementConverter()
protected int	getMaxBatchSize()
protected long	getMaxBatchSizeInBytes()
protected int	getMaxBufferedRequests()
protected int	getMaxInFlightRequests()
protected long	getMaxRecordSizeInBytes()
protected long	getMaxTimeInBufferMS()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.connector.sink2.StatefulSink

restoreWriter, restoreWriter

Methods inherited from interface org.apache.flink.api.connector.sink2.Sink

createWriter, createWriter

Methods inherited from interface org.apache.flink.api.connector.sink2.SupportsWriterState

getWriterStateSerializer

Constructor Detail

AsyncSinkBase

protected AsyncSinkBase(ElementConverter<InputT,RequestEntryT> elementConverter,
                        int maxBatchSize,
                        int maxInFlightRequests,
                        int maxBufferedRequests,
                        long maxBatchSizeInBytes,
                        long maxTimeInBufferMS,
                        long maxRecordSizeInBytes)

Method Detail

getElementConverter

protected ElementConverter<InputT,RequestEntryT> getElementConverter()

getMaxBatchSize

protected int getMaxBatchSize()

getMaxInFlightRequests

protected int getMaxInFlightRequests()

getMaxBufferedRequests

protected int getMaxBufferedRequests()

getMaxBatchSizeInBytes

protected long getMaxBatchSizeInBytes()

getMaxTimeInBufferMS

protected long getMaxTimeInBufferMS()

getMaxRecordSizeInBytes

protected long getMaxRecordSizeInBytes()