org.apache.flink.streaming.api.functions.sink

类 TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>

类型参数:

IN - Input type for SinkFunction.

TXN - Transaction to store all of the information required to handle a transaction.

CONTEXT - Context that will be shared across all invocations for the given TwoPhaseCommitSinkFunction instance. Context is created once

所有已实现的接口:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.common.state.CheckpointListener, CheckpointedFunction, SinkFunction<IN>

已过时。

This interface will be removed in future versions. Use the new Sink interface instead.

@Deprecated
 @PublicEvolving
public abstract class TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>
extends RichSinkFunction<IN>
implements CheckpointedFunction, org.apache.flink.api.common.state.CheckpointListener

This is a recommended base class for all of the SinkFunction that intend to implement exactly-once semantic. It does that by implementing two phase commit algorithm on top of the CheckpointedFunction and CheckpointListener. User should provide custom TXN (transaction handle) and implement abstract methods handling this transaction handle.

另请参阅:

序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	TwoPhaseCommitSinkFunction.State<TXN,CONTEXT> 已过时。 State POJO class coupling pendingTransaction, context and pendingCommitTransactions.
static class	TwoPhaseCommitSinkFunction.StateSerializer<TXN,CONTEXT> 已过时。 Custom TypeSerializer for the sink state.
static class	TwoPhaseCommitSinkFunction.StateSerializerSnapshot<TXN,CONTEXT> 已过时。 Snapshot for the TwoPhaseCommitSinkFunction.StateSerializer.
static class	TwoPhaseCommitSinkFunction.TransactionHolder<TXN> 已过时。 Adds metadata (currently only the start time of the transaction) to the transaction object.

从接口继承的嵌套类/接口 org.apache.flink.streaming.api.functions.sink.SinkFunction

SinkFunction.Context

字段概要

字段

限定符和类型	字段和说明
protected LinkedHashMap<Long,TwoPhaseCommitSinkFunction.TransactionHolder<TXN>>	pendingCommitTransactions 已过时。
protected org.apache.flink.api.common.state.ListState<TwoPhaseCommitSinkFunction.State<TXN,CONTEXT>>	state 已过时。
protected Optional<CONTEXT>	userContext 已过时。

构造器概要

构造器

构造器和说明
TwoPhaseCommitSinkFunction(org.apache.flink.api.common.typeutils.TypeSerializer<TXN> transactionSerializer, org.apache.flink.api.common.typeutils.TypeSerializer<CONTEXT> contextSerializer) 已过时。 Use default ListStateDescriptor for internal state serialization.

方法概要

限定符和类型	方法和说明
protected abstract void	abort(TXN transaction) 已过时。 Abort a transaction.
protected abstract TXN	beginTransaction() 已过时。 Method that starts a new transaction.
void	close() 已过时。
protected abstract void	commit(TXN transaction) 已过时。 Commit a pre-committed transaction.
protected TXN	currentTransaction() 已过时。
protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>	enableTransactionTimeoutWarnings(double warningRatio) 已过时。 Enables logging of warnings if a transaction's elapsed time reaches a specified ratio of the transactionTimeout.
void	finish() 已过时。 This method is called at the end of data processing.
protected void	finishProcessing(TXN transaction) 已过时。 This method is called at the end of data processing.
protected void	finishRecoveringContext(Collection<TXN> handledTransactions) 已过时。 Callback for subclasses which is called after restoring (each) user context.
protected Optional<CONTEXT>	getUserContext() 已过时。
protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>	ignoreFailuresAfterTransactionTimeout() 已过时。 If called, the sink will only log but not propagate exceptions thrown in recoverAndCommit(Object) if the transaction is older than a specified transaction timeout.
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context) 已过时。 This method is called when the parallel function instance is created during distributed execution.
protected Optional<CONTEXT>	initializeUserContext() 已过时。
void	invoke(IN value) 已过时。 This should not be implemented by subclasses.
void	invoke(IN value, SinkFunction.Context context) 已过时。 Writes the given value to the sink.
protected abstract void	invoke(TXN transaction, IN value, SinkFunction.Context context) 已过时。 Write value within a transaction.
void	notifyCheckpointAborted(long checkpointId) 已过时。
void	notifyCheckpointComplete(long checkpointId) 已过时。
protected Stream<Map.Entry<Long,TXN>>	pendingTransactions() 已过时。
protected abstract void	preCommit(TXN transaction) 已过时。 Pre commit previously created transaction.
protected void	recoverAndAbort(TXN transaction) 已过时。 Abort a transaction that was rejected by a coordinator after a failure.
protected void	recoverAndCommit(TXN transaction) 已过时。 Invoked on recovered transactions after a failure.
protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT>	setTransactionTimeout(long transactionTimeout) 已过时。 Sets the transaction timeout.
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context) 已过时。 This method is called when a snapshot for a checkpoint is requested.

从类继承的方法 org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.functions.sink.SinkFunction

writeWatermark

从接口继承的方法 org.apache.flink.api.common.functions.RichFunction

open

字段详细资料

pendingCommitTransactions

protected final LinkedHashMap<Long,TwoPhaseCommitSinkFunction.TransactionHolder<TXN>> pendingCommitTransactions

已过时。

userContext

protected transient Optional<CONTEXT> userContext

已过时。

state

protected transient org.apache.flink.api.common.state.ListState<TwoPhaseCommitSinkFunction.State<TXN,CONTEXT>> state

已过时。

构造器详细资料

TwoPhaseCommitSinkFunction

public TwoPhaseCommitSinkFunction(org.apache.flink.api.common.typeutils.TypeSerializer<TXN> transactionSerializer,
                                  org.apache.flink.api.common.typeutils.TypeSerializer<CONTEXT> contextSerializer)

已过时。

Use default ListStateDescriptor for internal state serialization. Helpful utilities for using this constructor are TypeInformation.of(Class), TypeHint and TypeInformation.of(TypeHint). Example:

 TwoPhaseCommitSinkFunction(TypeInformation.of(new TypeHint<State<TXN, CONTEXT>>() {}));
 

参数:

transactionSerializer - TypeSerializer for the transaction type of this sink

contextSerializer - TypeSerializer for the context type of this sink

方法详细资料

initializeUserContext

protected Optional<CONTEXT> initializeUserContext()

已过时。

getUserContext

protected Optional<CONTEXT> getUserContext()

已过时。

currentTransaction

@Nullable
protected TXN currentTransaction()

已过时。

pendingTransactions

@Nonnull
protected Stream<Map.Entry<Long,TXN>> pendingTransactions()

已过时。

invoke

protected abstract void invoke(TXN transaction,
                               IN value,
                               SinkFunction.Context context)
                        throws Exception

已过时。

Write value within a transaction.

抛出:

Exception

beginTransaction

protected abstract TXN beginTransaction()
                                 throws Exception

已过时。

Method that starts a new transaction.

返回:

newly created transaction.

抛出:

Exception

preCommit

protected abstract void preCommit(TXN transaction)
                           throws Exception

已过时。

Pre commit previously created transaction. Pre commit must make all of the necessary steps to prepare the transaction for a commit that might happen in the future. After this point the transaction might still be aborted, but underlying implementation must ensure that commit calls on already pre committed transactions will always succeed.

Usually implementation involves flushing the data.

抛出:

Exception

commit

protected abstract void commit(TXN transaction)

已过时。

Commit a pre-committed transaction. If this method fail, Flink application will be restarted and recoverAndCommit(Object) will be called again for the same transaction.

recoverAndCommit

protected void recoverAndCommit(TXN transaction)

已过时。

Invoked on recovered transactions after a failure. User implementation must ensure that this call will eventually succeed. If it fails, Flink application will be restarted and it will be invoked again. If it does not succeed eventually, a data loss will occur. Transactions will be recovered in an order in which they were created.

abort

protected abstract void abort(TXN transaction)

已过时。

Abort a transaction.

recoverAndAbort

protected void recoverAndAbort(TXN transaction)

已过时。

Abort a transaction that was rejected by a coordinator after a failure.

finishRecoveringContext

protected void finishRecoveringContext(Collection<TXN> handledTransactions)

已过时。

Callback for subclasses which is called after restoring (each) user context.

参数:

handledTransactions - transactions which were already committed or aborted and do not need further handling

finishProcessing

protected void finishProcessing(@Nullable
                                TXN transaction)

已过时。

This method is called at the end of data processing.

The method is expected to flush all remaining buffered data. Exceptions will cause the pipeline to be recognized as failed, because the last data items are not processed properly. You may use this method to flush remaining buffered elements in the state into the current transaction which will be committed in the last checkpoint.

invoke

public final void invoke(IN value)
                  throws Exception

已过时。

This should not be implemented by subclasses.

指定者:

invoke 在接口中 SinkFunction<IN>

抛出:

Exception

invoke

public final void invoke(IN value,
                         SinkFunction.Context context)
                  throws Exception

已过时。

从接口复制的说明: SinkFunction

Writes the given value to the sink. This function is called for every record.

You have to override this method when implementing a SinkFunction, this is a default method for backward compatibility with the old-style method only.

指定者:

invoke 在接口中 SinkFunction<IN>

参数:

value - The input record.

context - Additional context about the input record.

抛出:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

finish

public final void finish()
                  throws Exception

已过时。

从接口复制的说明: SinkFunction

This method is called at the end of data processing.

The method is expected to flush all remaining buffered data. Exceptions will cause the pipeline to be recognized as failed, because the last data items are not processed properly. You may use this method to flush remaining buffered elements in the state into transactions which you can commit in the last checkpoint.

NOTE:This method does not need to close any resources. You should release external resources in the AbstractRichFunction.close() method.

指定者:

finish 在接口中 SinkFunction<IN>

抛出:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

notifyCheckpointComplete

public final void notifyCheckpointComplete(long checkpointId)
                                    throws Exception

已过时。

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.api.common.state.CheckpointListener

抛出:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)

已过时。

指定者:

notifyCheckpointAborted 在接口中 org.apache.flink.api.common.state.CheckpointListener

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

已过时。

从接口复制的说明: CheckpointedFunction

This method is called when a snapshot for a checkpoint is requested. This acts as a hook to the function to ensure that all state is exposed by means previously offered through FunctionInitializationContext when the Function was initialized, or offered now by FunctionSnapshotContext itself.

指定者:

snapshotState 在接口中 CheckpointedFunction

参数:

context - the context for drawing a snapshot of the operator

抛出:

Exception - Thrown, if state could not be created ot restored.

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

已过时。

从接口复制的说明: CheckpointedFunction

This method is called when the parallel function instance is created during distributed execution. Functions typically set up their state storing data structures in this method.

指定者:

initializeState 在接口中 CheckpointedFunction

参数:

context - the context for initializing the operator

抛出:

Exception - Thrown, if state could not be created ot restored.

close

public void close()
           throws Exception

已过时。

指定者:

close 在接口中 org.apache.flink.api.common.functions.RichFunction

覆盖:

close 在类中 org.apache.flink.api.common.functions.AbstractRichFunction

抛出:

Exception

setTransactionTimeout

protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT> setTransactionTimeout(long transactionTimeout)

已过时。

Sets the transaction timeout. Setting only the transaction timeout has no effect in itself.

参数:

transactionTimeout - The transaction timeout in ms.

另请参阅:

ignoreFailuresAfterTransactionTimeout(), enableTransactionTimeoutWarnings(double)

ignoreFailuresAfterTransactionTimeout

protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT> ignoreFailuresAfterTransactionTimeout()

已过时。

If called, the sink will only log but not propagate exceptions thrown in recoverAndCommit(Object) if the transaction is older than a specified transaction timeout. The start time of an transaction is determined by System.currentTimeMillis(). By default, failures are propagated.

enableTransactionTimeoutWarnings

protected TwoPhaseCommitSinkFunction<IN,TXN,CONTEXT> enableTransactionTimeoutWarnings(double warningRatio)

已过时。

Enables logging of warnings if a transaction's elapsed time reaches a specified ratio of the transactionTimeout. If warningRatio is 0, a warning will be always logged when committing the transaction.

参数:

warningRatio - A value in the range [0,1].

返回: