org.apache.flink.connector.file.table.stream

类 AbstractStreamingWriter<IN,OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.connector.file.table.stream.AbstractStreamingWriter<IN,OUT>

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

直接已知子类:

CompactFileWriter, StreamingFileWriter

public abstract class AbstractStreamingWriter<IN,OUT>
extends org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,OUT>, org.apache.flink.streaming.api.operators.BoundedOneInput

Operator for file system sink. It is a operator version of StreamingFileSink. It can send file and bucket information to downstream.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.streaming.api.functions.sink.filesystem.Buckets<IN,String>	buckets
protected long	currentWatermark

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

构造器概要

构造器

构造器和说明
AbstractStreamingWriter(long bucketCheckInterval, org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<IN,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder)

方法概要

限定符和类型	方法和说明
void	close()
protected void	commitUpToCheckpoint(long checkpointId) Commit up to this checkpoint id.
void	endInput()
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
void	notifyCheckpointComplete(long checkpointId)
protected abstract void	onPartFileOpened(String partition, org.apache.flink.core.fs.Path newPath) Notifies a new file has been opened.
protected abstract void	partitionCreated(String partition) Notifies a partition created.
protected abstract void	partitionInactive(String partition) Notifies a partition become inactive.
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
void	processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
void	snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, open, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorID, initializeState, open, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

从接口继承的方法 org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processWatermarkStatus

字段详细资料

buckets

protected transient org.apache.flink.streaming.api.functions.sink.filesystem.Buckets<IN,String> buckets

currentWatermark

protected transient long currentWatermark

构造器详细资料

AbstractStreamingWriter

public AbstractStreamingWriter(long bucketCheckInterval,
                               org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<IN,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.StreamingFileSink.BucketsBuilder<IN,String,?>> bucketsBuilder)

方法详细资料

partitionCreated

protected abstract void partitionCreated(String partition)

Notifies a partition created.

partitionInactive

protected abstract void partitionInactive(String partition)

Notifies a partition become inactive. A partition becomes inactive after all the records received so far have been committed.

onPartFileOpened

protected abstract void onPartFileOpened(String partition,
                                         org.apache.flink.core.fs.Path newPath)

Notifies a new file has been opened.

Note that this does not mean that the file has been created in the file system. It is only created logically and the actual file will be generated after it is committed.

commitUpToCheckpoint

protected void commitUpToCheckpoint(long checkpointId)
                             throws Exception

Commit up to this checkpoint id.

抛出:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

指定者:

initializeState 在接口中 org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

initializeState 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception

指定者:

snapshotState 在接口中 org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

snapshotState 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

抛出:

Exception

processWatermark

public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception

指定者:

processWatermark 在接口中 org.apache.flink.streaming.api.operators.Input<IN>

覆盖:

processWatermark 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

抛出:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
                    throws Exception

指定者:

processElement 在接口中 org.apache.flink.streaming.api.operators.Input<IN>

抛出:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.api.common.state.CheckpointListener

覆盖:

notifyCheckpointComplete 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

抛出:

Exception

endInput

public void endInput()
              throws Exception

指定者:

endInput 在接口中 org.apache.flink.streaming.api.operators.BoundedOneInput

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<OUT>

覆盖:

close 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

抛出:

Exception