org.apache.flink.connector.file.sink.writer

类 FileWriter<IN>

java.lang.Object

org.apache.flink.connector.file.sink.writer.FileWriter<IN>

类型参数:

IN - The type of input elements.

所有已实现的接口:

AutoCloseable, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback, org.apache.flink.api.connector.sink2.SinkWriter<IN>, org.apache.flink.api.connector.sink2.StatefulSink.StatefulSinkWriter<IN,FileWriterBucketState>, org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink.PrecommittingSinkWriter<IN,FileSinkCommittable>

@Internal
public class FileWriter<IN>
extends Object
implements org.apache.flink.api.connector.sink2.StatefulSink.StatefulSinkWriter<IN,FileWriterBucketState>, org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink.PrecommittingSinkWriter<IN,FileSinkCommittable>, org.apache.flink.api.connector.sink2.SinkWriter<IN>, org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback

A SinkWriter implementation for FileSink.

It writes data to and manages the different active buckes in the FileSink.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.api.connector.sink2.SinkWriter

org.apache.flink.api.connector.sink2.SinkWriter.Context

构造器概要

构造器

构造器和说明

FileWriter(org.apache.flink.core.fs.Path basePath, org.apache.flink.metrics.groups.SinkWriterMetricGroup metricGroup, org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,String> bucketAssigner, FileWriterBucketFactory<IN> bucketFactory, org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter<IN,String> bucketWriter, org.apache.flink.streaming.api.functions.sink.filesystem.RollingPolicy<IN,String> rollingPolicy, org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig, org.apache.flink.api.common.operators.ProcessingTimeService processingTimeService, long bucketCheckInterval) A constructor creating a new empty bucket manager.

方法概要

限定符和类型	方法和说明
void	close()
void	flush(boolean endOfInput)
void	initializeState(Collection<FileWriterBucketState> bucketStates) Initializes the state after recovery from a failure.
void	onProcessingTime(long time)
Collection<FileSinkCommittable>	prepareCommit()
List<FileWriterBucketState>	snapshotState(long checkpointId)
void	write(IN element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.api.connector.sink2.SinkWriter

writeWatermark

构造器详细资料

FileWriter

public FileWriter(org.apache.flink.core.fs.Path basePath,
                  org.apache.flink.metrics.groups.SinkWriterMetricGroup metricGroup,
                  org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,String> bucketAssigner,
                  FileWriterBucketFactory<IN> bucketFactory,
                  org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter<IN,String> bucketWriter,
                  org.apache.flink.streaming.api.functions.sink.filesystem.RollingPolicy<IN,String> rollingPolicy,
                  org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig,
                  org.apache.flink.api.common.operators.ProcessingTimeService processingTimeService,
                  long bucketCheckInterval)

A constructor creating a new empty bucket manager.

参数:

basePath - The base path for our buckets.

metricGroup - SinkWriterMetricGroup to set sink writer specific metrics.

bucketAssigner - The BucketAssigner provided by the user.

bucketFactory - The FileWriterBucketFactory to be used to create buckets.

bucketWriter - The BucketWriter to be used when writing data.

rollingPolicy - The RollingPolicy as specified by the user.

方法详细资料

initializeState

public void initializeState(Collection<FileWriterBucketState> bucketStates)
                     throws IOException

Initializes the state after recovery from a failure.

During this process:

1. we set the initial value for part counter to the maximum value used before across all tasks and buckets. This guarantees that we do not overwrite valid data,
2. we commit any pending files for previous checkpoints (previous to the last successful one from which we restore),
3. we resume writing to the previous in-progress file of each bucket, and
4. if we receive multiple states for the same bucket, we merge them.

参数:

bucketStates - the state holding recovered state about active buckets.

抛出:

IOException - if anything goes wrong during retrieving the state or restoring/committing of any in-progress/pending part files

write

public void write(IN element,
                  org.apache.flink.api.connector.sink2.SinkWriter.Context context)
           throws IOException,
                  InterruptedException

指定者:

write 在接口中 org.apache.flink.api.connector.sink2.SinkWriter<IN>

抛出:

IOException

InterruptedException

flush

public void flush(boolean endOfInput)
           throws IOException,
                  InterruptedException

指定者:

flush 在接口中 org.apache.flink.api.connector.sink2.SinkWriter<IN>

抛出:

IOException

InterruptedException

prepareCommit

public Collection<FileSinkCommittable> prepareCommit()
                                              throws IOException

指定者:

prepareCommit 在接口中 org.apache.flink.api.connector.sink2.TwoPhaseCommittingSink.PrecommittingSinkWriter<IN,FileSinkCommittable>

抛出:

IOException

snapshotState

public List<FileWriterBucketState> snapshotState(long checkpointId)
                                          throws IOException

指定者:

snapshotState 在接口中 org.apache.flink.api.connector.sink2.StatefulSink.StatefulSinkWriter<IN,FileWriterBucketState>

抛出:

IOException

close

public void close()

指定者:

close 在接口中 AutoCloseable

onProcessingTime

public void onProcessingTime(long time)
                      throws IOException

指定者:

onProcessingTime 在接口中 org.apache.flink.api.common.operators.ProcessingTimeService.ProcessingTimeCallback

抛出:

IOException