org.apache.flink.runtime.io.network.partition.hybrid.tiered.file

类 SegmentPartitionFileWriter

java.lang.Object

org.apache.flink.runtime.io.network.partition.hybrid.tiered.file.SegmentPartitionFileWriter

所有已实现的接口:

PartitionFileWriter

public class SegmentPartitionFileWriter
extends Object
implements PartitionFileWriter

The implementation of PartitionFileWriter with segment file mode. In this mode, each segment of one subpartition is written to an independent file.

After finishing writing a segment, a segment-finish file is written to ensure the downstream reads only when the entire segment file is written, avoiding partial data reads. The downstream can determine if the current segment is complete by checking for the existence of the segment-finish file.

To minimize the number of files, each subpartition keeps only a single segment-finish file. For instance, if segment-finish file 5 exists, it indicates that segments 1 to 5 have all been finished.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.runtime.io.network.partition.hybrid.tiered.file.PartitionFileWriter

PartitionFileWriter.SegmentBufferContext, PartitionFileWriter.SubpartitionBufferContext

方法概要

限定符和类型	方法和说明
void	release() Release all the resources of the PartitionFileWriter.
CompletableFuture<Void>	write(TieredStoragePartitionId partitionId, List<PartitionFileWriter.SubpartitionBufferContext> buffersToWrite) Write the buffers to the partition file.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

方法详细资料

write

public CompletableFuture<Void> write(TieredStoragePartitionId partitionId,
                                     List<PartitionFileWriter.SubpartitionBufferContext> buffersToWrite)

从接口复制的说明: PartitionFileWriter

Write the buffers to the partition file. The written buffers may belong to multiple subpartitions, but these buffers will be consecutive in the file.

指定者:

write 在接口中 PartitionFileWriter

参数:

partitionId - the partition id

buffersToWrite - the buffers to be written to the partition file

返回:

the completable future indicating whether the writing file process has finished. If the CompletableFuture is completed, the written process is completed.

release

public void release()

从接口复制的说明: PartitionFileWriter

Release all the resources of the PartitionFileWriter.

指定者:

release 在接口中 PartitionFileWriter