Package org.apache.flink.runtime.io.network.partition

Class PartitionedFileWriter

java.lang.Object

org.apache.flink.runtime.io.network.partition.PartitionedFileWriter

All Implemented Interfaces:

AutoCloseable

@NotThreadSafe
public class PartitionedFileWriter
extends Object
implements AutoCloseable

File writer which can write buffers and generate PartitionedFile. Data is written region by region. Before writing a new region, the method startNewRegion(boolean) must be called. After writing all data, the method finish() must be called to close all opened files and return the target PartitionedFile.

Constructor Summary

Constructors

Constructor	Description
PartitionedFileWriter(int numSubpartitions, int maxIndexBufferSize, String basePath, int[] writeOrder)

Method Summary

Modifier and Type	Method	Description
void	close()
PartitionedFile	finish()	Finishes writing the PartitionedFile which closes the file channel and returns the corresponding PartitionedFile.
void	releaseQuietly()	Used to close and delete the failed PartitionedFile when any exception occurs.
void	startNewRegion(boolean isBroadcastRegion)	Persists the region index of the current data region and starts a new region to write.
void	writeBuffers(List<BufferWithSubpartition> bufferWithSubpartitions)	Writes a list of Buffers to this PartitionedFile.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionedFileWriter

public PartitionedFileWriter(int numSubpartitions,
                             int maxIndexBufferSize,
                             String basePath,
                             int[] writeOrder)
                      throws IOException

Throws:

IOException

Method Detail

startNewRegion

public void startNewRegion(boolean isBroadcastRegion)
                    throws IOException

Persists the region index of the current data region and starts a new region to write.

Note: The caller is responsible for releasing the failed PartitionedFile if any exception occurs.

Parameters:

isBroadcastRegion - Whether it's a broadcast region. See isBroadcastRegion.

Throws:

IOException

writeBuffers

public void writeBuffers(List<BufferWithSubpartition> bufferWithSubpartitions)
                  throws IOException

Writes a list of Buffers to this PartitionedFile. It guarantees that after the return of this method, the target buffers can be released. In a data region, all data of the same subpartition must be written together.

Note: The caller is responsible for recycling the target buffers and releasing the failed PartitionedFile if any exception occurs.

Throws:

IOException

finish

public PartitionedFile finish()
                       throws IOException

Finishes writing the PartitionedFile which closes the file channel and returns the corresponding PartitionedFile.

Note: The caller is responsible for releasing the failed PartitionedFile if any exception occurs.

Throws:

IOException

releaseQuietly

public void releaseQuietly()

Used to close and delete the failed PartitionedFile when any exception occurs.

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Throws:

IOException