org.apache.flink.runtime.checkpoint.filemerging

Class PhysicalFilePool

java.lang.Object

org.apache.flink.runtime.checkpoint.filemerging.PhysicalFilePool

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

BlockingPhysicalFilePool, NonBlockingPhysicalFilePool

public abstract class PhysicalFilePool
extends Object
implements Closeable

A pool for reusing PhysicalFile. This implementation should be thread-safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PhysicalFilePool.Type Types of supported physical file pool.

Field Summary

Fields

Modifier and Type	Field and Description
protected Queue<PhysicalFile>	exclusivePhysicalFilePool Queue maintaining exclusive physical files for reusing.
protected long	maxFileSize Max size for a physical file.
protected PhysicalFile.PhysicalFileCreator	physicalFileCreator creator to create a physical file.
protected Map<FileMergingSnapshotManager.SubtaskKey,Queue<PhysicalFile>>	sharedPhysicalFilePoolBySubtask Map maintaining queues of different subtasks for reusing shared physical files.

Constructor Summary

Constructors

Constructor and Description
PhysicalFilePool(long maxFileSize, PhysicalFile.PhysicalFileCreator physicalFileCreator)

Method Summary

Modifier and Type	Method and Description
void	close()
void	close(FileMergingSnapshotManager.SubtaskKey subtaskKey) Close files in shared file pool by subtaskKey and close all files in exclusive file pool.
protected abstract Queue<PhysicalFile>	createFileQueue() Create and return a file queue.
protected Queue<PhysicalFile>	getFileQueue(FileMergingSnapshotManager.SubtaskKey subtaskKey, CheckpointedStateScope scope) Get or create a file queue for specific subtaskKey and checkpoint scope.
boolean	isEmpty() Return whether the pool is empty or not.
abstract PhysicalFile	pollFile(FileMergingSnapshotManager.SubtaskKey subtaskKey, CheckpointedStateScope scope) Poll a physical file from the pool.
abstract boolean	tryPutFile(FileMergingSnapshotManager.SubtaskKey subtaskKey, PhysicalFile physicalFile) Try to put a physical file into file pool.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

physicalFileCreator

protected final PhysicalFile.PhysicalFileCreator physicalFileCreator

creator to create a physical file.

maxFileSize

protected final long maxFileSize

Max size for a physical file.

sharedPhysicalFilePoolBySubtask

protected final Map<FileMergingSnapshotManager.SubtaskKey,Queue<PhysicalFile>> sharedPhysicalFilePoolBySubtask

Map maintaining queues of different subtasks for reusing shared physical files.

exclusivePhysicalFilePool

protected final Queue<PhysicalFile> exclusivePhysicalFilePool

Queue maintaining exclusive physical files for reusing.

Constructor Detail

PhysicalFilePool

public PhysicalFilePool(long maxFileSize,
                        PhysicalFile.PhysicalFileCreator physicalFileCreator)

Method Detail

tryPutFile

public abstract boolean tryPutFile(FileMergingSnapshotManager.SubtaskKey subtaskKey,
                                   PhysicalFile physicalFile)
                            throws IOException

Try to put a physical file into file pool.

Parameters:

subtaskKey - the key of current subtask.

physicalFile - target physical file.

Returns:

true if file is in the pool, false otherwise.

Throws:

IOException

pollFile

@Nonnull
public abstract PhysicalFile pollFile(FileMergingSnapshotManager.SubtaskKey subtaskKey,
                                               CheckpointedStateScope scope)
                                        throws IOException

Poll a physical file from the pool.

Parameters:

subtaskKey - the key of current subtask.

scope - the scope of the checkpoint.

Returns:

a physical file.

Throws:

IOException

createFileQueue

protected abstract Queue<PhysicalFile> createFileQueue()

Create and return a file queue.

Returns:

a created file queue.

getFileQueue

protected Queue<PhysicalFile> getFileQueue(FileMergingSnapshotManager.SubtaskKey subtaskKey,
                                           CheckpointedStateScope scope)

Get or create a file queue for specific subtaskKey and checkpoint scope.

Parameters:

subtaskKey - the key of current subtask.

scope - the scope of the checkpoint.

Returns:

an existing or created file queue.

isEmpty

public boolean isEmpty()

Return whether the pool is empty or not.

Returns:

whether the pool is empty or not.

close

public void close(FileMergingSnapshotManager.SubtaskKey subtaskKey)
           throws IOException

Close files in shared file pool by subtaskKey and close all files in exclusive file pool.

Parameters:

subtaskKey - the key of current subtask.

Throws:

IOException - if anything goes wrong when closing files.

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException