org.apache.flink.runtime.state.filesystem

Class FileMergingCheckpointStateOutputStream

java.lang.Object

java.io.OutputStream

org.apache.flink.core.fs.FSDataOutputStream

org.apache.flink.runtime.state.CheckpointStateOutputStream

org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.FsCheckpointStateOutputStream

org.apache.flink.runtime.state.filesystem.FileMergingCheckpointStateOutputStream

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

public class FileMergingCheckpointStateOutputStream
extends FsCheckpointStreamFactory.FsCheckpointStateOutputStream

A CheckpointStateOutputStream that writes into a segment of a file and returns a SegmentFileStateHandle upon closing. Multiple FileMergingCheckpointStateOutputStream objects can reuse the same underlying file, so that the checkpoint files are merged.

Important: This implementation is NOT thread-safe. Multiple data streams multiplexing the same file should NOT write concurrently. Instead, it is expected that only after one data stream is closed, will other data streams reuse and write to the same underlying file.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	FileMergingCheckpointStateOutputStream.FileMergingSnapshotManagerProxy A proxy of the FileMergingSnapshotManager that owns this FileMergingCheckpointStateOutputStream, with the interfaces for dealing with physical files.

Constructor Summary

Constructors

Constructor and Description
FileMergingCheckpointStateOutputStream(int bufferSize, FileMergingCheckpointStateOutputStream.FileMergingSnapshotManagerProxy fileMergingSnapshotManagerProxy)

Method Summary

Modifier and Type	Method and Description
void	close() This method throws no exception if the close fails, but only logs the error.
SegmentFileStateHandle	closeAndGetHandle() Closes the stream and gets a state handle that can create an input stream producing the data written to this stream.
void	flush() Flush buffers to file if their size is above FsCheckpointStreamFactory.FsCheckpointStateOutputStream.localStateThreshold.
void	flushToFile()
org.apache.flink.core.fs.Path	getFilePath()
long	getPos()
void	sync()
void	write(byte[] b, int off, int len)
void	write(int b)

Methods inherited from class org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory.FsCheckpointStateOutputStream

isClosed

Methods inherited from class java.io.OutputStream

write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileMergingCheckpointStateOutputStream

public FileMergingCheckpointStateOutputStream(int bufferSize,
                                              FileMergingCheckpointStateOutputStream.FileMergingSnapshotManagerProxy fileMergingSnapshotManagerProxy)

Method Detail

getPos

public long getPos()
            throws IOException

Overrides:

getPos in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

write

public void write(int b)
           throws IOException

Overrides:

write in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws IOException

Overrides:

write in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

flush

public void flush()
           throws IOException

Description copied from class: FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Flush buffers to file if their size is above FsCheckpointStreamFactory.FsCheckpointStateOutputStream.localStateThreshold.

Specified by:

flush in interface Flushable

Overrides:

flush in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

sync

public void sync()
          throws IOException

Overrides:

sync in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

closeAndGetHandle

@Nullable
public SegmentFileStateHandle closeAndGetHandle()
                                                   throws IOException

Description copied from class: CheckpointStateOutputStream

Closes the stream and gets a state handle that can create an input stream producing the data written to this stream.

This closing must be called (also when the caller is not interested in the handle) to successfully close the stream and retain the produced resource. In contrast, the CheckpointStateOutputStream.close() method removes the target resource when called.

Overrides:

closeAndGetHandle in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Returns:

A state handle that can create an input stream producing the data written to this stream.

Throws:

IOException - Thrown, if the stream cannot be closed.

close

public void close()

This method throws no exception if the close fails, but only logs the error. This is to be consistent with FsCheckpointStreamFactory.FsCheckpointStateOutputStream.close().

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Overrides:

close in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

flushToFile

public void flushToFile()
                 throws IOException

Overrides:

flushToFile in class FsCheckpointStreamFactory.FsCheckpointStateOutputStream

Throws:

IOException

getFilePath

@VisibleForTesting
 @Nullable
public org.apache.flink.core.fs.Path getFilePath()