org.apache.flink.runtime.fs.hdfs

Class BaseHadoopFsRecoverableFsDataOutputStream

java.lang.Object

java.io.OutputStream

org.apache.flink.core.fs.FSDataOutputStream

org.apache.flink.core.fs.RecoverableFsDataOutputStream

org.apache.flink.core.fs.CommitterFromPersistRecoverableFsDataOutputStream<HadoopFsRecoverable>

org.apache.flink.runtime.fs.hdfs.BaseHadoopFsRecoverableFsDataOutputStream

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

@Internal
public abstract class BaseHadoopFsRecoverableFsDataOutputStream
extends org.apache.flink.core.fs.CommitterFromPersistRecoverableFsDataOutputStream<HadoopFsRecoverable>

Base class for ABFS and Hadoop recoverable stream.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.RecoverableFsDataOutputStream

org.apache.flink.core.fs.RecoverableFsDataOutputStream.Committer

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.fs.FileSystem	fs
protected long	initialFileSize
protected org.apache.hadoop.fs.FSDataOutputStream	out
protected org.apache.hadoop.fs.Path	targetFile
protected org.apache.hadoop.fs.Path	tempFile

Constructor Summary

Constructors

Constructor and Description
BaseHadoopFsRecoverableFsDataOutputStream()

Method Summary

Modifier and Type	Method and Description
void	close()
HadoopFsRecoverable	createHadoopFsRecoverable(long pos)
void	flush()
long	getPos()
HadoopFsRecoverable	persist()
void	sync()
void	write(byte[] b, int off, int len)
void	write(int b)

Methods inherited from class org.apache.flink.core.fs.CommitterFromPersistRecoverableFsDataOutputStream

closeForCommit, createCommitterFromResumeRecoverable

Methods inherited from class java.io.OutputStream

write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fs

protected org.apache.hadoop.fs.FileSystem fs

targetFile

protected org.apache.hadoop.fs.Path targetFile

tempFile

protected org.apache.hadoop.fs.Path tempFile

out

protected org.apache.hadoop.fs.FSDataOutputStream out

initialFileSize

protected long initialFileSize

Constructor Detail

BaseHadoopFsRecoverableFsDataOutputStream

public BaseHadoopFsRecoverableFsDataOutputStream()

Method Detail

getPos

public long getPos()
            throws IOException

Specified by:

getPos in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

write

public void write(int b)
           throws IOException

Specified by:

write in class OutputStream

Throws:

IOException

write

public void write(byte[] b,
                  int off,
                  int len)
           throws IOException

Overrides:

write in class OutputStream

Throws:

IOException

flush

public void flush()
           throws IOException

Specified by:

flush in interface Flushable

Specified by:

flush in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

sync

public void sync()
          throws IOException

Specified by:

sync in class org.apache.flink.core.fs.FSDataOutputStream

Throws:

IOException

persist

public HadoopFsRecoverable persist()
                            throws IOException

Specified by:

persist in class org.apache.flink.core.fs.CommitterFromPersistRecoverableFsDataOutputStream<HadoopFsRecoverable>

Throws:

IOException

createHadoopFsRecoverable

public HadoopFsRecoverable createHadoopFsRecoverable(long pos)

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in class org.apache.flink.core.fs.RecoverableFsDataOutputStream

Throws:

IOException