Class OutputStreamBasedPartFileWriter<IN,BucketID>
- java.lang.Object
-
- org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter<IN,BucketID>
-
- org.apache.flink.streaming.api.functions.sink.filesystem.OutputStreamBasedPartFileWriter<IN,BucketID>
-
- Type Parameters:
IN- the element typeBucketID- the bucket type
- All Implemented Interfaces:
CompactingFileWriter,InProgressFileWriter<IN,BucketID>,OutputStreamBasedCompactingFileWriter,PartFileInfo<BucketID>,RecordWiseCompactingFileWriter<IN>
- Direct Known Subclasses:
RowWisePartWriter
public abstract class OutputStreamBasedPartFileWriter<IN,BucketID> extends AbstractPartFileWriter<IN,BucketID> implements OutputStreamBasedCompactingFileWriter
The base class for all the part file writer that useRecoverableFsDataOutputStream.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classOutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableTheInProgressFileWriter.InProgressFileRecoverableimplementation forOutputStreamBasedPartFileWriter.OutputStreamBasedBucketWriter.static classOutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverableSerializerThe serializer forOutputStreamBasedPartFileWriter.OutputStreamBasedInProgressFileRecoverable.static classOutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableTheInProgressFileWriter.PendingFileRecoverableimplementation forOutputStreamBasedPartFileWriter.OutputStreamBasedBucketWriter.static classOutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverableSerializerThe serializer forOutputStreamBasedPartFileWriter.OutputStreamBasedPendingFileRecoverable.-
Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.CompactingFileWriter
CompactingFileWriter.Type
-
Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter
InProgressFileWriter.InProgressFileRecoverable, InProgressFileWriter.PendingFileRecoverable
-
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description OutputStreamasOutputStream()Gets the output stream underlying the writer.InProgressFileWriter.PendingFileRecoverablecloseForCommit()Closes the writer and gets theInProgressFileWriter.PendingFileRecoverableof the written compacting file.voiddispose()Dispose the part file.protected voidensureWriteType(CompactingFileWriter.Type type)longgetSize()InProgressFileWriter.InProgressFileRecoverablepersist()-
Methods inherited from class org.apache.flink.streaming.api.functions.sink.filesystem.AbstractPartFileWriter
getBucketId, getCreationTime, getLastUpdateTime, markWrite
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.flink.streaming.api.functions.sink.filesystem.InProgressFileWriter
write, write
-
-
-
-
Method Detail
-
persist
public InProgressFileWriter.InProgressFileRecoverable persist() throws IOException
- Specified by:
persistin interfaceInProgressFileWriter<IN,BucketID>- Returns:
- The state of the current part file.
- Throws:
IOException- Thrown if persisting the part file fails.
-
closeForCommit
public InProgressFileWriter.PendingFileRecoverable closeForCommit() throws IOException
Description copied from interface:CompactingFileWriterCloses the writer and gets theInProgressFileWriter.PendingFileRecoverableof the written compacting file.- Specified by:
closeForCommitin interfaceCompactingFileWriter- Specified by:
closeForCommitin interfaceInProgressFileWriter<IN,BucketID>- Returns:
- The state of the pending part file.
Bucketuses this to commit the pending file. - Throws:
IOException- Thrown if an I/O error occurs.
-
dispose
public void dispose()
Description copied from interface:InProgressFileWriterDispose the part file.- Specified by:
disposein interfaceInProgressFileWriter<IN,BucketID>
-
getSize
public long getSize() throws IOException- Specified by:
getSizein interfacePartFileInfo<IN>- Returns:
- The size of the currently open part file.
- Throws:
IOException
-
asOutputStream
public OutputStream asOutputStream() throws IOException
Description copied from interface:OutputStreamBasedCompactingFileWriterGets the output stream underlying the writer. The close method of the returned stream should never be called.- Specified by:
asOutputStreamin interfaceOutputStreamBasedCompactingFileWriter- Returns:
- The output stream to write the compacting file.
- Throws:
IOException- Thrown if acquiring the stream fails.
-
ensureWriteType
protected void ensureWriteType(CompactingFileWriter.Type type)
-
-