org.apache.hudi.common.table.log

Class HoodieLogFileReader

java.lang.Object

org.apache.hudi.common.table.log.HoodieLogFileReader

All Implemented Interfaces:

Closeable, AutoCloseable, Iterator<HoodieLogBlock>, HoodieLogFormat.Reader

public class HoodieLogFileReader
extends Object
implements HoodieLogFormat.Reader

Scans a log file and provides block level iterator on the log file Loads the entire block contents in memory Can emit either a DataBlock, CommandBlock, DeleteBlock or CorruptBlock (if one is found).

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_BUFFER_SIZE

Constructor Summary

Constructors

Constructor and Description
HoodieLogFileReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize)
HoodieLogFileReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean reverseReader)
HoodieLogFileReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean reverseReader, boolean enableRecordLookups, String keyField)
HoodieLogFileReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean reverseReader, boolean enableRecordLookups, String keyField, InternalSchema internalSchema)

Method Summary

Modifier and Type	Method and Description
void	close()
static SeekableDataInputStream	getDataInputStream(HoodieStorage storage, HoodieLogFile logFile, int bufferSize) Fetch the right SeekableDataInputStream to be used by wrapping with required input streams.
HoodieLogFile	getLogFile()
boolean	hasNext()
boolean	hasPrev() hasPrev is not idempotent.
long	moveToPrev() Reverse pointer, does not read the block.
HoodieLogBlock	next()
HoodieLogBlock	prev() This is a reverse iterator Note: At any point, an instance of HoodieLogFileReader should either iterate reverse (prev) or forward (next).
void	remove()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

DEFAULT_BUFFER_SIZE

public static final int DEFAULT_BUFFER_SIZE

See Also:

Constant Field Values

Constructor Detail

HoodieLogFileReader

public HoodieLogFileReader(HoodieStorage storage,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(HoodieStorage storage,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean reverseReader)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(HoodieStorage storage,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean reverseReader,
                           boolean enableRecordLookups,
                           String keyField)
                    throws IOException

Throws:

IOException

HoodieLogFileReader

public HoodieLogFileReader(HoodieStorage storage,
                           HoodieLogFile logFile,
                           org.apache.avro.Schema readerSchema,
                           int bufferSize,
                           boolean reverseReader,
                           boolean enableRecordLookups,
                           String keyField,
                           InternalSchema internalSchema)
                    throws IOException

Throws:

IOException

Method Detail

getLogFile

public HoodieLogFile getLogFile()

Specified by:

getLogFile in interface HoodieLogFormat.Reader

Returns:

the path to this HoodieLogFormat

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

hasNext

public boolean hasNext()

Specified by:

hasNext in interface Iterator<HoodieLogBlock>

next

public HoodieLogBlock next()

Specified by:

next in interface Iterator<HoodieLogBlock>

hasPrev

public boolean hasPrev()

hasPrev is not idempotent.

Specified by:

hasPrev in interface HoodieLogFormat.Reader

Returns:

prev

public HoodieLogBlock prev()
                    throws IOException

This is a reverse iterator Note: At any point, an instance of HoodieLogFileReader should either iterate reverse (prev) or forward (next). Doing both in the same instance is not supported WARNING : Every call to prev() should be preceded with hasPrev()

Specified by:

prev in interface HoodieLogFormat.Reader

Returns:

Throws:

IOException

moveToPrev

public long moveToPrev()
                throws IOException

Reverse pointer, does not read the block. Return the current position of the log file (in reverse) If the pointer (inputstream) is moved in any way, it is the job of the client of this class to seek/reset it back to the file position returned from the method to expect correct results

Throws:

IOException

remove

public void remove()

Specified by:

remove in interface Iterator<HoodieLogBlock>

getDataInputStream

public static SeekableDataInputStream getDataInputStream(HoodieStorage storage,
                                                         HoodieLogFile logFile,
                                                         int bufferSize)

Fetch the right SeekableDataInputStream to be used by wrapping with required input streams.

Parameters:

storage - instance of HoodieStorage in use.

logFile - the log file to read.

bufferSize - buffer size to be used.

Returns:

the right SeekableDataInputStream as required.