org.apache.flink.orc

Class AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT>

java.lang.Object

org.apache.flink.orc.AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT>

Type Parameters:

T - The type of the records returned by the reader.

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T>

Enclosing class:

AbstractOrcFileInputFormat<T,BatchT,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

protected static final class AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT>
extends Object
implements org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T>

A vectorized ORC reader. This reader reads an ORC BatchT at a time and converts it to one or more records to be returned. An ORC Row-wise reader would convert the batch into a set of rows, while a reader for a vectorized query processor might return the whole batch as one record.

The conversion of the VectorizedRowBatch happens in the specific AbstractOrcFileInputFormat.OrcReaderBatch implementation.

The reader tracks its current position using ORC's row numbers. Each record in a batch is addressed by the starting row number of the batch, plus the number of records to be skipped before.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	OrcVectorizedReader(OrcShim<BatchT> shim, org.apache.orc.RecordReader orcReader, org.apache.flink.connector.file.src.util.Pool<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> pool)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T>	readBatch()
void	seek(org.apache.flink.connector.file.src.util.CheckpointedPosition position) The argument of RecordReader.seekToRow(long) must come from RecordReader.getRowNumber().

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OrcVectorizedReader

protected OrcVectorizedReader(OrcShim<BatchT> shim,
                              org.apache.orc.RecordReader orcReader,
                              org.apache.flink.connector.file.src.util.Pool<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> pool)

Method Detail

readBatch

@Nullable
public org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T> readBatch()
                                                                                            throws IOException

Specified by:

readBatch in interface org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T>

Throws:

IOException

seek

public void seek(org.apache.flink.connector.file.src.util.CheckpointedPosition position)
          throws IOException

The argument of RecordReader.seekToRow(long) must come from RecordReader.getRowNumber(). The internal implementation of ORC is very confusing. It has special behavior when dealing with Predicate.

Throws:

IOException