org.apache.flink.orc

Class AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>

java.lang.Object

org.apache.flink.orc.AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>

Enclosing class:

AbstractOrcFileInputFormat<T,BatchT,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

protected abstract static class AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>
extends Object

The OrcReaderBatch class holds the data structures containing the batch data (column vectors, row arrays, ...) and performs the batch conversion from the ORC representation to the result format.

This base class only holds the ORC Column Vectors, subclasses hold additionally the result structures and implement the conversion in convertAndGetIterator(OrcVectorizedBatchWrapper, long).

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	OrcReaderBatch(OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch, org.apache.flink.connector.file.src.util.Pool.Recycler<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> recycler)

Method Summary

Modifier and Type	Method and Description
abstract org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T>	convertAndGetIterator(OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch, long startingOffset) Converts the ORC VectorizedRowBatch into the result structure and returns an iterator over the entries.
OrcVectorizedBatchWrapper<BatchT>	orcVectorizedRowBatch() Gets the ORC VectorizedRowBatch structure from this batch.
void	recycle() Puts this batch back into the pool.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OrcReaderBatch

protected OrcReaderBatch(OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch,
                         org.apache.flink.connector.file.src.util.Pool.Recycler<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> recycler)

Method Detail

recycle

public void recycle()

Puts this batch back into the pool. This should be called after all records from the batch have been returned, typically in the RecordIterator#releaseBatch() method.

orcVectorizedRowBatch

public OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch()

Gets the ORC VectorizedRowBatch structure from this batch.

convertAndGetIterator

public abstract org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T> convertAndGetIterator(OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch,
                                                                                                              long startingOffset)
                                                                                                       throws IOException

Converts the ORC VectorizedRowBatch into the result structure and returns an iterator over the entries.

This method may, for example, return a single element iterator that returns the entire batch as one, or (as another example) return an iterator over the rows projected from this column batch.

The position information in the result needs to be constructed as follows: The value of startingOffset is the offset value (RecordAndPosition.getOffset()) for all rows in the batch. Each row then increments the records-to-skip value (RecordAndPosition.getRecordSkipCount()).

Throws:

IOException