org.apache.flink.orc

类 AbstractOrcFileInputFormat<T,BatchT,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

java.lang.Object

org.apache.flink.orc.AbstractOrcFileInputFormat<T,BatchT,SplitT>

类型参数:

T - The type of records produced by this reader format.

所有已实现的接口:

Serializable, org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>, org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT>

直接已知子类:

OrcColumnarRowInputFormat

public abstract class AbstractOrcFileInputFormat<T,BatchT,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>
extends Object
implements org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT>

The base for ORC readers for the FileSource. Implements the reader initialization, vectorized reading, and pooling of column vector objects.

Subclasses implement the conversion to the specific result record(s) that they return by creating via extending AbstractOrcFileInputFormat.OrcReaderBatch.

另请参阅:

序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
protected static class	AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT> The OrcReaderBatch class holds the data structures containing the batch data (column vectors, row arrays, ...) and performs the batch conversion from the ORC representation to the result format.
protected static class	AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT> A vectorized ORC reader.

从接口继承的嵌套类/接口 org.apache.flink.connector.file.src.reader.BulkFormat

org.apache.flink.connector.file.src.reader.BulkFormat.Reader<T>, org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T>

字段概要

字段

限定符和类型	字段和说明
protected int	batchSize
protected List<OrcFilters.Predicate>	conjunctPredicates
protected SerializableHadoopConfigWrapper	hadoopConfigWrapper
protected org.apache.orc.TypeDescription	schema
protected int[]	selectedFields
protected OrcShim<BatchT>	shim

构造器概要

构造器

限定符	构造器和说明
protected	AbstractOrcFileInputFormat(OrcShim<BatchT> shim, org.apache.hadoop.conf.Configuration hadoopConfig, org.apache.orc.TypeDescription schema, int[] selectedFields, List<OrcFilters.Predicate> conjunctPredicates, int batchSize)

方法概要

限定符和类型	方法和说明
AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT>	createReader(org.apache.flink.configuration.Configuration config, SplitT split)
abstract AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>	createReaderBatch(SplitT split, OrcVectorizedBatchWrapper<BatchT> orcBatch, org.apache.flink.connector.file.src.util.Pool.Recycler<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> recycler, int batchSize) Creates the AbstractOrcFileInputFormat.OrcReaderBatch structure, which is responsible for holding the data structures that hold the batch data (column vectors, row arrays, ...) and the batch conversion from the ORC representation to the result format.
abstract org.apache.flink.api.common.typeinfo.TypeInformation<T>	getProducedType() Gets the type produced by this format.
boolean	isSplittable()
AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT>	restoreReader(org.apache.flink.configuration.Configuration config, SplitT split)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

shim

protected final OrcShim<BatchT> shim

hadoopConfigWrapper

protected final SerializableHadoopConfigWrapper hadoopConfigWrapper

schema

protected final org.apache.orc.TypeDescription schema

selectedFields

protected final int[] selectedFields

conjunctPredicates

protected final List<OrcFilters.Predicate> conjunctPredicates

batchSize

protected final int batchSize

构造器详细资料

AbstractOrcFileInputFormat

protected AbstractOrcFileInputFormat(OrcShim<BatchT> shim,
                                     org.apache.hadoop.conf.Configuration hadoopConfig,
                                     org.apache.orc.TypeDescription schema,
                                     int[] selectedFields,
                                     List<OrcFilters.Predicate> conjunctPredicates,
                                     int batchSize)

参数:

shim - the shim for various Orc dependent versions. If you use the latest version, please use OrcShim.defaultShim() directly.

hadoopConfig - the hadoop config for orc reader.

schema - the full schema of orc format.

selectedFields - the read selected field of orc format.

conjunctPredicates - the filter predicates that can be evaluated.

batchSize - the batch size of orc reader.

方法详细资料

createReader

public AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT> createReader(org.apache.flink.configuration.Configuration config,
                                                                             SplitT split)
                                                                      throws IOException

指定者:

createReader 在接口中 org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

抛出:

IOException

restoreReader

public AbstractOrcFileInputFormat.OrcVectorizedReader<T,BatchT> restoreReader(org.apache.flink.configuration.Configuration config,
                                                                              SplitT split)
                                                                       throws IOException

指定者:

restoreReader 在接口中 org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

抛出:

IOException

isSplittable

public boolean isSplittable()

指定者:

isSplittable 在接口中 org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>

createReaderBatch

public abstract AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT> createReaderBatch(SplitT split,
                                                                                      OrcVectorizedBatchWrapper<BatchT> orcBatch,
                                                                                      org.apache.flink.connector.file.src.util.Pool.Recycler<AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>> recycler,
                                                                                      int batchSize)

Creates the AbstractOrcFileInputFormat.OrcReaderBatch structure, which is responsible for holding the data structures that hold the batch data (column vectors, row arrays, ...) and the batch conversion from the ORC representation to the result format.

getProducedType

public abstract org.apache.flink.api.common.typeinfo.TypeInformation<T> getProducedType()

Gets the type produced by this format.

指定者:

getProducedType 在接口中 org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>

指定者:

getProducedType 在接口中 org.apache.flink.connector.file.src.reader.BulkFormat<T,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>