org.apache.flink.orc.shim

Class OrcShimV200

java.lang.Object

org.apache.flink.orc.shim.OrcShimV200

All Implemented Interfaces:

Serializable, OrcShim<org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch>

Direct Known Subclasses:

OrcShimV210

public class OrcShimV200
extends Object
implements OrcShim<org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch>

Shim orc for Hive version 2.0.0 and upper versions.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
OrcShimV200()

Method Summary

Modifier and Type	Method and Description
static boolean[]	computeProjectionMask(org.apache.orc.TypeDescription schema, int[] selectedFields) Computes the ORC projection mask of the fields to include from the selected fields.rowOrcInputFormat.nextRecord(null).
HiveOrcBatchWrapper	createBatchWrapper(org.apache.orc.TypeDescription schema, int batchSize)
protected org.apache.orc.Reader	createReader(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf)
org.apache.orc.RecordReader	createRecordReader(org.apache.hadoop.conf.Configuration conf, org.apache.orc.TypeDescription schema, int[] selectedFields, List<OrcFilters.Predicate> conjunctPredicates, org.apache.flink.core.fs.Path path, long splitStart, long splitLength) Create orc RecordReader from conf, schema and etc...
protected org.apache.orc.RecordReader	createRecordReader(org.apache.orc.Reader reader, org.apache.orc.Reader.Options options)
static org.apache.flink.api.java.tuple.Tuple2<Long,Long>	getOffsetAndLengthForSplit(long splitStart, long splitLength, List<org.apache.orc.StripeInformation> stripes)
boolean	nextBatch(org.apache.orc.RecordReader reader, org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch rowBatch) Read the next row batch.
protected org.apache.orc.Reader.Options	readOrcConf(org.apache.orc.Reader.Options options, org.apache.hadoop.conf.Configuration conf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.orc.shim.OrcShim

createShim, defaultShim

Constructor Detail

OrcShimV200

public OrcShimV200()

Method Detail

createReader

protected org.apache.orc.Reader createReader(org.apache.hadoop.fs.Path path,
                                             org.apache.hadoop.conf.Configuration conf)
                                      throws IOException

Throws:

IOException

createRecordReader

protected org.apache.orc.RecordReader createRecordReader(org.apache.orc.Reader reader,
                                                         org.apache.orc.Reader.Options options)
                                                  throws IOException

Throws:

IOException

readOrcConf

protected org.apache.orc.Reader.Options readOrcConf(org.apache.orc.Reader.Options options,
                                                    org.apache.hadoop.conf.Configuration conf)

createRecordReader

public org.apache.orc.RecordReader createRecordReader(org.apache.hadoop.conf.Configuration conf,
                                                      org.apache.orc.TypeDescription schema,
                                                      int[] selectedFields,
                                                      List<OrcFilters.Predicate> conjunctPredicates,
                                                      org.apache.flink.core.fs.Path path,
                                                      long splitStart,
                                                      long splitLength)
                                               throws IOException

Description copied from interface: OrcShim

Create orc RecordReader from conf, schema and etc...

Specified by:

createRecordReader in interface OrcShim<org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch>

Throws:

IOException

createBatchWrapper

public HiveOrcBatchWrapper createBatchWrapper(org.apache.orc.TypeDescription schema,
                                              int batchSize)

Specified by:

createBatchWrapper in interface OrcShim<org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch>

nextBatch

public boolean nextBatch(org.apache.orc.RecordReader reader,
                         org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch rowBatch)
                  throws IOException

Description copied from interface: OrcShim

Read the next row batch.

Specified by:

nextBatch in interface OrcShim<org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch>

Throws:

IOException

getOffsetAndLengthForSplit

@VisibleForTesting
public static org.apache.flink.api.java.tuple.Tuple2<Long,Long> getOffsetAndLengthForSplit(long splitStart,
                                                                                                              long splitLength,
                                                                                                              List<org.apache.orc.StripeInformation> stripes)

computeProjectionMask

public static boolean[] computeProjectionMask(org.apache.orc.TypeDescription schema,
                                              int[] selectedFields)

Computes the ORC projection mask of the fields to include from the selected fields.rowOrcInputFormat.nextRecord(null).

Returns:

The ORC projection mask.