All Classes and Interfaces

Class
Description
The purpose of this is to finally generate a bloom filter with the optimal bit size according to the number of real data distinct values.
Provides ability to chain two filters together.
Deprecated. 
 
Plain encoding reader for BINARY values.
 
Class for truncating min/max values for binary types.
 
a column reader that packs the ints in the number of bits required based on the maximum size.
a column writer that packs the ints in the number of bits required based on the maximum size.
 
A Bloom filter is a compact structure to indicate whether an item is not in a set or probably in a set.
 
 
 
 
Contains all writers for all columns of a row group
Decodes PLAIN-encoded booleans: one bit per value, packed 8 per byte, little-endian bit order (bit 0 of each byte is the first value).
An implementation of the PLAIN encoding for BOOLEAN values.
 
 
Enum for BoundaryOrder.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Describes a column's type as well as its position in its containing schema.
Column index containing min/max and null count values for the pages in a column chunk.
Builder implementation to create ColumnIndex objects.
Filter implementation based on column indexes.
Provides the ColumnIndex and OffsetIndex objects for a row-group.
Exception thrown in case of an offset index is missing for any of the columns.
a structure used to serialize deserialize records
Factory constructing the ColumnIO structure from the schema
Class representing the column order with all the related parameters.
The enum type of the column order.
ColumnPredicates class provides checks for column values.
 
 
 
 
 
 
 
Reader for (repetition level, definition level, values) triplets.
ColumnReader implementation for the scenario when column indexes are not used (all values are read)
Container which can produce a ColumnReader for any given column in a schema.
Implementation of the ColumnReadStore
Record filter which applies the supplied predicate to the specified column.
writer for (repetition level, definition level, values) triplets
Container which can construct writers for multiple columns to be stored together.
 
 
thrown when a problem occured while compiling the column reader
Recursively rewrites Contains predicates composed using And or Or into a single Contains predicate containing all predicate assertions.
Utility functions to convert from Java-like map and list types to equivalent Parquet types.
Represent a tree of converters that materializes tuples
 
There was a bug (https://github.com/apache/parquet-java/issues/1433) that caused the statistics metadata for binary columns to be corrupted in the write path.
one data page in a chunk
 
 
 
Deprecated.
use LogicalTypeAnnotation.DecimalLogicalTypeAnnotation with proper precision and scale parameters instead
 
 
Handles ValuesWriter creation statically based on the types of the columns and the writer version.
Read values written by DeltaBinaryPackingValuesWriter
Write integers with delta encoding and binary packing The format is as follows:
Write integers (INT32) with delta encoding and binary packing.
Write longs (INT64) with delta encoding and binary packing.
Reads binary data written by DeltaByteArrayWriter
Write prefix lengths using delta encoding, followed by suffixes with Delta length byte arrays
Reads binary data written by DeltaLengthByteArrayValuesWriter
Write lengths of byte-arrays using delta encoding, followed by concatenated byte-arrays
This is a special writer that doesn't write anything.
a dictionary to decode dictionary based encodings
Data for a dictionary page
Interface to read dictionary pages for all the columns of a row group
Reads values that have been dictionary encoded
Will attempt to encode values using a dictionary and fall back to plain encoding if the dictionary gets too big
 
 
 
 
 
 
 
 
Dummy implementation for perf tests
Edge interpolation algorithm for Geography logical type
encoding of the data
EncodingStats track dictionary and data page encodings for a single column within a row group.
Used to build EncodingStats from metadata or to accumulate stats as pages are written.
 
The Filter API is expressed through these static methods.
Parquet currently has two ways to specify a filter for dropping records at read time.
 
 
 
 
Anyone wanting to use a FilterCompat.Filter need only implement this interface, per the visitor pattern.
A pass-through proxy for a RecordMaterializer that updates a IncrementallyUpdatedFilterPredicate as it receives concrete values for the current record.
A FilterPredicate is an expression tree describing the criteria for which records to keep when loading data from a parquet file.
A FilterPredicate Visitor must visit all the operators in a FilterPredicate expression tree, and must handle recursion itself, per the visitor pattern.
ValuesReader for FIXED_LEN_BYTE_ARRAY.
ValuesWriter for FIXED_LEN_BYTE_ARRAY.
The class is a utility class to manipulate half-precision 16-bit IEEE 754 floating point data types (also called fp16 or binary16).
 
 
 
A structure for capturing metadata for estimating the unencoded, uncompressed size of geospatial data written.
Builder to create a GeospatialStatistics.
 
 
Group level of the IO structure
converter for group nodes
 
 
Represents a group type: a list of fields
 
 
A interface contains a set of hash functions used by Bloom filter.
 
 
 
thrown when we are trying to read together files with incompatible schemas.
A rewritten version of a FilterPredicate which receives the values for a record's columns one by one and internally tracks whether the predicate is satisfied, unsatisfied, or unknown.
 
 
A ValueInspector implementation that keeps state for one or more delegate inspectors.
 
This is the leaf node of a filter predicate.
A Visitor for an IncrementallyUpdatedFilterPredicate, per the visitor pattern.
This class is auto-generated by org.apache.parquet.filter2.IncrementallyUpdatedFilterPredicateGenerator Do not manually edit!
The implementation of this abstract class is auto-generated by org.apache.parquet.filter2.IncrementallyUpdatedFilterPredicateGenerator
Determines whether an IncrementallyUpdatedFilterPredicate is satisfied or not.
Iterator implementation for page indexes.
 
 
An append-only integer list avoids autoboxing and buffer resizing
to iterate on the content of the list not an actual iterator to avoid autoboxing
 
 
thrown when an invalid record is encountered
 
Recursively removes all use of the not() operator in a predicate by replacing all instances of not(x) with the inverse(x), eg: not(and(eq(), not(eq(y))) -> or(notEq(), eq(y))
Converts a FilterPredicate to its logical inverse.
 
 
 
 
 
 
 
 
 
 
 
 
Implement this interface to visit a logical type annotation in the schema.
 
 
 
 
 
 
 
 
 
 
 
Message level of the IO structure
The root of a schema
Parses a schema from a textual format similar to that described in the Dremel paper.
This class calculates the max and min values of an iterable collection.
 
Provides ability to negate the result of a filter.
Offset index containing the offset and size of the page and the index of the first row in the page.
Builder implementation to create OffsetIndex objects during writing a parquet file.
These are the operators in a filter predicate expression tree.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Base class for Operators.In and Operators.NotIn.
 
 
 
 
 
Deprecated.
use LogicalTypeAnnotation to represent logical types instead
Provides ability to chain two filters together.
one page in a chunk
Filter which will only materialize a page worth of results.
Reader for a sequence a page from a given column chunk
contains all the readers for all the columns of the corresponding row group
a writer for all the pages of a given column chunk
contains all the writers for the columns in the corresponding row group
Examples from the Dremel Paper
thrown when an encoding problem occured
thrown when a decoding problem occured
This class represents all the configurable Parquet properties.
 
 
a simple implementation of dictionary for plain encoded values
a simple implementation of dictionary for plain encoded binary
a simple implementation of dictionary for plain encoded boolean values
a simple implementation of dictionary for plain encoded double values
a simple implementation of dictionary for plain encoded float values
a simple implementation of dictionary for plain encoded integer values
a simple implementation of dictionary for plain encoded long values
Plain encoding for float, double, int, long.
 
 
 
 
Plain encoding except for booleans
 
Primitive level of the IO structure
Comparator implementation that also supports the comparison of the related primitive type to avoid the performance penalty of boxing/unboxing.
converter for leaves of the schema
Class that provides string representations for the primitive values.
Converts a Class<primitive> to it's corresponding Class<Boxed>, eg Class<int> to Class<Integer>
Representation of a Primitive type
Supported Primitive types
 
Abstraction for writing records It decouples the striping algorithm from the actual record model example:
This class can be used to wrap an actual RecordConsumer and log all calls
Filter to be applied to a record to work out whether to skip it.
Top-level class which should be implemented in order to materialize objects from a stream of Parquet data.
This exception signals that the current record is cannot be converted from parquet columns to a materialized record, but can be skipped if requested.
used to read reassembled records
Used to add extra behavior to a ValuesWriter that requires fallback
 
Class representing row ranges in a row-group.
Deprecated.
moved to RowRanges.
Constructs a RowRanges by appending selected row indices in strictly increasing order.
 
Decodes values written in the grammar described in RunLengthBitPackingHybridEncoder
Encodes values using a combination of run length encoding and bit packing, according to the following grammar:
This ValuesReader does all the reading in RunLengthBitPackingHybridValuesReader.initFromPage(int, org.apache.parquet.bytes.ByteBufferInputStream) and stores the values in an in memory buffer, which is less than ideal.
 
Inspects the column types found in the provided FilterPredicate and compares them to the actual schema found in the parquet file.
 
 
A structure for capturing metadata for estimating the unencoded, uncompressed size of data written.
Builder to create a SizeStatistics.
Statistics class to keep track of statistics in parquet pages and column chunks
Contains statistics about a group of records
Builder class to build Statistics objects.
Thrown if the two Statistics objects have mismatching types
Represents the declared type for a field in a schema.
represents a field ID
Constraint on the repetition of a field
to convert a MessageType tree
This class provides fluent builders that produce Parquet schema Types.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
A base builder for Type objects.
A builder for GroupType objects.
 
 
 
A builder for PrimitiveType objects.
 
Implement this interface to visit a schema
Builder for a record filter.
Thrown if the specified column is unknown in the underlying storage
Thrown if the specified column type is unknown in the underlying storage
A UserDefinedPredicate decides whether a record should be kept or dropped, first by inspecting meta data about a group of records to see if the entire group can be dropped, then by inspecting actual values of a single column.
Wraps a record consumer Validates the record written against the schema and pass down the event to the wrapped consumer
Contains all valid mappings from class -> parquet type (and vice versa) for use in FilterPredicates
Base class to implement an encoding for a given column type.
The different type of values we can store in columns
base class to implement an encoding for a given column
Can be overridden to allow users to manually test different strategies to create ValuesWriters.
The implementation of HashFunction interface.
ColumnReader which does not read any actual data, but rather simply produces an endless stream of constant values.