Package io.delta.kernel.defaults.internal.parquet

Class ParquetFilterUtils

Object
io.delta.kernel.defaults.internal.parquet.ParquetFilterUtils
public class ParquetFilterUtils extends Object
Utilities to convert the Kernel Predicate into `parquet-mr` FilterPredicate.

  • Method Summary

    Modifier and Type
    Method
    Description
    static Optional<org.apache.parquet.filter2.predicate.FilterPredicate>
    toParquetFilter(org.apache.parquet.schema.MessageType parquetFileSchema, io.delta.kernel.expressions.Predicate kernelPredicate)
    Convert the given Kernel predicate kernelPredicate into `parquet-mr` predicate.

    Methods inherited from class java.lang.Object

    equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • toParquetFilter

      public static Optional<org.apache.parquet.filter2.predicate.FilterPredicate> toParquetFilter(org.apache.parquet.schema.MessageType parquetFileSchema, io.delta.kernel.expressions.Predicate kernelPredicate)
      Convert the given Kernel predicate kernelPredicate into `parquet-mr` predicate.
      Parameters:
      parquetFileSchema - Schema of the Parquet file. We need it to find what columns exists in the Parquet file in order to remove predicates on columns that do not exist in the file. There is no clear way to handle the predicate on columns that don't exist in the Parquet file.
      kernelPredicate - Kernel predicate to convert.
      Returns:
      instance of FilterCompat.Filter (`parquet-mr` filter)