Class ParquetProperties.Builder

java.lang.Object
org.apache.parquet.column.ParquetProperties.Builder
Enclosing class:
ParquetProperties

public static class ParquetProperties.Builder extends Object
  • Method Details

    • withPageSize

      public ParquetProperties.Builder withPageSize(int pageSize)
      Set the Parquet format page size.
      Parameters:
      pageSize - an integer size in bytes
      Returns:
      this builder for method chaining.
    • withDictionaryEncoding

      public ParquetProperties.Builder withDictionaryEncoding(boolean enableDictionary)
      Enable or disable dictionary encoding.
      Parameters:
      enableDictionary - whether dictionary encoding should be enabled
      Returns:
      this builder for method chaining.
    • withDictionaryEncoding

      public ParquetProperties.Builder withDictionaryEncoding(String columnPath, boolean enableDictionary)
      Enable or disable dictionary encoding for the specified column.
      Parameters:
      columnPath - the path of the column (dot-string)
      enableDictionary - whether dictionary encoding should be enabled
      Returns:
      this builder for method chaining.
    • withByteStreamSplitEncoding

      public ParquetProperties.Builder withByteStreamSplitEncoding(boolean enable)
      Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT and DOUBLE columns.
      Parameters:
      enable - whether BYTE_STREAM_SPLIT encoding should be enabled
      Returns:
      this builder for method chaining.
    • withByteStreamSplitEncoding

      public ParquetProperties.Builder withByteStreamSplitEncoding(String columnPath, boolean enable)
      Enable or disable BYTE_STREAM_SPLIT encoding for specified columns.
      Parameters:
      columnPath - the path of the column (dot-string)
      enable - whether BYTE_STREAM_SPLIT encoding should be enabled
      Returns:
      this builder for method chaining.
    • withExtendedByteStreamSplitEncoding

      public ParquetProperties.Builder withExtendedByteStreamSplitEncoding(boolean enable)
      Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT, DOUBLE, INT32, INT64 and FIXED_LEN_BYTE_ARRAY columns.
      Parameters:
      enable - whether BYTE_STREAM_SPLIT encoding should be enabled
      Returns:
      this builder for method chaining.
    • withDictionaryPageSize

      public ParquetProperties.Builder withDictionaryPageSize(int dictionaryPageSize)
      Set the Parquet format dictionary page size.
      Parameters:
      dictionaryPageSize - an integer size in bytes
      Returns:
      this builder for method chaining.
    • withWriterVersion

      public ParquetProperties.Builder withWriterVersion(ParquetProperties.WriterVersion version)
      Parameters:
      version - a WriterVersion
      Returns:
      this builder for method chaining.
    • withMinRowCountForPageSizeCheck

      public ParquetProperties.Builder withMinRowCountForPageSizeCheck(int min)
    • withMaxRowCountForPageSizeCheck

      public ParquetProperties.Builder withMaxRowCountForPageSizeCheck(int max)
    • withPageValueCountThreshold

      public ParquetProperties.Builder withPageValueCountThreshold(int value)
    • estimateRowCountForPageSizeCheck

      public ParquetProperties.Builder estimateRowCountForPageSizeCheck(boolean estimateNextSizeCheck)
    • withAllocator

      public ParquetProperties.Builder withAllocator(org.apache.parquet.bytes.ByteBufferAllocator allocator)
    • withValuesWriterFactory

      public ParquetProperties.Builder withValuesWriterFactory(ValuesWriterFactory factory)
    • withColumnIndexTruncateLength

      public ParquetProperties.Builder withColumnIndexTruncateLength(int length)
    • withStatisticsTruncateLength

      public ParquetProperties.Builder withStatisticsTruncateLength(int length)
    • withMaxBloomFilterBytes

      public ParquetProperties.Builder withMaxBloomFilterBytes(int maxBloomFilterBytes)
      Set max Bloom filter bytes for related columns.
      Parameters:
      maxBloomFilterBytes - the max bytes of a Bloom filter bitset for a column.
      Returns:
      this builder for method chaining
    • withBloomFilterNDV

      public ParquetProperties.Builder withBloomFilterNDV(String columnPath, long ndv)
      Set Bloom filter NDV (number of distinct values) for the specified column. If set for a column then the writing of the bloom filter for that column will be automatically enabled (see withBloomFilterEnabled(String, boolean)).
      Parameters:
      columnPath - the path of the column (dot-string)
      ndv - the NDV of the column
      Returns:
      this builder for method chaining
    • withBloomFilterFPP

      public ParquetProperties.Builder withBloomFilterFPP(String columnPath, double fpp)
    • withBloomFilterEnabled

      public ParquetProperties.Builder withBloomFilterEnabled(boolean enabled)
      Enable or disable the bloom filter for the columns not specified by withBloomFilterEnabled(String, boolean).
      Parameters:
      enabled - whether bloom filter shall be enabled for all columns
      Returns:
      this builder for method chaining
    • withAdaptiveBloomFilterEnabled

      public ParquetProperties.Builder withAdaptiveBloomFilterEnabled(boolean enabled)
      Whether to use adaptive bloom filter to automatically adjust the bloom filter size according to `parquet.bloom.filter.max.bytes`. If NDV (number of distinct values) for a specified column is set, it will be ignored
      Parameters:
      enabled - whether to use adaptive bloom filter
    • withBloomFilterCandidatesNumber

      public ParquetProperties.Builder withBloomFilterCandidatesNumber(String columnPath, int number)
      When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.
      Parameters:
      columnPath - the path of the column (dot-string)
      number - the number of candidates
    • withBloomFilterEnabled

      public ParquetProperties.Builder withBloomFilterEnabled(String columnPath, boolean enabled)
      Enable or disable the bloom filter for the specified column. One may either disable bloom filters for all columns by invoking withBloomFilterEnabled(boolean) with a false value and then enable the bloom filters for the required columns one-by-one by invoking this method or vice versa.
      Parameters:
      columnPath - the path of the column (dot-string)
      enabled - whether bloom filter shall be enabled
      Returns:
      this builder for method chaining
    • withRowGroupRowCountLimit

      public ParquetProperties.Builder withRowGroupRowCountLimit(int rowCount)
    • withPageRowCountLimit

      public ParquetProperties.Builder withPageRowCountLimit(int rowCount)
    • withPageWriteChecksumEnabled

      public ParquetProperties.Builder withPageWriteChecksumEnabled(boolean val)
    • withExtraMetaData

      public ParquetProperties.Builder withExtraMetaData(Map<String,String> extraMetaData)
    • withStatisticsEnabled

      public ParquetProperties.Builder withStatisticsEnabled(String columnPath, boolean enabled)
      Enable or disable the statistics for given column. All column statistics are enabled by default.
      Parameters:
      columnPath - the given column
      enabled - enable or disable
      Returns:
      this builder for method chaining
    • withStatisticsEnabled

      public ParquetProperties.Builder withStatisticsEnabled(boolean enabled)
    • withSizeStatisticsEnabled

      public ParquetProperties.Builder withSizeStatisticsEnabled(boolean enabled)
      Sets whether size statistics are enabled globally. When disabled, size statistics will not be collected for any column unless explicitly enabled for specific columns.
      Parameters:
      enabled - whether to collect size statistics globally
      Returns:
      this builder for method chaining
    • withSizeStatisticsEnabled

      public ParquetProperties.Builder withSizeStatisticsEnabled(String columnPath, boolean enabled)
      Sets the size statistics enabled/disabled for the specified column. All column size statistics are enabled by default.
      Parameters:
      columnPath - the path of the column (dot-string)
      enabled - whether to collect size statistics for the column
      Returns:
      this builder for method chaining
    • withCompressionCodec

      public ParquetProperties.Builder withCompressionCodec(String columnPath, org.apache.parquet.hadoop.metadata.CompressionCodecName codec)
      Set the compression codec for the specified column.
      Parameters:
      columnPath - the path of the column (dot-string)
      codec - the compression codec to use for this column
      Returns:
      this builder for method chaining
    • withCompressionLevel

      public ParquetProperties.Builder withCompressionLevel(String columnPath, Integer level)
      Set the compression level for the specified column. The valid range is codec-specific (e.g. ZSTD: negative levels trade ratio for speed, up to 22 for best compression (default 3), GZIP: 0–9 or -1 (default 6), BROTLI: 0–11 (default 1)). Pass null to unset a previously configured level for that column.
      Parameters:
      columnPath - the path of the column (dot-string)
      level - the compression level, or null to unset
      Returns:
      this builder for method chaining
    • build

      public ParquetProperties build()