Package org.apache.parquet.column
Class ParquetProperties.Builder
java.lang.Object
org.apache.parquet.column.ParquetProperties.Builder
- Enclosing class:
- ParquetProperties
-
Method Summary
Modifier and TypeMethodDescriptionbuild()estimateRowCountForPageSizeCheck(boolean estimateNextSizeCheck) withAdaptiveBloomFilterEnabled(boolean enabled) Whether to use adaptive bloom filter to automatically adjust the bloom filter size according to `parquet.bloom.filter.max.bytes`.withAllocator(org.apache.parquet.bytes.ByteBufferAllocator allocator) withBloomFilterCandidatesNumber(String columnPath, int number) When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.withBloomFilterEnabled(boolean enabled) Enable or disable the bloom filter for the columns not specified bywithBloomFilterEnabled(String, boolean).withBloomFilterEnabled(String columnPath, boolean enabled) Enable or disable the bloom filter for the specified column.withBloomFilterFPP(String columnPath, double fpp) withBloomFilterNDV(String columnPath, long ndv) Set Bloom filter NDV (number of distinct values) for the specified column.withByteStreamSplitEncoding(boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT and DOUBLE columns.withByteStreamSplitEncoding(String columnPath, boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for specified columns.withColumnIndexTruncateLength(int length) withCompressionCodec(String columnPath, org.apache.parquet.hadoop.metadata.CompressionCodecName codec) Set the compression codec for the specified column.withCompressionLevel(String columnPath, Integer level) Set the compression level for the specified column.withDictionaryEncoding(boolean enableDictionary) Enable or disable dictionary encoding.withDictionaryEncoding(String columnPath, boolean enableDictionary) Enable or disable dictionary encoding for the specified column.withDictionaryPageSize(int dictionaryPageSize) Set the Parquet format dictionary page size.withExtendedByteStreamSplitEncoding(boolean enable) Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT, DOUBLE, INT32, INT64 and FIXED_LEN_BYTE_ARRAY columns.withExtraMetaData(Map<String, String> extraMetaData) withMaxBloomFilterBytes(int maxBloomFilterBytes) Set max Bloom filter bytes for related columns.withMaxRowCountForPageSizeCheck(int max) withMinRowCountForPageSizeCheck(int min) withPageRowCountLimit(int rowCount) withPageSize(int pageSize) Set the Parquet format page size.withPageValueCountThreshold(int value) withPageWriteChecksumEnabled(boolean val) withRowGroupRowCountLimit(int rowCount) withSizeStatisticsEnabled(boolean enabled) Sets whether size statistics are enabled globally.withSizeStatisticsEnabled(String columnPath, boolean enabled) Sets the size statistics enabled/disabled for the specified column.withStatisticsEnabled(boolean enabled) withStatisticsEnabled(String columnPath, boolean enabled) Enable or disable the statistics for given column.withStatisticsTruncateLength(int length) Set theformat version.
-
Method Details
-
withPageSize
Set the Parquet format page size.- Parameters:
pageSize- an integer size in bytes- Returns:
- this builder for method chaining.
-
withDictionaryEncoding
Enable or disable dictionary encoding.- Parameters:
enableDictionary- whether dictionary encoding should be enabled- Returns:
- this builder for method chaining.
-
withDictionaryEncoding
public ParquetProperties.Builder withDictionaryEncoding(String columnPath, boolean enableDictionary) Enable or disable dictionary encoding for the specified column.- Parameters:
columnPath- the path of the column (dot-string)enableDictionary- whether dictionary encoding should be enabled- Returns:
- this builder for method chaining.
-
withByteStreamSplitEncoding
Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT and DOUBLE columns.- Parameters:
enable- whether BYTE_STREAM_SPLIT encoding should be enabled- Returns:
- this builder for method chaining.
-
withByteStreamSplitEncoding
Enable or disable BYTE_STREAM_SPLIT encoding for specified columns.- Parameters:
columnPath- the path of the column (dot-string)enable- whether BYTE_STREAM_SPLIT encoding should be enabled- Returns:
- this builder for method chaining.
-
withExtendedByteStreamSplitEncoding
Enable or disable BYTE_STREAM_SPLIT encoding for FLOAT, DOUBLE, INT32, INT64 and FIXED_LEN_BYTE_ARRAY columns.- Parameters:
enable- whether BYTE_STREAM_SPLIT encoding should be enabled- Returns:
- this builder for method chaining.
-
withDictionaryPageSize
Set the Parquet format dictionary page size.- Parameters:
dictionaryPageSize- an integer size in bytes- Returns:
- this builder for method chaining.
-
withWriterVersion
Set theformat version.- Parameters:
version- aWriterVersion- Returns:
- this builder for method chaining.
-
withMinRowCountForPageSizeCheck
-
withMaxRowCountForPageSizeCheck
-
withPageValueCountThreshold
-
estimateRowCountForPageSizeCheck
-
withAllocator
public ParquetProperties.Builder withAllocator(org.apache.parquet.bytes.ByteBufferAllocator allocator) -
withValuesWriterFactory
-
withColumnIndexTruncateLength
-
withStatisticsTruncateLength
-
withMaxBloomFilterBytes
Set max Bloom filter bytes for related columns.- Parameters:
maxBloomFilterBytes- the max bytes of a Bloom filter bitset for a column.- Returns:
- this builder for method chaining
-
withBloomFilterNDV
Set Bloom filter NDV (number of distinct values) for the specified column. If set for a column then the writing of the bloom filter for that column will be automatically enabled (seewithBloomFilterEnabled(String, boolean)).- Parameters:
columnPath- the path of the column (dot-string)ndv- the NDV of the column- Returns:
- this builder for method chaining
-
withBloomFilterFPP
-
withBloomFilterEnabled
Enable or disable the bloom filter for the columns not specified bywithBloomFilterEnabled(String, boolean).- Parameters:
enabled- whether bloom filter shall be enabled for all columns- Returns:
- this builder for method chaining
-
withAdaptiveBloomFilterEnabled
Whether to use adaptive bloom filter to automatically adjust the bloom filter size according to `parquet.bloom.filter.max.bytes`. If NDV (number of distinct values) for a specified column is set, it will be ignored- Parameters:
enabled- whether to use adaptive bloom filter
-
withBloomFilterCandidatesNumber
When `AdaptiveBloomFilter` is enabled, set how many bloom filter candidates to use.- Parameters:
columnPath- the path of the column (dot-string)number- the number of candidates
-
withBloomFilterEnabled
Enable or disable the bloom filter for the specified column. One may either disable bloom filters for all columns by invokingwithBloomFilterEnabled(boolean)with afalsevalue and then enable the bloom filters for the required columns one-by-one by invoking this method or vice versa.- Parameters:
columnPath- the path of the column (dot-string)enabled- whether bloom filter shall be enabled- Returns:
- this builder for method chaining
-
withRowGroupRowCountLimit
-
withPageRowCountLimit
-
withPageWriteChecksumEnabled
-
withExtraMetaData
-
withStatisticsEnabled
Enable or disable the statistics for given column. All column statistics are enabled by default.- Parameters:
columnPath- the given columnenabled- enable or disable- Returns:
- this builder for method chaining
-
withStatisticsEnabled
-
withSizeStatisticsEnabled
Sets whether size statistics are enabled globally. When disabled, size statistics will not be collected for any column unless explicitly enabled for specific columns.- Parameters:
enabled- whether to collect size statistics globally- Returns:
- this builder for method chaining
-
withSizeStatisticsEnabled
Sets the size statistics enabled/disabled for the specified column. All column size statistics are enabled by default.- Parameters:
columnPath- the path of the column (dot-string)enabled- whether to collect size statistics for the column- Returns:
- this builder for method chaining
-
withCompressionCodec
public ParquetProperties.Builder withCompressionCodec(String columnPath, org.apache.parquet.hadoop.metadata.CompressionCodecName codec) Set the compression codec for the specified column.- Parameters:
columnPath- the path of the column (dot-string)codec- the compression codec to use for this column- Returns:
- this builder for method chaining
-
withCompressionLevel
Set the compression level for the specified column. The valid range is codec-specific (e.g. ZSTD: negative levels trade ratio for speed, up to 22 for best compression (default 3), GZIP: 0–9 or -1 (default 6), BROTLI: 0–11 (default 1)). Passnullto unset a previously configured level for that column.- Parameters:
columnPath- the path of the column (dot-string)level- the compression level, ornullto unset- Returns:
- this builder for method chaining
-
build
-