| Constructor and Description |
|---|
BatchCompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem,IOException> fsFactory,
CompactReader.Factory<T> readerFactory,
CompactWriter.Factory<T> writerFactory) |
| Modifier and Type | Method and Description |
|---|---|
static <T> CompactWriter.Factory<T> |
CompactBucketWriter.factory(org.apache.flink.util.function.SupplierWithException<org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter<T,String>,IOException> factory) |
| Constructor and Description |
|---|
CompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem,IOException> fsFactory,
CompactReader.Factory<T> readerFactory,
CompactWriter.Factory<T> writerFactory) |
| Modifier and Type | Method and Description |
|---|---|
static <T> org.apache.flink.core.fs.Path |
CompactFileUtils.doCompact(org.apache.flink.core.fs.FileSystem fileSystem,
String partition,
List<org.apache.flink.core.fs.Path> paths,
org.apache.flink.core.fs.Path target,
org.apache.flink.configuration.Configuration config,
CompactReader.Factory<T> readerFactory,
CompactWriter.Factory<T> writerFactory)
Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do
atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and
writing.
|
Copyright © 2014–2025 The Apache Software Foundation. All rights reserved.