org.apache.flink.connector.file.table.utils

Class CompactFileUtils

java.lang.Object

org.apache.flink.connector.file.table.utils.CompactFileUtils

public class CompactFileUtils
extends Object

Utils for compacting files.

Constructor Summary

Constructors

Constructor and Description
CompactFileUtils()

Method Summary

Modifier and Type

Method and Description

static <T> org.apache.flink.core.fs.Path

doCompact(org.apache.flink.core.fs.FileSystem fileSystem, String partition, List<org.apache.flink.core.fs.Path> paths, org.apache.flink.core.fs.Path target, org.apache.flink.configuration.Configuration config, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompactFileUtils

public CompactFileUtils()

Method Detail

doCompact

@Nullable
public static <T> org.apache.flink.core.fs.Path doCompact(org.apache.flink.core.fs.FileSystem fileSystem,
                                                                    String partition,
                                                                    List<org.apache.flink.core.fs.Path> paths,
                                                                    org.apache.flink.core.fs.Path target,
                                                                    org.apache.flink.configuration.Configuration config,
                                                                    CompactReader.Factory<T> readerFactory,
                                                                    CompactWriter.Factory<T> writerFactory)
                                                             throws IOException

Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.

Throws:

IOException