Uses of Interface
org.apache.flink.connector.file.table.stream.compact.CompactReader.Factory
-
-
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.batch
Methods in org.apache.flink.connector.file.table.batch with parameters of type CompactReader.Factory Modifier and Type Method Description static <T> org.apache.flink.streaming.api.datastream.DataStreamSink<?>BatchSink. createBatchCompactSink(org.apache.flink.streaming.api.datastream.DataStream<CompactMessages.CoordinatorInput> dataStream, org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T,String,?>> builder, CompactReader.Factory<T> readFactory, FileSystemFactory fsFactory, TableMetaStoreFactory metaStoreFactory, PartitionCommitPolicyFactory partitionCommitPolicyFactory, String[] partitionColumns, LinkedHashMap<String,String> staticPartitionSpec, org.apache.flink.core.fs.Path tmpPath, org.apache.flink.table.catalog.ObjectIdentifier identifier, long compactAverageSize, long compactTargetSize, boolean isToLocal, boolean overwrite, int compactParallelism, boolean compactParallelismConfigured) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.batch.compact
Constructors in org.apache.flink.connector.file.table.batch.compact with parameters of type CompactReader.Factory Constructor Description BatchCompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem,IOException> fsFactory, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.stream
Methods in org.apache.flink.connector.file.table.stream with parameters of type CompactReader.Factory Modifier and Type Method Description static <T> org.apache.flink.streaming.api.datastream.DataStream<PartitionCommitInfo>StreamingSink. compactionWriter(org.apache.flink.table.connector.ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<T> inputStream, long bucketCheckInterval, org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T,String,? extends org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<T,String,?>> bucketsBuilder, FileSystemFactory fsFactory, org.apache.flink.core.fs.Path path, CompactReader.Factory<T> readFactory, long targetFileSize, int parallelism, boolean parallelismConfigured)Create a file writer with compaction operators by input stream. -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.stream.compact
Methods in org.apache.flink.connector.file.table.stream.compact that return CompactReader.Factory Modifier and Type Method Description static <T> CompactReader.Factory<T>CompactBulkReader. factory(BulkFormat<T,FileSourceSplit> format)static <T> CompactReader.Factory<T>FileInputFormatCompactReader. factory(org.apache.flink.api.common.io.FileInputFormat<T> format)Constructors in org.apache.flink.connector.file.table.stream.compact with parameters of type CompactReader.Factory Constructor Description CompactOperator(org.apache.flink.util.function.SupplierWithException<org.apache.flink.core.fs.FileSystem,IOException> fsFactory, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory) -
Uses of CompactReader.Factory in org.apache.flink.connector.file.table.utils
Methods in org.apache.flink.connector.file.table.utils with parameters of type CompactReader.Factory Modifier and Type Method Description static <T> org.apache.flink.core.fs.PathCompactFileUtils. doCompact(org.apache.flink.core.fs.FileSystem fileSystem, String partition, List<org.apache.flink.core.fs.Path> paths, org.apache.flink.core.fs.Path target, org.apache.flink.configuration.Configuration config, CompactReader.Factory<T> readerFactory, CompactWriter.Factory<T> writerFactory)Do Compaction: - Target file exists, do nothing. - Can do compaction: - Single file, do atomic renaming, there are optimizations for FileSystem. - Multiple file, do reading and writing.
-