org.apache.flink.fs.s3presto

Class FlinkS3PrestoFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

org.apache.flink.fs.s3.common.FlinkS3FileSystem

org.apache.flink.fs.s3presto.FlinkS3PrestoFileSystem

All Implemented Interfaces:

org.apache.flink.core.fs.EntropyInjectingFileSystem

public class FlinkS3PrestoFileSystem
extends org.apache.flink.fs.s3.common.FlinkS3FileSystem

FlinkS3PrestoFileSystem provides custom recursive deletion functionality to work around a bug in the internally used Presto file system.

https://github.com/prestodb/presto/issues/17416

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

org.apache.flink.core.fs.FileSystem.WriteMode

Field Summary

Fields inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

S3_MULTIPART_MIN_PART_SIZE

Constructor Summary

Constructors

Constructor and Description
FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem, String localTmpDirectory, String entropyInjectionKey, int entropyLength, org.apache.flink.fs.s3.common.writer.S3AccessHelper s3UploadHelper, long s3uploadPartSize, int maxConcurrentUploadsPerStream)

Method Summary

Modifier and Type	Method and Description
boolean	delete(org.apache.flink.core.fs.Path path, boolean recursive)

Methods inherited from class org.apache.flink.fs.s3.common.FlinkS3FileSystem

createRecoverableWriter, generateEntropy, getEntropyInjectionKey, getKind, getLocalTmpDir

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

create, create, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename, toHadoopPath

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkS3PrestoFileSystem

public FlinkS3PrestoFileSystem(org.apache.hadoop.fs.FileSystem hadoopS3FileSystem,
                               String localTmpDirectory,
                               @Nullable
                               String entropyInjectionKey,
                               int entropyLength,
                               @Nullable
                               org.apache.flink.fs.s3.common.writer.S3AccessHelper s3UploadHelper,
                               long s3uploadPartSize,
                               int maxConcurrentUploadsPerStream)

Method Detail

delete

public boolean delete(org.apache.flink.core.fs.Path path,
                      boolean recursive)
               throws IOException

Overrides:

delete in class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

Throws:

IOException