org.apache.flink.fs.s3presto

Class S3PrestoFileSystem

java.lang.Object

org.apache.flink.core.fs.FileSystem

org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

org.apache.flink.fs.s3presto.S3PrestoFileSystem

All Implemented Interfaces:

org.apache.flink.core.fs.EntropyInjectingFileSystem

public class S3PrestoFileSystem
extends org.apache.flink.runtime.fs.hdfs.HadoopFileSystem
implements org.apache.flink.core.fs.EntropyInjectingFileSystem

Implementation of the Flink FileSystem interface for S3. This class implements the common behavior implemented directly by Flink and delegates common calls to an implementation of Hadoop's filesystem abstraction.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.core.fs.FileSystem

org.apache.flink.core.fs.FileSystem.WriteMode

Constructor Summary

Constructors

Constructor and Description
S3PrestoFileSystem(com.facebook.presto.hive.PrestoS3FileSystem prestoFs) Creates a S3PrestoFileSystem based on the given Presto S3 file system.
S3PrestoFileSystem(com.facebook.presto.hive.PrestoS3FileSystem prestoFs, String entropyInjectionKey, int entropyLength) Creates a FlinkS3FileSystem based on the given Hadoop S3 file system.

Method Summary

Modifier and Type	Method and Description
String	generateEntropy()
String	getEntropyInjectionKey()
org.apache.flink.core.fs.FileSystemKind	getKind()

Methods inherited from class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem

create, create, createRecoverableWriter, delete, exists, getDefaultBlockSize, getFileBlockLocations, getFileStatus, getHadoopFileSystem, getHomeDirectory, getUri, getWorkingDirectory, isDistributedFS, listStatus, mkdirs, open, open, rename

Methods inherited from class org.apache.flink.core.fs.FileSystem

create, get, getDefaultFsUri, getLocalFileSystem, getUnguardedFileSystem, initialize, initOutPathDistFS, initOutPathLocalFS

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3PrestoFileSystem

public S3PrestoFileSystem(com.facebook.presto.hive.PrestoS3FileSystem prestoFs)

Creates a S3PrestoFileSystem based on the given Presto S3 file system. The given Hadoop file system object is expected to be initialized already.

S3PrestoFileSystem

public S3PrestoFileSystem(com.facebook.presto.hive.PrestoS3FileSystem prestoFs,
                          @Nullable
                          String entropyInjectionKey,
                          int entropyLength)

Creates a FlinkS3FileSystem based on the given Hadoop S3 file system. The given Hadoop file system object is expected to be initialized already.

This constructor additionally configures the entropy injection for the file system.

Parameters:

prestoFs - The Presto S3 File System that will be used under the hood.

entropyInjectionKey - The substring that will be replaced by entropy or removed.

entropyLength - The number of random alphanumeric characters to inject as entropy.

Method Detail

getEntropyInjectionKey

@Nullable
public String getEntropyInjectionKey()

Specified by:

getEntropyInjectionKey in interface org.apache.flink.core.fs.EntropyInjectingFileSystem

generateEntropy

public String generateEntropy()

Specified by:

generateEntropy in interface org.apache.flink.core.fs.EntropyInjectingFileSystem

getKind

public org.apache.flink.core.fs.FileSystemKind getKind()

Overrides:

getKind in class org.apache.flink.runtime.fs.hdfs.HadoopFileSystem