org.apache.flink.runtime.blob

Class FileSystemBlobStore

java.lang.Object

org.apache.flink.runtime.blob.FileSystemBlobStore

All Implemented Interfaces:

Closeable, AutoCloseable, BlobStore, BlobStoreService, BlobView

public class FileSystemBlobStore
extends Object
implements BlobStoreService

Blob store backed by FileSystem.

This is used in addition to the local blob storage for high availability.

Field Summary

Fields

Modifier and Type	Field and Description
static String	BLOB_PATH_NAME The name of the blob path.

Constructor Summary

Constructors

Constructor and Description
FileSystemBlobStore(org.apache.flink.core.fs.FileSystem fileSystem, String storagePath)

Method Summary

Modifier and Type	Method and Description
void	cleanupAllData() Cleans up the store.
void	close()
boolean	delete(org.apache.flink.api.common.JobID jobId, BlobKey blobKey) Tries to delete a blob from storage.
boolean	deleteAll(org.apache.flink.api.common.JobID jobId) Tries to delete all blobs for the given job from storage.
boolean	get(org.apache.flink.api.common.JobID jobId, BlobKey blobKey, File localFile) Copies a blob to a local file.
boolean	put(File localFile, org.apache.flink.api.common.JobID jobId, BlobKey blobKey) Copies the local file to the blob store.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BLOB_PATH_NAME

public static final String BLOB_PATH_NAME

The name of the blob path.

See Also:

Constant Field Values

Constructor Detail

FileSystemBlobStore

public FileSystemBlobStore(org.apache.flink.core.fs.FileSystem fileSystem,
                           String storagePath)
                    throws IOException

Throws:

IOException

Method Detail

put

public boolean put(File localFile,
                   org.apache.flink.api.common.JobID jobId,
                   BlobKey blobKey)
            throws IOException

Description copied from interface: BlobStore

Copies the local file to the blob store.

Specified by:

put in interface BlobStore

Parameters:

localFile - The file to copy

jobId - ID of the job this blob belongs to (or null if job-unrelated)

blobKey - The ID for the file in the blob store

Returns:

whether the file was copied (true) or not (false)

Throws:

IOException - If the copy fails

get

public boolean get(org.apache.flink.api.common.JobID jobId,
                   BlobKey blobKey,
                   File localFile)
            throws IOException

Description copied from interface: BlobView

Copies a blob to a local file.

Specified by:

get in interface BlobView

Parameters:

jobId - ID of the job this blob belongs to (or null if job-unrelated)

blobKey - The blob ID

localFile - The local file to copy to

Returns:

whether the file was copied (true) or not (false)

Throws:

IOException - If the copy fails

delete

public boolean delete(org.apache.flink.api.common.JobID jobId,
                      BlobKey blobKey)

Description copied from interface: BlobStore

Tries to delete a blob from storage.

NOTE: This also tries to delete any created directories if empty.

Specified by:

delete in interface BlobStore

Parameters:

jobId - ID of the job this blob belongs to (or null if job-unrelated)

blobKey - The blob ID

Returns:

true if the given blob is successfully deleted or non-existing; false otherwise

deleteAll

public boolean deleteAll(org.apache.flink.api.common.JobID jobId)

Description copied from interface: BlobStore

Tries to delete all blobs for the given job from storage.

NOTE: This also tries to delete any created directories if empty.

Specified by:

deleteAll in interface BlobStore

Parameters:

jobId - The JobID part of all blobs to delete

Returns:

true if the job directory is successfully deleted or non-existing; false otherwise

cleanupAllData

public void cleanupAllData()

Description copied from interface: BlobStoreService

Cleans up the store. This entails the deletion of all blobs.

Specified by:

cleanupAllData in interface BlobStoreService

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException