org.apache.paimon.fs.hadoop

类 HadoopFileIO

java.lang.Object

org.apache.paimon.fs.hadoop.HadoopFileIO

所有已实现的接口:

Serializable, FileIO

public class HadoopFileIO
extends Object
implements FileIO

Hadoop FileIO.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected Map<Pair<String,String>,org.apache.hadoop.fs.FileSystem>	fsMap
protected SerializableConfiguration	hadoopConf

从接口继承的字段 org.apache.paimon.fs.FileIO

LOG

构造器概要

构造器

构造器和说明
HadoopFileIO()

方法概要

限定符和类型	方法和说明
void	configure(CatalogContext context) Configure by CatalogContext.
protected org.apache.hadoop.fs.FileSystem	createFileSystem(org.apache.hadoop.fs.Path path)
boolean	delete(Path path, boolean recursive) Delete a file.
boolean	exists(Path path) Check if exists.
FileStatus	getFileStatus(Path path) Return a file status object that represents the path.
boolean	isObjectStore()
FileStatus[]	listStatus(Path path) List the statuses of the files/directories in the given path if the path is a directory.
boolean	mkdirs(Path path) Make the given file and all non-existent parents into directories.
SeekableInputStream	newInputStream(Path path) Opens an SeekableInputStream at the indicated Path.
PositionOutputStream	newOutputStream(Path path, boolean overwrite) Opens an PositionOutputStream at the indicated Path.
void	overwriteFileUtf8(Path path, String content) Overwrite file by content atomically, different FileIOs have different atomic implementations.
boolean	rename(Path src, Path dst) Renames the file/directory src to dst.
void	setFileSystem(Path path, org.apache.hadoop.fs.FileSystem fs)
boolean	tryAtomicOverwriteViaRename(Path dst, String content)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.paimon.fs.FileIO

checkOrMkdirs, copyFileUtf8, deleteDirectoryQuietly, deleteQuietly, discoverLoaders, get, getFileSize, isDir, readFileUtf8, readOverwrittenFileUtf8, writeFileUtf8

字段详细资料

hadoopConf

protected SerializableConfiguration hadoopConf

fsMap

protected transient volatile Map<Pair<String,String>,org.apache.hadoop.fs.FileSystem> fsMap

构造器详细资料

HadoopFileIO

public HadoopFileIO()

方法详细资料

setFileSystem

@VisibleForTesting
public void setFileSystem(Path path,
                                             org.apache.hadoop.fs.FileSystem fs)
                                      throws IOException

抛出:

IOException

isObjectStore

public boolean isObjectStore()

指定者:

isObjectStore 在接口中 FileIO

configure

public void configure(CatalogContext context)

从接口复制的说明: FileIO

Configure by CatalogContext.

指定者:

configure 在接口中 FileIO

newInputStream

public SeekableInputStream newInputStream(Path path)
                                   throws IOException

从接口复制的说明: FileIO

Opens an SeekableInputStream at the indicated Path.

指定者:

newInputStream 在接口中 FileIO

参数:

path - the file to open

抛出:

IOException

newOutputStream

public PositionOutputStream newOutputStream(Path path,
                                            boolean overwrite)
                                     throws IOException

从接口复制的说明: FileIO

Opens an PositionOutputStream at the indicated Path.

指定者:

newOutputStream 在接口中 FileIO

参数:

path - the file name to open

overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.

抛出:

IOException - Thrown, if the stream could not be opened because of an I/O, or because a file already exists at that path and the write mode indicates to not overwrite the file.

getFileStatus

public FileStatus getFileStatus(Path path)
                         throws IOException

从接口复制的说明: FileIO

Return a file status object that represents the path.

指定者:

getFileStatus 在接口中 FileIO

参数:

path - The path we want information from

返回:

a FileStatus object

抛出:

FileNotFoundException - when the path does not exist; IOException see specific implementation

IOException

listStatus

public FileStatus[] listStatus(Path path)
                        throws IOException

从接口复制的说明: FileIO

List the statuses of the files/directories in the given path if the path is a directory.

指定者:

listStatus 在接口中 FileIO

参数:

path - given path

返回:

the statuses of the files/directories in the given path

抛出:

IOException

exists

public boolean exists(Path path)
               throws IOException

从接口复制的说明: FileIO

Check if exists.

指定者:

exists 在接口中 FileIO

参数:

path - source file

抛出:

IOException

delete

public boolean delete(Path path,
                      boolean recursive)
               throws IOException

从接口复制的说明: FileIO

Delete a file.

指定者:

delete 在接口中 FileIO

参数:

path - the path to delete

recursive - if path is a directory and set to true, the directory is deleted else throws an exception. In case of a file the recursive can be set to either true or false

返回:

true if delete is successful, false otherwise

抛出:

IOException

mkdirs

public boolean mkdirs(Path path)
               throws IOException

从接口复制的说明: FileIO

Make the given file and all non-existent parents into directories. Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error.

指定者:

mkdirs 在接口中 FileIO

参数:

path - the directory/directories to be created

返回:

true if at least one new directory has been created, false otherwise

抛出:

IOException - thrown if an I/O error occurs while creating the directory

rename

public boolean rename(Path src,
                      Path dst)
               throws IOException

从接口复制的说明: FileIO

Renames the file/directory src to dst.

指定者:

rename 在接口中 FileIO

参数:

src - the file/directory to rename

dst - the new name of the file/directory

返回:

true if the renaming was successful, false otherwise

抛出:

IOException

overwriteFileUtf8

public void overwriteFileUtf8(Path path,
                              String content)
                       throws IOException

从接口复制的说明: FileIO

Overwrite file by content atomically, different FileIOs have different atomic implementations.

指定者:

overwriteFileUtf8 在接口中 FileIO

抛出:

IOException

createFileSystem

protected org.apache.hadoop.fs.FileSystem createFileSystem(org.apache.hadoop.fs.Path path)
                                                    throws IOException

抛出:

IOException

tryAtomicOverwriteViaRename

public boolean tryAtomicOverwriteViaRename(Path dst,
                                           String content)
                                    throws IOException

抛出:

IOException