public class FileListUtils extends Object
FileSystem.FileSystem| Modifier and Type | Field and Description |
|---|---|
static Comparator<org.apache.hadoop.fs.FileStatus> |
LATEST_MOD_TIME_ORDER |
static org.apache.hadoop.fs.PathFilter |
NO_OP_PATH_FILTER |
| Constructor and Description |
|---|
FileListUtils() |
| Modifier and Type | Method and Description |
|---|---|
static org.apache.hadoop.fs.FileStatus |
getAnyNonHiddenFile(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Get any data file, which is not hidden or a directory, from the given path
|
static List<org.apache.hadoop.fs.FileStatus> |
listFilesRecursively(org.apache.hadoop.fs.FileSystem fs,
Iterable<org.apache.hadoop.fs.Path> paths) |
static List<org.apache.hadoop.fs.FileStatus> |
listFilesRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path) |
static List<org.apache.hadoop.fs.FileStatus> |
listFilesRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter)
Helper method to list out all files under a specified path.
|
static List<org.apache.hadoop.fs.FileStatus> |
listFilesRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter,
boolean applyFilterToDirectories)
Helper method to list out all files under a specified path.
|
static List<org.apache.hadoop.fs.FileStatus> |
listFilesToCopyAtPath(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter,
boolean includeEmptyDirectories)
Given a path to copy, list all files rooted at the given path to copy
|
static List<org.apache.hadoop.fs.FileStatus> |
listFilesToCopyAtPath(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter,
boolean applyFilterToDirectories,
boolean includeEmptyDirectories)
Given a path to copy, list all files rooted at the given path to copy
|
static List<org.apache.hadoop.fs.FileStatus> |
listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs,
Iterable<org.apache.hadoop.fs.Path> paths) |
static List<org.apache.hadoop.fs.FileStatus> |
listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Method to list out all files, or directory if no file exists, under a specified path.
|
static List<org.apache.hadoop.fs.FileStatus> |
listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter)
Method to list out all files, or directory if no file exists, under a specified path.
|
static List<org.apache.hadoop.fs.FileStatus> |
listPathsRecursively(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.fs.PathFilter fileFilter)
Helper method to list out all paths under a specified path.
|
public static final Comparator<org.apache.hadoop.fs.FileStatus> LATEST_MOD_TIME_ORDER
public static final org.apache.hadoop.fs.PathFilter NO_OP_PATH_FILTER
public static List<org.apache.hadoop.fs.FileStatus> listFilesRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) throws IOException
IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listFilesRecursively(org.apache.hadoop.fs.FileSystem fs, Iterable<org.apache.hadoop.fs.Path> paths) throws IOException
IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listFilesToCopyAtPath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter, boolean includeEmptyDirectories) throws IOException
fs - the file system of the pathpath - root path to copyfileFilter - a filter only applied to rootincludeEmptyDirectories - a control to include empty directories for copyIOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listFilesToCopyAtPath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter, boolean applyFilterToDirectories, boolean includeEmptyDirectories) throws IOException
fs - the file system of the pathpath - root path to copyfileFilter - a filter only applied to rootapplyFilterToDirectories - a control to decide whether to apply filter to directoriesincludeEmptyDirectories - a control to include empty directories for copyIOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listFilesRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter) throws IOException
PathFilter is treated as a file
filter, that is it is only applied to file Paths.IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listFilesRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter, boolean applyFilterToDirectories) throws IOException
PathFilter will only be applied to files.IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) throws IOException
IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs, Iterable<org.apache.hadoop.fs.Path> paths) throws IOException
IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listMostNestedPathRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter) throws IOException
PathFilter is treated as a file filter, that is it is only applied to file Paths.IOExceptionpublic static List<org.apache.hadoop.fs.FileStatus> listPathsRecursively(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.PathFilter fileFilter) throws IOException
FileSystem is
unable to list the contents of a relevant directory, will log an error and skip.IOExceptionpublic static org.apache.hadoop.fs.FileStatus getAnyNonHiddenFile(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
throws IOException
IOException