org.apache.flink.connector.file.src.enumerate

类 NonSplittingRecursiveEnumerator

java.lang.Object

org.apache.flink.connector.file.src.enumerate.NonSplittingRecursiveEnumerator

所有已实现的接口:

FileEnumerator

直接已知子类:

BlockSplittingRecursiveEnumerator

@PublicEvolving
public class NonSplittingRecursiveEnumerator
extends Object
implements FileEnumerator

This FileEnumerator enumerates all files under the given paths recursively. Each file becomes one split; this enumerator does not split files into smaller "block" units.

The default instantiation of this enumerator filters files with the common hidden file prefixes '.' and '_'. A custom file filter can be specified.

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.connector.file.src.enumerate.FileEnumerator

FileEnumerator.Provider

构造器概要

构造器

构造器和说明
NonSplittingRecursiveEnumerator() Creates a NonSplittingRecursiveEnumerator that enumerates all files except hidden files.
NonSplittingRecursiveEnumerator(java.util.function.Predicate<org.apache.flink.core.fs.Path> fileFilter) Creates a NonSplittingRecursiveEnumerator that uses the given predicate as a filter for file paths.

方法概要

限定符和类型	方法和说明
protected void	convertToSourceSplits(org.apache.flink.core.fs.FileStatus file, org.apache.flink.core.fs.FileSystem fs, List<FileSourceSplit> target)
Collection<FileSourceSplit>	enumerateSplits(org.apache.flink.core.fs.Path[] paths, int minDesiredSplits) Generates all file splits for the relevant files under the given paths.
protected String	getNextId()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

NonSplittingRecursiveEnumerator

public NonSplittingRecursiveEnumerator()

Creates a NonSplittingRecursiveEnumerator that enumerates all files except hidden files. Hidden files are considered files where the filename starts with '.' or with '_'.

NonSplittingRecursiveEnumerator

public NonSplittingRecursiveEnumerator(java.util.function.Predicate<org.apache.flink.core.fs.Path> fileFilter)

Creates a NonSplittingRecursiveEnumerator that uses the given predicate as a filter for file paths.

方法详细资料

enumerateSplits

public Collection<FileSourceSplit> enumerateSplits(org.apache.flink.core.fs.Path[] paths,
                                                   int minDesiredSplits)
                                            throws IOException

从接口复制的说明: FileEnumerator

Generates all file splits for the relevant files under the given paths. The minDesiredSplits is an optional hint indicating how many splits would be necessary to exploit parallelism properly.

指定者:

enumerateSplits 在接口中 FileEnumerator

抛出:

IOException

convertToSourceSplits

protected void convertToSourceSplits(org.apache.flink.core.fs.FileStatus file,
                                     org.apache.flink.core.fs.FileSystem fs,
                                     List<FileSourceSplit> target)
                              throws IOException

抛出:

IOException

getNextId

protected final String getNextId()