Package org.apache.flink.connector.file.src.enumerate

Class NonSplittingRecursiveAllDirEnumerator

  • All Implemented Interfaces:
    FileEnumerator
    @Internal
public class NonSplittingRecursiveAllDirEnumerator
extends NonSplittingRecursiveEnumerator
    This FileEnumerator enumerates all files under the given paths recursively except the hidden directories. Each file matched the given regex pattern becomes one split; this enumerator does not split files into smaller "block" units.

    The default instantiation of this enumerator filters files with the common hidden file prefixes '.' and '_'. A custom file filter can be specified.

    Compared to NonSplittingRecursiveEnumerator, this enumerator will enumerate all files even through its parent directory is filtered out by the file filter.

    • Constructor Detail

      • NonSplittingRecursiveAllDirEnumerator

        public NonSplittingRecursiveAllDirEnumerator​(String pathRegexPattern)
        Creates a NonSplittingRegexEnumerator that enumerates all files whose file path matches the regex except hidden files. Hidden files are considered files where the filename starts with '.' or with '_'.

      • NonSplittingRecursiveAllDirEnumerator

        public NonSplittingRecursiveAllDirEnumerator​(Predicate<org.apache.flink.core.fs.Path> fileFilter)
        Creates a NonSplittingRegexEnumerator that enumerates all files whose file path matches the regex. Support to use given custom predicate as a filter for file paths.