Class FilenameInputFormat

java.lang.Object
org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
org.archive.hadoop.FilenameInputFormat
All Implemented Interfaces:
org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
public class FilenameInputFormat extends org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
Handy "input format" which maps the input filename into a "record" which just has the filename. This is very useful for map-reduce jobs where you want to pass the filenames into the map() function. Use this as the input format, and the input filenames will be passed to the map(). The full pathname is given as both the key and the value to the map().

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileInputFormat

    org.apache.hadoop.mapred.FileInputFormat.Counter

  • Field Summary

    Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

    INPUT_DIR_NONRECURSIVE_IGNORE_SUBDIRS, INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES

  • Constructor Summary

    Constructors
    Constructor
    Description
    FilenameInputFormat()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    configure(org.apache.hadoop.mapred.JobConf conf)
    Configure per Hadoop properties
    org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
    getRecordReader(org.apache.hadoop.mapred.InputSplit genericSplit, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)
    Return a RecordReader which returns 1 record: the file path from the InputSplit.
    protected boolean
    isSplitable(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file)
    By definition, not splitable.

    Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

    addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, getSplits, listStatus, makeSplit, makeSplit, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • FilenameInputFormat

      public FilenameInputFormat()

  • Method Details

    • configure

      public void configure(org.apache.hadoop.mapred.JobConf conf)
      Configure per Hadoop properties

    • isSplitable

      protected boolean isSplitable(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path file)
      By definition, not splitable.
      Overrides:
      isSplitable in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>

    • getRecordReader

      public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text> getRecordReader(org.apache.hadoop.mapred.InputSplit genericSplit, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter) throws IOException
      Return a RecordReader which returns 1 record: the file path from the InputSplit.
      Specified by:
      getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
      Specified by:
      getRecordReader in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.Text, org.apache.hadoop.io.Text>
      Throws:
      IOException