edu.umd.cloud9.collection.wikipedia
Class WikipediaPageInputFormat

java.lang.Object
  org.apache.hadoop.mapreduce.InputFormat<K,V>
      org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
          edu.umd.cloud9.collection.IndexableFileInputFormat<LongWritable,WikipediaPage>
              edu.umd.cloud9.collection.wikipedia.WikipediaPageInputFormat

public class WikipediaPageInputFormat
extends IndexableFileInputFormat<LongWritable,WikipediaPage>

Hadoop InputFormat for processing Wikipedia pages from the XML dumps.

Author:

Jimmy Lin, Peter Exner

Nested Class Summary

static class

WikipediaPageInputFormat.WikipediaPageRecordReader

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

INPUT_DIR, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

WikipediaPageInputFormat()

Method Summary

RecordReader<LongWritable,WikipediaPage>

createRecordReader(InputSplit split, TaskAttemptContext context)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPaths, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WikipediaPageInputFormat

public WikipediaPageInputFormat()

Method Detail

createRecordReader

public RecordReader<LongWritable,WikipediaPage> createRecordReader(InputSplit split,
                                                                   TaskAttemptContext context)
                                                            throws IOException,
                                                                   InterruptedException

Specified by:

createRecordReader in class InputFormat<LongWritable,WikipediaPage>

Throws:

IOException

InterruptedException