edu.umd.cloud9.collection
Class ExtractHTMLFieldCollection

java.lang.Object
  org.apache.hadoop.conf.Configured
      edu.umd.cloud9.util.PowerTool
          edu.umd.cloud9.collection.ExtractHTMLFieldCollection

All Implemented Interfaces:

Configurable, Tool

public class ExtractHTMLFieldCollection
extends PowerTool

Tool for generating 'per-field' collections from HTML documents. The output of this tool is a new collection, in TREC format (in the form of a SequenceFile) that only consists of the text contained within the target tag. This is useful for various document structure and/or field-based retrieval tasks.

Author:

fangyue, metzler

Nested Class Summary

static class

ExtractHTMLFieldCollection.MyMapper

Field Summary

static String[]

RequiredParameters

Constructor Summary

ExtractHTMLFieldCollection(Configuration conf)

Method Summary

String[]	getRequiredParameters()
static void	main(String[] args)
static void	recursivelyAddInputPaths(Job job, String path)
int	runTool()

Methods inherited from class edu.umd.cloud9.util.PowerTool

run, run

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

RequiredParameters

public static final String[] RequiredParameters

Constructor Detail

ExtractHTMLFieldCollection

public ExtractHTMLFieldCollection(Configuration conf)

Method Detail

getRequiredParameters

public String[] getRequiredParameters()

Specified by:

getRequiredParameters in class PowerTool

runTool

public int runTool()
            throws Exception

Specified by:

runTool in class PowerTool

Throws:

Exception

recursivelyAddInputPaths

public static void recursivelyAddInputPaths(Job job,
                                            String path)
                                     throws IOException

Throws:

IOException

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception