edu.umd.cloud9.webgraph.driver
Class TrecDriver

java.lang.Object
  org.apache.hadoop.conf.Configured
      edu.umd.cloud9.webgraph.driver.TrecDriver

All Implemented Interfaces:

Configurable, Tool

public class TrecDriver
extends Configured
implements Tool

Main driver program for extracting the web graph, reverse web graph, and lines of anchor text. Command-line arguments are as follows:

• [-input collection_base_path] the base path to the collection
• [-output output-base-path]: the base path under which the output would be stored
• [-collection {trecweb|trec|wt10g|gov2}]: the collection used
• [-inputFormat inputFormatClass]: use user specified input format class
• [-docnoClass userSpecifiedDocnoMappingClass]: use user specified docno mapping class
• [-il]: use this for including the internal links (i.e., links within a domain) remove for not
• [-caw]: use this to compute the default weights for lines of external anchor text, remove for not
• [-normalizer normalizer] A normalizer class used to normalize the lines of anchor text, must extend *.anchor.normalize.AnchorTextNormalizer.
• [ ..]: key-value pairs to put in configuration files. It shall also be used as input method for user specified classes

The default weight used in this program was originally proposed by Metzler et. al in the following paper:
D. Metzler, J. Novak, H. Cui, and S. Reddy. Building enriched document representations using aggregated anchor text. In Proc. 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 219{226, New York, NY, USA, 2009. ACM.

Author:

Nima Asadi , Modified by Fangyue Wang

Constructor Summary

TrecDriver()

Method Summary

static void	main(String[] args)
int	run(String[] args)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

TrecDriver

public TrecDriver()

Method Detail

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception