Class ResourceExtractor

java.lang.Object

org.archive.extract.ResourceExtractor

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool, ResourceConstants

public class ResourceExtractor
extends Object
implements ResourceConstants, org.apache.hadoop.util.Tool

Field Summary

Fields

Modifier and Type	Field	Description
static final String	TOOL_DESCRIPTION
static final String	TOOL_NAME

Fields inherited from interface org.archive.resource.ResourceConstants

ARC_HEADER_LENGTH, ARC_HEADER_METADATA, CONTAINER, CONTAINER_COMPRESSED, CONTAINER_FILENAME, CONTAINER_OFFSET, CR_NL_CHARS, DAT_METADATA, DNS_DATE, DNS_ENTRIES, DNS_METADATA, DNS_NAME, DNS_NETCLASS, DNS_TTL, DNS_TYPE, DNS_VALUE, ENVELOPE, ENVELOPE_FORMAT, ENVELOPE_FORMAT_ARC, ENVELOPE_FORMAT_WARC, ENVELOPE_FORMAT_WARC_1_0, FILEDESC_DATA, FILEDESC_FORMAT, FILEDESC_MAJOR, FILEDESC_METADATA, FILEDESC_MINOR, FILEDESC_ORGANIZATION, GZIP, GZIP_COMMENT_LENGTH, GZIP_DEFLATE_LENGTH, GZIP_FEXTRA, GZIP_FEXTRA_NAME, GZIP_FEXTRA_VALUE, GZIP_FILENAME, GZIP_FOOTER_LENGTH, GZIP_HEADER_CRC, GZIP_HEADER_LENGTH, GZIP_INFLATED_CRC, GZIP_INFLATED_LENGTH, HTML_BASE, HTML_HEAD, HTML_LINK_TAGS, HTML_LINKS, HTML_META_TAGS, HTML_METADATA, HTML_SCRIPT_TAGS, HTML_TITLE, HTTP_ENTITY_DIGEST, HTTP_ENTITY_LENGTH, HTTP_ENTITY_TRAILING_SLOP, HTTP_HEADERS_CORRUPT, HTTP_HEADERS_LENGTH, HTTP_HEADERS_LIST, HTTP_MESSAGE_METHOD, HTTP_MESSAGE_PATH, HTTP_MESSAGE_REASON, HTTP_MESSAGE_STATUS, HTTP_MESSAGE_VERSION, HTTP_REQUEST_MESSAGE, HTTP_REQUEST_METADATA, HTTP_RESPONSE_MESSAGE, HTTP_RESPONSE_METADATA, METADATA_KV_NAME, METADATA_KV_VALUE, PAYLOAD_CONTENT_TYPE, PAYLOAD_DIGEST, PAYLOAD_LEADING_SLOP_BYTES, PAYLOAD_LENGTH, PAYLOAD_METADATA, PAYLOAD_SLOP_BYTES, PAYLOAD_TYPE_DAT, PAYLOAD_TYPE_DNS, PAYLOAD_TYPE_FILEDESC, PAYLOAD_TYPE_HTTP_REQUEST, PAYLOAD_TYPE_HTTP_RESPONSE, PAYLOAD_TYPE_WARC_META_FIELDS, PAYLOAD_TYPE_WARCINFO, WARC_HEADER_LENGTH, WARC_HEADER_METADATA, WARC_META_FIELDS_CORRUPT, WARC_META_FIELDS_LIST, WARC_META_FIELDS_METADATA, WARCINFO_METADATA

Constructor Summary

Constructors

Constructor	Description
ResourceExtractor()

Method Summary

Modifier and Type	Method	Description
org.apache.hadoop.conf.Configuration	getConf()
OutputStream	getOut()
static void	main(String[] args)
int	run(String[] args)
void	setConf(org.apache.hadoop.conf.Configuration conf)
void	setOut(OutputStream out)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

TOOL_NAME

public static final String TOOL_NAME

See Also:

• Constant Field Values

TOOL_DESCRIPTION

public static final String TOOL_DESCRIPTION

See Also:

• Constant Field Values

Constructor Details

ResourceExtractor

public ResourceExtractor()

Method Details

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

run

public int run(String[] args)
        throws IndexOutOfBoundsException,
FileNotFoundException,
IOException,
ResourceParseException,
URISyntaxException

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

IndexOutOfBoundsException

FileNotFoundException

IOException

ResourceParseException

URISyntaxException

getOut

public OutputStream getOut()

Returns:

the out

setOut

public void setOut(OutputStream out)

Parameters:

out - the out to set