org.archive.io.arc

Class ARCReaderFactory.CompressedARCReader

java.lang.Object

org.archive.io.ArchiveReader

org.archive.io.arc.ARCReader

org.archive.io.arc.ARCReaderFactory.CompressedARCReader

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<ArchiveRecord>, ARCConstants, ArchiveFileConstants

Enclosing class:

ARCReaderFactory

public class ARCReaderFactory.CompressedARCReader
extends ARCReader

Compressed arc file reader.

Author:

stack

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.io.ArchiveReader

ArchiveReader.ArchiveRecordIterator

Nested classes/interfaces inherited from interface org.archive.format.arc.ARCConstants

ARCConstants.ArcRecordErrors

Field Summary

Fields inherited from class org.archive.io.ArchiveReader

in, MAX_ALLOWED_RECOVERABLES

Fields inherited from interface org.archive.format.arc.ARCConstants

ACTUAL_LENGTH_KEY, ALEXA_DAT_MIME, ARC_FILE_EXTENSION, ARC_GZIP_EXTRA_FIELD, ARC_MAGIC_NUMBER, ARC_META_CHARSET, CARRIAGE_RETURN_ORD, CHECKSUM_FIELD_KEY, CHECKSUM_HEADER_FIELD_KEY, CODE_HEADER_FIELD_KEY, COMPRESSED_ARC_FILE_EXTENSION, DATE_STRING_KEY, DECLARED_LENGTH_KEY, DEFAULT_ENCODING, DEFAULT_GZIP_HEADER_LENGTH, DEFAULT_MAX_ARC_FILE_SIZE, DEFAULT_MIME, DELIMITER, DNS_MIME, DOT_ARC_FILE_EXTENSION, DOT_COMPRESSED_ARC_FILE_EXTENSION, DOT_COMPRESSED_FILE_EXTENSION, FIELD_COUNT, FILEDESC_SCHEME, FILENAME_FIELD_KEY, FILENAME_HEADER_FIELD_KEY, GZIP_HEADER_BEGIN, HEADER_FIELD_SEPARATOR, HEADER_LENGTH, IP_HEADER_FIELD_KEY, IP_KEY, LEADING_NEWLINES_KEY, LINE_SEPARATOR, LOCATION_HEADER_FIELD_KEY, MAX_HEADER_LINE_LENGTH, MAX_META_LENGTH, MAX_METADATA_LINE_LENGTH, MIME_KEY, MINIMUM_RECORD_LENGTH, NEW_LINE_ORD, OFFSET_FIELD_KEY, OFFSET_HEADER_FIELD_KEY, REQUIRED_VERSION_1_HEADER_FIELDS, STATUSCODE_FIELD_KEY, TOKENIZED_PREFIX, TRAILING_CRUFT_KEY, TRAILING_NEWLINES_KEY, URL_KEY

Fields inherited from interface org.archive.format.ArchiveFileConstants

ABSOLUTE_OFFSET_KEY, CDX, CDX_FILE, CDX_LINE_BUFFER_SIZE, CRLF, DATE_FIELD_KEY, DEFAULT_DIGEST_METHOD, DUMP, GZIP_DUMP, HEADER, INVALID_SUFFIX, LENGTH_FIELD_KEY, MIMETYPE_FIELD_KEY, NOHEAD, OCCUPIED_SUFFIX, READER_IDENTIFIER_FIELD_KEY, RECORD_IDENTIFIER_FIELD_KEY, SINGLE_SPACE, TYPE_FIELD_KEY, URL_FIELD_KEY, VERSION_FIELD_KEY

Constructor Summary

Constructors

Constructor and Description
ARCReaderFactory.CompressedARCReader(File f) Constructor.
ARCReaderFactory.CompressedARCReader(File f, long offset) Constructor.
ARCReaderFactory.CompressedARCReader(String f, InputStream is, boolean atFirstRecord) Constructor.

Method Summary

Methods

Modifier and Type	Method and Description
ARCRecord	get(long offset) Get record at passed offset.
protected void	gotoEOR(ArchiveRecord rec) Skip over any trailing new lines at end of the record so we're lined up ready to read the next.
Iterator<ArchiveRecord>	iterator() Returns an ArchiveRecord iterator.

Methods inherited from class org.archive.io.arc.ARCReader

createArchiveRecord, createCDXIndexFile, dump, getDeleteFileOnCloseReader, getDotFileExtension, getFileExtension, getVersion, isAlignedOnFirstRecord, isParseHttpHeaders, main, output, output, outputRecord, setAlignedOnFirstRecord, setParseHttpHeaders

Methods inherited from class org.archive.io.ArchiveReader

cdxOutput, cleanupCurrentRecord, close, currentRecord, get, getCurrentRecord, getFileName, getIn, getInputStream, getLogger, getOptions, getReaderIdentifier, getStrippedFileName, getStrippedFileName, getTrueOrFalse, initialize, isCompressed, isDigest, isStrict, isValid, logStdErr, outputRecord, positionForRecord, setCompressed, setDigest, setIn, setReaderIdentifier, setStrict, setVersion, stripExtension, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.io.Closeable

close

Constructor Detail

ARCReaderFactory.CompressedARCReader

public ARCReaderFactory.CompressedARCReader(File f)
                                     throws IOException

Constructor.

Parameters:

f - Compressed arcfile to read.

Throws:

IOException

ARCReaderFactory.CompressedARCReader

public ARCReaderFactory.CompressedARCReader(File f,
                                    long offset)
                                     throws IOException

Constructor.

Parameters:

f - Compressed arcfile to read.

offset - Position at where to start reading file.

Throws:

IOException

ARCReaderFactory.CompressedARCReader

public ARCReaderFactory.CompressedARCReader(String f,
                                    InputStream is,
                                    boolean atFirstRecord)
                                     throws IOException

Constructor.

Parameters:

f - Compressed arcfile.

is - InputStream to use.

Throws:

IOException

Method Detail

get

public ARCRecord get(long offset)
              throws IOException

Get record at passed offset.

Overrides:

get in class ArchiveReader

Parameters:

offset - Byte index into arcfile at which a record starts.

Returns:

An ARCRecord reference.

Throws:

IOException

iterator

public Iterator<ArchiveRecord> iterator()

Description copied from class: ArchiveReader

Returns an ArchiveRecord iterator. Of note, on IOException, especially if ZipException reading compressed ARCs, rather than fail the iteration, try moving to the next record. If ArchiveReader.strict is not set, this will usually succeed.

Specified by:

iterator in interface Iterable<ArchiveRecord>

Overrides:

iterator in class ArchiveReader

Returns:

An iterator over ARC records.

gotoEOR

protected void gotoEOR(ArchiveRecord rec)
                throws IOException

Description copied from class: ARCReader

Skip over any trailing new lines at end of the record so we're lined up ready to read the next.

Overrides:

gotoEOR in class ARCReader

Throws:

IOException