All Classes and Interfaces
Class
Description
Class which implements the bulk of the functionality needed for Stream
behavior, leaving concrete subclasses to implement only 3 simple methods
beyond their constructors:
doSeek()
doRead()
doClose()
This class maintains a buffer of bytes read, and attempts to be efficient
about seeks within that buffer.
Create a 'Wide' CDX from an ARC.
Constants used by ARC files and in ARC file processing.
set of known errors encountered reading ARCs
Constants used by Archive files and in Archive file processing.
Pig Storage Loader for Archive meta data.
Reader for an Archive file of Archive
ArchiveRecords.Factory that returns an Archive file Reader.
Archive file Record.
Archive Record Header.
Miscellaneous useful methods.
Datastructure to hold ARC record location.
Get an iterator on an ARC file or get a record by absolute position.
Factory that returns an ARCReader.
An ARC file record.
An immutable class to hold an ARC record meta data.
Write ARC files.
A pool of ARCWriters.
A repositionable stream backed by an array.
Base32 - encodes and decodes RFC3548 Base32
(see http://www.faqs.org/rfcs/rfc3548.html )
Imported public-domain code of Bitzi.
Canonicalizer that does more or less basic fixup.
Buffers data from some other SeekInputStream.
A bridge between byte buffers and input streams.
Abstract class containing common methods for determining the character
encoding of a text Resource, most of which should be refactored into a
Util package.
Provides a subsequence view onto a CharSequence.
Iterator with a close method that frees up any resources associated with
the Iterator.
Wrap a regular Iterator<S> to create a CloseableIterator<S> where the close() is a no-op
A configurable trust manager built on X509TrustManager.
Write a message and stack trace to the 'org.archive.util.DevUtils' logger.
An URL Connection that pre-downloads URL reference before passing back a
Stream reference.
ANVL 'data element'.
Reads integers stored in big or little endian streams.
Special SLR wrapper (SeekableLineReader) that extracts a certain field
from the reader and only returns that field
Specify a format for FieldSplitLine with names, and nameToIndex mapping for faster lookup
Base class for text lines that are split by a delimiter Some examples will be
cdx lines, summary lines, etc...
Handy "input format" which maps the input filename into a "record"
which just has the filename.
Utility methods for manipulating files and directories.
Indicates that a FTP operation failed due to a protocol violation.
FileHandler with support for rotating the current file to
an archival name with a specified integer suffix, and
provision of a new replacement FileHandler with the current
filename.
(Replay)CharSequence view on recorded streams.
Read in the GZIP header.
Class which encapsulates all the logic in reading multiple gzip members from
a single stream.
A replacement for GZIPInputStream; offers GZIP decompression, without any
artificial stop after the first member in a concatenated series (in
pre-JDK6u23), and offers direct access to discovered GZIP member
boundaries (in compressed offsets) via the getMemberNumber(),
getCurrentMemberStart(), getCurrentMemberEnd() accessors, both pre- and
post- JDK6u23 (but see below for caveat about getCurrentMemberEnd()).
The first 10-bytes of a GZip Member record.
A protocol handler for an 'md5' URI scheme.
A protocol handler that uses native rsync client to do copy.
An ArchiveRecord whose content has a preamble of RFC822-like headers: e.g.
Miscellaneous useful methods.
InetAddress utility.
CharSequence that noticed thread interrupts -- as might be necessary
to recover from a loose regex on unexpected challenging input.
A LineIterator that also implements Iterable, so that it can be used with
the java enhanced for-each loop syntax.
Class which provides a column-oriented view of a JSON structure.
A Modified version of HttpParser which doesn't throw exceptions on bad header lines
A utility class for parsing http header values according to
RFC-2616 Section 4 and 19.3.
URI subclass which allows partial/inconsistent encoding, matching
the URIs which will be relayed in requests from popular web
browsers (esp.
Utility class providing an Iterator interface over line-oriented
text input, as a thin wrapper over a BufferedReader.
Superclass for Iterators which must probe ahead to know if
a 'next' exists, and thus have a cached next between a call
to hasNext() and next().
ObjectOutputStream that logs class name of each object that is written
to the stream.
Md5 URL connection.
A nested structure of JSONObjects to hold the metadata of
content in nested containers, e.g. a HTML page as payload of a HTTP response
in a WARC record stored as gzip "member".
Class of mimetype utilities.
A filter stream that both counts bytes written, and optionally swallows
flush() requests.
A SeekableLineReaderFactory that opens a file ONCE at construction.
Enhanced ObjectOutputStream with support for restoring
files that had been saved, in parallel with object
serialization.
Enhanced ObjectOutputStream which maintains (a stack of) auxiliary
directories and offers convenience methods for serialized objects
to save their related disk files alongside their serialized version.
This class implements a stream filter for reading compressed data in
the GZIP file format.
This class implements a stream filter for uncompressing data in the
"deflate" compression format.
Idea of these rules is to accomplish roughly the equivalent of
UsableURIFactory fixup plus BasicURLCanonicalizer fixup.Idea of this canonicalizer is to accomplish roughly the equivalent of
UsableURIFactory fixup plus BasicURLCanonicalizer fixup.Alters the origin of some other SeekInputStream.
OutputFormat that directs the output to a file named according to
the input file.
Utility class for maintaining sorted set of string prefixes.
Interface indicating a logging Formatter can preformat a record (outside
the standard-implementation synchronized block) and cache it, returning it
for the next request for formatting from the same thread.
Class to run an external process.
Utilities for dealing with Java Properties (incl.
Utility class for making use of the information about 'public suffixes' at
http://publicsuffix.org.
prefix tree node. each Node represents sequence of letters (prefix)
and alternative sequences following it (list of Node's).
Wraps a RandomAccessFile with an InputStream interface.
Wraps a RandomAccessFile with OutputStream interface.
Interface for objects that can provide a Reader view of their
contents.
Pairs together a RecordingInputStream and RecordingOutputStream
to capture exactly a single HTTP transaction.
Indicates a length exception thrown by the Recorder.
Indicates a timeout thrown by the RecordingInputStream.
Indicates a too much header material exception thrown by the Recorder
(specificially the RecordingOutputStream)
A
record-id generator.Stream which records all data read from it, which it acquires from a wrapped
input stream.
An output stream that records all writes to wrapped output
stream.
A decorator on IOException for IOEs that are likely not fatal or at least
merit retry.
Utility class providing an Iterator interface over line-oriented
text input.
CharSequence interface with addition of a
ReplayCharSequence.close() method.Replays the bytes recorded from a RecordingInputStream or
RecordingOutputStream.
Wrapper around an
InputStream to make a primitive Repositionable
stream.A basic interface specifying positioning methods for a byte stream.
A container for one or more Resource objects.
Rsync URL connection.
Enables multiple concurrent streams based on the same underlying stream.
Base class for repositionable input streams.
Base class for repositionable readers.
Implemented by threads that provide extra information.
Supports creating CDXLine for certain defined standard CDX formats
Current options: a 9-field cdx format (default) and 11 field cdx format
Alternate simplified interface for accessing data from an underlying source
of bytes.
Sort-friendly URI Reordering Transform.
Specialized TreeSet for keeping a set of String prefixes.
provides iterative Url reduction for prefix matching to find ever coarser
grained URL-specific configuration.
Superclass for Iterators which transform and/or filter results
from a wrapped Iterator.
The URI parsing and escape encoding exception.
Usable URI.
Factory that returns UsableURIs.
Marker Interface for instances that can be serialized as UTF8 bytes.
WARC Constants used by WARC readers and writers.
WARC Record Types.
WARCReader.
Factory for WARC Readers.
A WARC file Record.
WARC implementation.
A pool of WARCWriters.
Settings object for a
WARCWriterPool.WrappedSeekableLineReader that wraps an existing SeekableLineReader for custom extension
Pool of Writers.
Member of
WriterPool.Settings object for a
WriterPool.