org.archive.util.zip

Class GZIPMembersInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

org.archive.util.zip.OpenJDK7InflaterInputStream

org.archive.util.zip.OpenJDK7GZIPInputStream

org.archive.util.zip.GZIPMembersInputStream

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

GZIPMembersInputStream

public class GZIPMembersInputStream
extends OpenJDK7GZIPInputStream

A replacement for GZIPInputStream; offers GZIP decompression, without any artificial stop after the first member in a concatenated series (in pre-JDK6u23), and offers direct access to discovered GZIP member boundaries (in compressed offsets) via the getMemberNumber(), getCurrentMemberStart(), getCurrentMemberEnd() accessors, both pre- and post- JDK6u23 (but see below for caveat about getCurrentMemberEnd()). (This replaces our previous workaround, 'GzippedInputStream', for pre-JDK6u23 GZIPInputStream behavior.) By default, will read straight through members, returning all uncompressed data from concatenated compressed members as one stream, per the JDK6u23-and-higher behavior. The data returned from a single read() will not straddle a member boundary, *but* only after reading the first byte of the next member can certainty be offered as to whether the previous member ended. Thus, in this default mode, until the end of all input, the getAtMemberEnd() method will always return false, and getCurrentMemberEnd() will always return -1, because any read that discovered a definitive member-end will have begun the next member. In this mode, member-ends should be deduced by watching the increment of getMemberNumber(), and using the start of the current record as the (exclusive) end-position of the previous record. The setEofEachMember() method may be used to change behavior to mimic that of pre-6u23 GZIPInputStream: reaching the end of a GZIP member will result in a returned EOF. When receiving this EOF, getAtMemberEnd() will return true and getCurrentMemberEnd() will return the (exclusive) member-end position. Calling nextMember() after receiving an EOF will allow reading to proceed into the next member (if any).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	GZIPMembersInputStream.GZIPEnvelopeIterator Provides iterator-ish interface to members in a concatenated multi-member GZIP stream for backward compatibility with our prior workaround.

Field Summary

Fields

Modifier and Type	Field and Description
protected long	currentMemberEnd
protected long	currentMemberStart
protected long	holdAtMemberNumber
protected long	memberNumber
protected InputStream	originalIn

Fields inherited from class org.archive.util.zip.OpenJDK7GZIPInputStream

crc, eos, FCOMMENT, FEXTRA, FHCRC, FNAME, FTEXT, GZIP_MAGIC

Fields inherited from class org.archive.util.zip.OpenJDK7InflaterInputStream

buf, inf, len, usesDefaultInflater

Fields inherited from class java.io.FilterInputStream

in

Constructor Summary

Constructors

Constructor and Description
GZIPMembersInputStream(InputStream in)
GZIPMembersInputStream(InputStream in, int size)

Method Summary

Methods

Modifier and Type	Method and Description
void	compressedSeek(long position) Seek forward to a particular offset in the compressed stream.
void	compressedSkip(long offset) Skip forward the given number of bytes in the compressed stream.
protected static InputStream	countingStream(InputStream in, int lookback) A CountingInputStream is inserted to read compressed-offsets.
boolean	getAtMemberEnd() Test whether last read resulted in reaching the exact end of one GZIP member.
long	getCurrentMemberEnd() Get the compressed offset where the current, just-completed member ends.
long	getCurrentMemberStart() Get the compressed offset where the current member began.
Inflater	getInflater() Helpful for testing/debugging
long	getMemberNumber() Get the ordinal number, starting at zero, of the currently-being-read GZIP member, counting from the creation of this stream.
Iterator<GZIPMembersInputStream>	memberIterator() Deprecated. for backward compatibility; better to use direct facilities in future
void	nextMember() Advance to next member (if the stream has been set to return EOF at the end of each member).
int	read(byte[] buf, int off, int len) Reads uncompressed data into an array of bytes.
protected boolean	readTrailer()
void	setEofEachMember(boolean eofPerMember) Set stream behavior to match JDK 6u22-and-earlier behavior, where reaching the end of any one GZIP member results in EOFs from all read()s as if no more data is available.
protected void	startNewMember()
protected void	updateInnerMark()

Methods inherited from class org.archive.util.zip.OpenJDK7GZIPInputStream

close, readHeader, readUByte, readUInt, readUShort, skipBytes

Methods inherited from class org.archive.util.zip.OpenJDK7InflaterInputStream

available, fill, mark, markSupported, read, reset, skip

Methods inherited from class java.io.FilterInputStream

read

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

memberNumber

protected long memberNumber

holdAtMemberNumber

protected long holdAtMemberNumber

currentMemberStart

protected long currentMemberStart

currentMemberEnd

protected long currentMemberEnd

originalIn

protected InputStream originalIn

Constructor Detail

GZIPMembersInputStream

public GZIPMembersInputStream(InputStream in)
                       throws IOException

Throws:

IOException

GZIPMembersInputStream

public GZIPMembersInputStream(InputStream in,
                      int size)
                       throws IOException

Throws:

IOException

Method Detail

countingStream

protected static InputStream countingStream(InputStream in,
                         int lookback)
                                     throws IOException

A CountingInputStream is inserted to read compressed-offsets.

Parameters:

in - stream to wrap

lookback - tolerance of initial mark

Returns:

original stream wrapped in CountingInputStream

Throws:

IOException

updateInnerMark

protected void updateInnerMark()

read

public int read(byte[] buf,
       int off,
       int len)
         throws IOException

Description copied from class: OpenJDK7GZIPInputStream

Reads uncompressed data into an array of bytes. If len is not zero, the method will block until some input can be decompressed; otherwise, no bytes are read and 0 is returned.

Overrides:

read in class OpenJDK7GZIPInputStream

Parameters:

buf - the buffer into which the data is read

off - the start offset in the destination array b

len - the maximum number of bytes read

Returns:

the actual number of bytes read, or -1 if the end of the compressed input stream is reached

Throws:

ZipException - if the compressed input data is corrupt.

IOException - if an I/O error has occurred.

readTrailer

protected boolean readTrailer()
                       throws IOException

Overrides:

readTrailer in class OpenJDK7GZIPInputStream

Throws:

IOException

compressedSeek

public void compressedSeek(long position)
                    throws IOException

Seek forward to a particular offset in the compressed stream. Note that after any seek/skip the memberNumbers may not reflect a member's true ordinal position from the beginning of the stream.

Parameters:

position - target position

Throws:

IOException

compressedSkip

public void compressedSkip(long offset)
                    throws IOException

Skip forward the given number of bytes in the compressed stream. Note that after any seek/skip the memberNumbers may not reflect a member's true ordinal position from the beginning of the stream.

Parameters:

offset - bytes to skip

Throws:

IOException

EOFException

startNewMember

protected void startNewMember()
                       throws IOException

Throws:

IOException

getAtMemberEnd

public boolean getAtMemberEnd()

Test whether last read resulted in reaching the exact end of one GZIP member.

Returns:

true if exactly at member end

getMemberNumber

public long getMemberNumber()

Get the ordinal number, starting at zero, of the currently-being-read GZIP member, counting from the creation of this stream. If reading straight through, this will be an accurate index relative to all members in the underlying stream. If any seeks/skips have been used, the number will only be relative to the members actually read.

Returns:

ordinal number of member-in-progres

getCurrentMemberStart

public long getCurrentMemberStart()

Get the compressed offset where the current member began.

Returns:

position in compressed stream where current member began

getCurrentMemberEnd

public long getCurrentMemberEnd()

Get the compressed offset where the current, just-completed member ends. Only accurate after the read which finishes a member (when getAtMemberEnd returns true). Otherwise, returns -1 to indicate not-yet-found.

Returns:

position in compressed stream where member just finished, or -1 if member end not yet reached

setEofEachMember

public void setEofEachMember(boolean eofPerMember)

Set stream behavior to match JDK 6u22-and-earlier behavior, where reaching the end of any one GZIP member results in EOFs from all read()s as if no more data is available. (However, nextMember() may be used to advance to the next member.)

Parameters:

eofPerMember - true to set EOF-each-member behavior

nextMember

public void nextMember()

Advance to next member (if the stream has been set to return EOF at the end of each member). Each call before reaching the end of a member will cause one additional member boundary to be passed. (Has no effect if not in EOF-each-member mode.)

getInflater

public Inflater getInflater()

Helpful for testing/debugging

Returns:

Inflater

memberIterator

public Iterator<GZIPMembersInputStream> memberIterator()

Deprecated. for backward compatibility; better to use direct facilities in future

Get an Iterator-ish interface to each member in turn. Has the effect of putting stream in EOF-each-member mode; thereafter reading should occur through the stream returned by the iterator's next(). Reading of one stream from next() should finish (reaching EOF) before the iterator's hasNext() or next() is called.

Returns:

Iterator of