Class GZIPMembersInputStream

java.lang.Object
java.io.InputStream
java.io.FilterInputStream
org.archive.util.zip.OpenJDK7InflaterInputStream
org.archive.util.zip.OpenJDK7GZIPInputStream
org.archive.util.zip.GZIPMembersInputStream
All Implemented Interfaces:
Closeable, AutoCloseable
public class GZIPMembersInputStream extends OpenJDK7GZIPInputStream
A replacement for GZIPInputStream; offers GZIP decompression, without any artificial stop after the first member in a concatenated series (in pre-JDK6u23), and offers direct access to discovered GZIP member boundaries (in compressed offsets) via the getMemberNumber(), getCurrentMemberStart(), getCurrentMemberEnd() accessors, both pre- and post- JDK6u23 (but see below for caveat about getCurrentMemberEnd()). (This replaces our previous workaround, 'GzippedInputStream', for pre-JDK6u23 GZIPInputStream behavior.) By default, will read straight through members, returning all uncompressed data from concatenated compressed members as one stream, per the JDK6u23-and-higher behavior. The data returned from a single read() will not straddle a member boundary, *but* only after reading the first byte of the next member can certainty be offered as to whether the previous member ended. Thus, in this default mode, until the end of all input, the getAtMemberEnd() method will always return false, and getCurrentMemberEnd() will always return -1, because any read that discovered a definitive member-end will have begun the next member. In this mode, member-ends should be deduced by watching the increment of getMemberNumber(), and using the start of the current record as the (exclusive) end-position of the previous record. The setEofEachMember() method may be used to change behavior to mimic that of pre-6u23 GZIPInputStream: reaching the end of a GZIP member will result in a returned EOF. When receiving this EOF, getAtMemberEnd() will return true and getCurrentMemberEnd() will return the (exclusive) member-end position. Calling nextMember() after receiving an EOF will allow reading to proceed into the next member (if any).
Author:
gojomo

  • Field Details

    • memberNumber

      protected long memberNumber

    • holdAtMemberNumber

      protected long holdAtMemberNumber

    • currentMemberStart

      protected long currentMemberStart

    • currentMemberEnd

      protected long currentMemberEnd

    • originalIn

      protected InputStream originalIn

  • Constructor Details

  • Method Details

    • countingStream

      protected static InputStream countingStream(InputStream in, int lookback) throws IOException
      A CountingInputStream is inserted to read compressed-offsets.
      Parameters:
      in - stream to wrap
      lookback - tolerance of initial mark
      Returns:
      original stream wrapped in CountingInputStream
      Throws:
      IOException

    • updateInnerMark

      protected void updateInnerMark()

    • read

      public int read(byte[] buf, int off, int len) throws IOException
      Description copied from class: OpenJDK7GZIPInputStream
      Reads uncompressed data into an array of bytes. If len is not zero, the method will block until some input can be decompressed; otherwise, no bytes are read and 0 is returned.
      Overrides:
      read in class OpenJDK7GZIPInputStream
      Parameters:
      buf - the buffer into which the data is read
      off - the start offset in the destination array b
      len - the maximum number of bytes read
      Returns:
      the actual number of bytes read, or -1 if the end of the compressed input stream is reached
      Throws:
      IOException - if an I/O error has occurred.

    • readTrailer

      protected boolean readTrailer() throws IOException
      Overrides:
      readTrailer in class OpenJDK7GZIPInputStream
      Throws:
      IOException

    • compressedSeek

      public void compressedSeek(long position) throws IOException
      Seek forward to a particular offset in the compressed stream. Note that after any seek/skip the memberNumbers may not reflect a member's true ordinal position from the beginning of the stream.
      Parameters:
      position - target position
      Throws:
      IOException

    • compressedSkip

      public void compressedSkip(long offset) throws IOException
      Skip forward the given number of bytes in the compressed stream. Note that after any seek/skip the memberNumbers may not reflect a member's true ordinal position from the beginning of the stream.
      Parameters:
      offset - bytes to skip
      Throws:
      IOException
      EOFException

    • startNewMember

      protected void startNewMember() throws IOException
      Throws:
      IOException

    • getAtMemberEnd

      public boolean getAtMemberEnd()
      Test whether last read resulted in reaching the exact end of one GZIP member.
      Returns:
      true if exactly at member end

    • getMemberNumber

      public long getMemberNumber()
      Get the ordinal number, starting at zero, of the currently-being-read GZIP member, counting from the creation of this stream. If reading straight through, this will be an accurate index relative to all members in the underlying stream. If any seeks/skips have been used, the number will only be relative to the members actually read.
      Returns:
      ordinal number of member-in-progres

    • getCurrentMemberStart

      public long getCurrentMemberStart()
      Get the compressed offset where the current member began.
      Returns:
      position in compressed stream where current member began

    • getCurrentMemberEnd

      public long getCurrentMemberEnd()
      Get the compressed offset where the current, just-completed member ends. Only accurate after the read which finishes a member (when getAtMemberEnd returns true). Otherwise, returns -1 to indicate not-yet-found.
      Returns:
      position in compressed stream where member just finished, or -1 if member end not yet reached

    • setEofEachMember

      public void setEofEachMember(boolean eofPerMember)
      Set stream behavior to match JDK 6u22-and-earlier behavior, where reaching the end of any one GZIP member results in EOFs from all read()s as if no more data is available. (However, nextMember() may be used to advance to the next member.)
      Parameters:
      eofPerMember - true to set EOF-each-member behavior

    • nextMember

      public void nextMember()
      Advance to next member (if the stream has been set to return EOF at the end of each member). Each call before reaching the end of a member will cause one additional member boundary to be passed. (Has no effect if not in EOF-each-member mode.)

    • getInflater

      public Inflater getInflater()
      Helpful for testing/debugging
      Returns:
      Inflater

    • memberIterator

      public Iterator<GZIPMembersInputStream> memberIterator()
      Deprecated.
      for backward compatibility; better to use direct facilities in future
      Get an Iterator-ish interface to each member in turn. Has the effect of putting stream in EOF-each-member mode; thereafter reading should occur through the stream returned by the iterator's next(). Reading of one stream from next() should finish (reaching EOF) before the iterator's hasNext() or next() is called.