Class Recorder

java.lang.Object
org.archive.util.Recorder
public class Recorder extends Object
Pairs together a RecordingInputStream and RecordingOutputStream to capture exactly a single HTTP transaction. Initially only supports HTTP/1.0 (one request, one response per stream) Call markContentBegin() to demarc the transition between HTTP header and body.
Author:
gojomo

  • Field Details

    • logger

      protected static Logger logger

    • characterEncoding

      protected String characterEncoding
      recording-input (ris) content character encoding.

    • charset

      protected Charset charset
      Charset to use for CharSequence provision. Will be UTF-8 if no encoding ever requested; a Charset matching above characterEncoding if possible; ISO_8859 if above characterEncoding is unsatisfiable. TODO: unify to UTF-8 for unspecified and bad-specified cases? (current behavior is for consistency with our prior but perhaps not optimal behavior)

    • inputIsChunked

      protected boolean inputIsChunked
      whether recording-input (ris) message-body is chunked

    • contentEncoding

      protected String contentEncoding
      recording-input (ris) entity content-encoding (eg gzip, deflate), if any

    • currentRecorder

      protected static ThreadLocal<Recorder> currentRecorder

    • SUPPORTED_ENCODINGS

      protected static Set<String> SUPPORTED_ENCODINGS

  • Constructor Details

    • Recorder

      public Recorder(File tempDir, String backingFilenameBase, int outBufferSize, int inBufferSize)
      Create an HttpRecorder.
      Parameters:
      tempDir - Directory into which we drop backing files for recorded input and output.
      backingFilenameBase - Backing filename base to which we'll append suffices ris for recorded input stream and ros for recorded output stream.
      outBufferSize - Size of output buffer to use.
      inBufferSize - Size of input buffer to use.

    • Recorder

      public Recorder(File file, int outBufferSize, int inBufferSize)

    • Recorder

      public Recorder(File tempDir, String backingFilenameBase)
      Create an HttpRecorder.
      Parameters:
      tempDir - Directory into which we drop backing files for recorded input and output.
      backingFilenameBase - Backing filename base to which we'll append suffices ris for recorded input stream and ros for recorded output stream.

  • Method Details

    • inputWrap

      public InputStream inputWrap(InputStream is) throws IOException
      Wrap the provided stream with the internal RecordingInputStream open() throws an exception if RecordingInputStream is already open.
      Parameters:
      is - InputStream to wrap.
      Returns:
      The input stream wrapper which itself is an input stream. Pass this in place of the passed stream so input can be recorded.
      Throws:
      IOException

    • outputWrap

      public OutputStream outputWrap(OutputStream os) throws IOException
      Wrap the provided stream with the internal RecordingOutputStream open() throws an exception if RecordingOutputStream is already open.
      Parameters:
      os - The output stream to wrap.
      Returns:
      The output stream wrapper which is itself an output stream. Pass this in place of the passed stream so output can be recorded.
      Throws:
      IOException

    • close

      public void close()
      Close all streams.

    • getRecordedInput

      public RecordingInputStream getRecordedInput()
      Return the internal RecordingInputStream
      Returns:
      A RIS.

    • getRecordedOutput

      public RecordingOutputStream getRecordedOutput()
      Returns:
      The RecordingOutputStream.

    • markContentBegin

      public void markContentBegin()
      Mark current position as the point where the HTTP headers end.

    • getResponseContentLength

      public long getResponseContentLength()

    • closeRecorders

      public void closeRecorders()
      Close both input and output recorders. Recorders are the output streams to which we are recording. close() closes the stream that is being recorded and the recorder. This method explicitly closes the recorder only.

    • cleanup

      public void cleanup()
      Cleanup backing files. Call when completely done w/ recorder. Removes any backing files that may have been dropped.

    • setHttpRecorder

      public static void setHttpRecorder(Recorder httpRecorder)

    • getHttpRecorder

      public static Recorder getHttpRecorder()
      Get the current threads' HttpRecorder.
      Returns:
      This threads' HttpRecorder. Returns null if can't find a HttpRecorder in current instance.

    • setCharset

      public void setCharset(Charset cs)
      Parameters:
      cs - Character encoding of input recording.

    • getCharset

      public Charset getCharset()
      Returns:
      effective Charset of input recording

    • setInputIsChunked

      public void setInputIsChunked(boolean chunked)

    • setContentEncoding

      public void setContentEncoding(String contentEncoding)
      Parameters:
      contentEncoding - declared content-encoding of input recording.

    • getContentEncoding

      public String getContentEncoding()
      Returns:
      Returns the characterEncoding.

    • getContentReplayCharSequence

      public ReplayCharSequence getContentReplayCharSequence() throws IOException
      Returns:
      A ReplayCharSequence. Caller may call ReplayCharSequence.close() when finished. However, in heritrix, the ReplayCharSequence is closed automatically when url processing has finished; in that context it's preferable not to close, so that processors can reuse the same instance.
      Throws:
      IOException
      See Also:

    • getContentReplayCharSequence

      public ReplayCharSequence getContentReplayCharSequence(Charset requestedCharset) throws IOException
      Parameters:
      requestedCharset - Encoding of recorded stream.
      Returns:
      A ReplayCharSequence Will return null if an IOException. Call close on returned RCS when done.
      Throws:
      IOException

    • calcRecommendedCharBufferSize

      protected static int calcRecommendedCharBufferSize(RecordingInputStream inStream)
      Calculate a recommended size for an in-memory decoded-character buffer of this content. We seek a size that is itself no larger (in 2-byte chars) than the memory already used by the RecordingInputStream's internal raw byte buffer, and also no larger than likely necessary. So, we take the minimum of the actual recorded byte size and the RecordingInputStream's max buffer size.
      Parameters:
      inStream -
      Returns:
      int length for in-memory decoded-character buffer

    • getReplayInputStream

      public ReplayInputStream getReplayInputStream() throws IOException
      Get a raw replay of all recorded data (including, for example, HTTP protocol headers)
      Returns:
      A replay input stream.
      Throws:
      IOException

    • getMessageBodyReplayInputStream

      public ReplayInputStream getMessageBodyReplayInputStream() throws IOException
      Get a raw replay of the 'message-body'. For the common case of HTTP, this is the raw, possibly chunked-transfer-encoded message contents not including the leading headers.
      Returns:
      A replay input stream.
      Throws:
      IOException

    • getEntityReplayInputStream

      public InputStream getEntityReplayInputStream() throws IOException
      Get a raw replay of the 'entity'. For the common case of HTTP, this is the message-body after any (usually-unnecessary) transfer-decoding but before any content-encoding (eg gzip) decoding
      Returns:
      A replay input stream.
      Throws:
      IOException

    • getContentReplayInputStream

      public InputStream getContentReplayInputStream() throws IOException
      Get a replay cued up for the 'content' (after all leading headers)
      Returns:
      A replay input stream.
      Throws:
      IOException

    • getContentReplayPrefixString

      public String getContentReplayPrefixString(int size)
      Return a short prefix of the presumed-textual content as a String.
      Parameters:
      size - max length of String to return
      Returns:
      String prefix, or empty String (with logged exception) on any error

    • getContentReplayPrefixString

      public String getContentReplayPrefixString(int size, Charset cs)
      Return a short prefix of the presumed-textual content as a String.
      Parameters:
      size - max length of String to return
      Returns:
      String prefix, or empty String (with logged exception) on any error

    • copyContentBodyTo

      public void copyContentBodyTo(File tempFile) throws IOException
      Parameters:
      tempFile -
      Throws:
      IOException

    • wrapInputStreamWithHttpRecord

      public static Recorder wrapInputStreamWithHttpRecord(File dir, String basename, InputStream in, String encoding) throws IOException
      Record the input stream for later playback by an extractor, etc. This is convenience method used to setup an artificial HttpRecorder scenario used in unit tests, etc.
      Parameters:
      dir - Directory to write backing file to.
      basename - of what we're recording.
      in - Stream to read.
      encoding - Stream encoding.
      Returns:
      An Recorder.
      Throws:
      IOException

    • endReplays

      public void endReplays()