Class WriterPool

java.lang.Object
org.archive.io.WriterPool
Direct Known Subclasses:
ARCWriterPool, WARCWriterPool
public abstract class WriterPool extends Object
Pool of Writers. Abstract. Override and pass in the Constructor a factory that creates WriterPoolMember implementations.
Author:
stack

  • Field Details

    • serialNo

      protected final AtomicInteger serialNo
      Used to generate unique filename sequences.

    • DEFAULT_MAX_ACTIVE

      public static final int DEFAULT_MAX_ACTIVE
      Default maximum active number of files in the pool.
      See Also:

    • LARGEST_MAX_ACTIVE

      protected static final int LARGEST_MAX_ACTIVE
      Assumed largest possible value of maxActive; pool will have this maximum capacity, so dynamic changes beyond this number won't work.
      See Also:

    • DEFAULT_MAX_WAIT_FOR_IDLE

      public static final int DEFAULT_MAX_WAIT_FOR_IDLE
      Maximum time to wait on a free file before considering making a new one (if not already at max)
      See Also:

    • settings

      protected final WriterPoolSettings settings
      File settings. Keep in data structure rather than as individual values.

    • maxActive

      protected int maxActive
      maximum number of writers to create at a time

    • maxWait

      protected int maxWait
      maximum ms to wait before considering creation of a writer

    • currentActive

      protected int currentActive
      current count of active writers; only read/mutated in synchronized blocks

    • availableWriters

      protected BlockingQueue<WriterPoolMember> availableWriters
      round-robin queue of available writers

    • lastWriterNeededTime

      protected long lastWriterNeededTime
      system time when writer was last wanted (because one was not ready in time)

    • lastWriterRolloverTime

      protected long lastWriterRolloverTime
      system time when writer was last 'rolled over' (imminent creation of new file)

  • Constructor Details

    • WriterPool

      public WriterPool(AtomicInteger serial, WriterPoolSettings settings, int poolMaximumActive, int poolMaximumWait)
      Constructor
      Parameters:
      serial - Used to generate unique filename sequences
      settings - Settings for this pool.

  • Method Details

    • borrowFile

      public WriterPoolMember borrowFile() throws IOException
      Check out a WriterPoolMember. This method should be followed by a call to returnFile(WriterPoolMember) or invalidateFile(WriterPoolMember) else pool starts leaking.
      Returns:
      Writer checked out of a pool of files or created
      Throws:
      IOException - Problem getting Writer from pool (Converted from Exception to IOException so this pool can live as a good citizen down in depths of ARCSocketFactory).

    • makeNewWriterIfAppropriate

      protected WriterPoolMember makeNewWriterIfAppropriate()
      Create a new writer instance, if still below maxActive count. Remember times to help make later decision when writer should be discarded.
      Returns:
      WriterPoolMember or null if already at max

    • makeWriter

      protected abstract WriterPoolMember makeWriter()
      Returns:
      new WriterPoolMember of appropriate type

    • destroyWriter

      public void destroyWriter(WriterPoolMember writer) throws IOException
      Discard a previously-used writer, cleanly closing it and leaving it out of the pool.
      Parameters:
      writer -
      Throws:
      IOException

    • returnFile

      public void returnFile(WriterPoolMember writer) throws IOException
      Return a writer, for likely reuse unless (1) writer's current file has reached its target size; and (2) there's been no demand for additional writers since the last time a new writer-file was rolled-over. In that case, the possibly-superfluous writer instance is discarded.
      Parameters:
      writer - Writer to return to the pool.
      Throws:
      IOException - Problem returning File to pool.

    • invalidateFile

      public void invalidateFile(WriterPoolMember f) throws IOException
      Close and discard a writer that experienced a potentially-corrupting error.
      Parameters:
      f - writer with problem
      Throws:
      IOException

    • getNumActive

      public int getNumActive() throws UnsupportedOperationException
      Returns:
      Number of WriterPoolMembers checked out of pool.
      Throws:
      UnsupportedOperationException

    • getNumIdle

      public int getNumIdle() throws UnsupportedOperationException
      Returns:
      Number of WriterPoolMember instances still in the pool.
      Throws:
      UnsupportedOperationException

    • close

      public void close()
      Close all WriterPoolMembers in pool.

    • getSettings

      public WriterPoolSettings getSettings()
      Returns:
      Returns settings.

    • getPoolState

      protected String getPoolState()
      Returns:
      State of the pool string

    • getSerialNo

      public AtomicInteger getSerialNo()
      Returns the atomic integer used to generate serial numbers for files.
      Returns:
      the serial number generator

    • drainAllWriters

      protected Collection<WriterPoolMember> drainAllWriters()
      Drains all the active writers from availableWriters, blocking to wait for any writers currently in use to become available.

      When finished with writers, call availableWriters.addAll(...) to put them back into the rotation.

      Returns:
      all the active writers

    • flush

      public void flush()

    • jsonStatus

      public org.json.JSONArray jsonStatus() throws org.json.JSONException
      Throws:
      org.json.JSONException