Package com.google.cloud.hadoop.util

Class AbstractGoogleAsyncWriteChannel<T>

java.lang.Object
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel<T>
Type Parameters:
T - The type of the result of the completed upload operation.
All Implemented Interfaces:
Closeable, AutoCloseable, Channel, WritableByteChannel
public abstract class AbstractGoogleAsyncWriteChannel<T> extends Object implements WritableByteChannel
Skeletal implementation of a WritableByteChannel that executes an asynchronous upload operation and optionally handles the result.

  • Field Details

    • logger

      protected static final com.google.common.flogger.GoogleLogger logger

    • threadPool

      protected final ExecutorService threadPool

    • channelOptions

      protected final AsyncWriteChannelOptions channelOptions

    • uploadOperation

      protected Future<T> uploadOperation

    • cumulativeCrc32cHasher

      protected final com.google.common.hash.Hasher cumulativeCrc32cHasher

  • Constructor Details

    • AbstractGoogleAsyncWriteChannel

      public AbstractGoogleAsyncWriteChannel(ExecutorService threadPool, AsyncWriteChannelOptions channelOptions)
      Construct a new channel using the given ExecutorService to run background uploads.

  • Method Details

    • handleResponse

      public void handleResponse(T response) throws IOException
      Handle the API response.

      This method is invoked after the upload has completed on the same thread that invokes close(). It can throw IOException if checksum matching is enabled and mismatched.

      Parameters:
      response - The API response object.
      Throws:
      IOException

    • isDirectUploadEnabled

      public boolean isDirectUploadEnabled()
      Returns true if direct media uploads are enabled.

    • write

      public int write(ByteBuffer buffer) throws IOException
      Writes contents of the given buffer to this channel.

      Note: The data that one writes gets written to a pipe which must not block if the pipe has sufficient buffer space. A success code returned from this method does not mean that the specific data was successfully written to the underlying storage. It simply means that there is no error at present. The data upload may encounter an error on a separate thread. Such error is not ignored; it shows up as an exception during a subsequent call to write() or close(). The only way to be sure of successful upload is when the close() method returns successfully.

      Specified by:
      write in interface WritableByteChannel
      Parameters:
      buffer - buffer to write
      Throws:
      IOException - on IO error

    • isOpen

      public boolean isOpen()
      Tells whether this channel is open.
      Specified by:
      isOpen in interface Channel
      Returns:
      a value indicating whether this channel is open

    • close

      public void close() throws IOException
      Closes this channel.

      Note: The method returns only after all data has been successfully written to GCS or if there is a non-retry-able error.

      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Channel
      Specified by:
      close in interface Closeable
      Throws:
      IOException - on IO error

    • initialize

      public void initialize() throws IOException
      Initialize this channel object for writing.
      Throws:
      IOException

    • startUpload

      public abstract void startUpload(InputStream pipeSource) throws IOException
      Create a new thread which handles the upload.
      Throws:
      IOException

    • getResourceString

      protected abstract String getResourceString()