Packages

class ChecksumFSDataInputStream extends FSDataInputStream with Logging

An implementation of FSDataInputStream that calculates the checksum of the file that the client is reading (main file) incrementally, while it is being read. It then does checksum verification on close, to verify that the computed checksum matches the expected checksum in the checksum file.

Computing the checksum incrementally and doing the verification after file read is complete is for better performance, instead of first reading the entire file and doing verification before the client starts reading the file.

Linear Supertypes
Logging, FSDataInputStream, IOStatisticsSource, ByteBufferPositionedReadable, StreamCapabilities, CanUnbuffer, HasEnhancedByteBufferAccess, CanSetReadahead, CanSetDropBehind, HasFileDescriptor, ByteBufferReadable, PositionedReadable, Seekable, DataInputStream, DataInput, FilterInputStream, InputStream, Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ChecksumFSDataInputStream
  2. Logging
  3. FSDataInputStream
  4. IOStatisticsSource
  5. ByteBufferPositionedReadable
  6. StreamCapabilities
  7. CanUnbuffer
  8. HasEnhancedByteBufferAccess
  9. CanSetReadahead
  10. CanSetDropBehind
  11. HasFileDescriptor
  12. ByteBufferReadable
  13. PositionedReadable
  14. Seekable
  15. DataInputStream
  16. DataInput
  17. FilterInputStream
  18. InputStream
  19. Closeable
  20. AutoCloseable
  21. AnyRef
  22. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new ChecksumFSDataInputStream(mainStream: FSDataInputStream, path: Path, expectedChecksumStream: FSDataInputStream, threadPool: ExecutionContext)

    mainStream

    Input stream for the main file the client wants to read

    path

    The path of the main file

    expectedChecksumStream

    The input stream for the checksum file

    threadPool

    Thread pool to use for concurrently operating on the main and checksum file

Type Members

  1. implicit class LogStringContext extends AnyRef
    Definition Classes
    Logging

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def MDC(key: LogKey, value: Any): MDC
    Attributes
    protected
    Definition Classes
    Logging
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def available(): Int
    Definition Classes
    FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
  8. def close(): Unit
    Definition Classes
    ChecksumFSDataInputStream → FilterInputStream → InputStream → Closeable → AutoCloseable
  9. def closeWithoutChecksumVerification(): Unit

    This is used to skip checksum verification on close.

    This is used to skip checksum verification on close. Avoid using this, and it is only used for a situation where the file is opened, read, then closed multiple times, and we want to avoid doing verification each time and only want to do it once.

  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  12. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  13. def getFileDescriptor(): FileDescriptor
    Definition Classes
    FSDataInputStream → HasFileDescriptor
    Annotations
    @throws(classOf[java.io.IOException])
  14. def getIOStatistics(): IOStatistics
    Definition Classes
    FSDataInputStream → IOStatisticsSource
  15. def getPos(): Long
    Definition Classes
    FSDataInputStream → Seekable
    Annotations
    @throws(classOf[java.io.IOException])
  16. def getWrappedStream(): InputStream
    Definition Classes
    FSDataInputStream
    Annotations
    @Public() @Stable()
  17. def hasCapability(arg0: String): Boolean
    Definition Classes
    FSDataInputStream → StreamCapabilities
  18. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @IntrinsicCandidate() @native()
  19. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  20. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  21. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  22. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  23. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  24. def logBasedOnLevel(level: Level)(f: => MessageWithContext): Unit
    Attributes
    protected
    Definition Classes
    Logging
  25. def logDebug(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  26. def logDebug(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  27. def logDebug(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  28. def logDebug(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  29. def logError(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  30. def logError(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  31. def logError(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  32. def logError(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  33. def logInfo(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  34. def logInfo(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  35. def logInfo(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  36. def logInfo(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  37. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  38. def logTrace(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  39. def logTrace(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  40. def logTrace(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  41. def logTrace(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  42. def logWarning(msg: => String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  43. def logWarning(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  44. def logWarning(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    Logging
  45. def logWarning(msg: => String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  46. def mark(arg0: Int): Unit
    Definition Classes
    FilterInputStream → InputStream
  47. def markSupported(): Boolean
    Definition Classes
    FilterInputStream → InputStream
  48. def maxReadSizeForVectorReads(): Int
    Definition Classes
    FSDataInputStream → PositionedReadable
  49. def minSeekForVectorReads(): Int
    Definition Classes
    FSDataInputStream → PositionedReadable
  50. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  51. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  52. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @IntrinsicCandidate() @native()
  53. def read(arg0: Long, arg1: ByteBuffer): Int
    Definition Classes
    FSDataInputStream → ByteBufferPositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  54. final def read(arg0: ByteBufferPool, arg1: Int): ByteBuffer
    Definition Classes
    FSDataInputStream
    Annotations
    @throws(classOf[java.io.IOException]) @throws(classOf[java.lang.UnsupportedOperationException])
  55. def read(arg0: ByteBufferPool, arg1: Int, arg2: EnumSet[ReadOption]): ByteBuffer
    Definition Classes
    FSDataInputStream → HasEnhancedByteBufferAccess
    Annotations
    @throws(classOf[java.io.IOException]) @throws(classOf[java.lang.UnsupportedOperationException])
  56. def read(arg0: ByteBuffer): Int
    Definition Classes
    FSDataInputStream → ByteBufferReadable
    Annotations
    @throws(classOf[java.io.IOException])
  57. def read(arg0: Long, arg1: Array[Byte], arg2: Int, arg3: Int): Int
    Definition Classes
    FSDataInputStream → PositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  58. final def read(arg0: Array[Byte], arg1: Int, arg2: Int): Int
    Definition Classes
    DataInputStream → FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  59. final def read(arg0: Array[Byte]): Int
    Definition Classes
    DataInputStream → FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  60. def read(): Int
    Definition Classes
    FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  61. def readAllBytes(): Array[Byte]
    Definition Classes
    InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  62. final def readBoolean(): Boolean
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  63. final def readByte(): Byte
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  64. final def readChar(): Char
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  65. final def readDouble(): Double
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  66. final def readFloat(): Float
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  67. def readFully(arg0: Long, arg1: ByteBuffer): Unit
    Definition Classes
    FSDataInputStream → ByteBufferPositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  68. def readFully(arg0: Long, arg1: Array[Byte]): Unit
    Definition Classes
    FSDataInputStream → PositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  69. def readFully(arg0: Long, arg1: Array[Byte], arg2: Int, arg3: Int): Unit
    Definition Classes
    FSDataInputStream → PositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  70. final def readFully(arg0: Array[Byte], arg1: Int, arg2: Int): Unit
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  71. final def readFully(arg0: Array[Byte]): Unit
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  72. final def readInt(): Int
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  73. final def readLong(): Long
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  74. def readNBytes(arg0: Array[Byte], arg1: Int, arg2: Int): Int
    Definition Classes
    InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  75. def readNBytes(arg0: Int): Array[Byte]
    Definition Classes
    InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  76. final def readShort(): Short
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  77. final def readUTF(): String
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  78. final def readUnsignedByte(): Int
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  79. final def readUnsignedShort(): Int
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  80. def readVectored(arg0: List[_ <: FileRange], arg1: IntFunction[ByteBuffer], arg2: Consumer[ByteBuffer]): Unit
    Definition Classes
    FSDataInputStream → PositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  81. def readVectored(arg0: List[_ <: FileRange], arg1: IntFunction[ByteBuffer]): Unit
    Definition Classes
    FSDataInputStream → PositionedReadable
    Annotations
    @throws(classOf[java.io.IOException])
  82. def releaseBuffer(arg0: ByteBuffer): Unit
    Definition Classes
    FSDataInputStream → HasEnhancedByteBufferAccess
  83. def reset(): Unit
    Definition Classes
    FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  84. def seek(arg0: Long): Unit
    Definition Classes
    FSDataInputStream → Seekable
    Annotations
    @throws(classOf[java.io.IOException])
  85. def seekToNewSource(arg0: Long): Boolean
    Definition Classes
    FSDataInputStream → Seekable
    Annotations
    @throws(classOf[java.io.IOException])
  86. def setDropBehind(arg0: Boolean): Unit
    Definition Classes
    FSDataInputStream → CanSetDropBehind
    Annotations
    @throws(classOf[java.io.IOException]) @throws(classOf[java.lang.UnsupportedOperationException])
  87. def setReadahead(arg0: Long): Unit
    Definition Classes
    FSDataInputStream → CanSetReadahead
    Annotations
    @throws(classOf[java.io.IOException]) @throws(classOf[java.lang.UnsupportedOperationException])
  88. def skip(arg0: Long): Long
    Definition Classes
    FilterInputStream → InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  89. final def skipBytes(arg0: Int): Int
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException])
  90. def skipNBytes(arg0: Long): Unit
    Definition Classes
    InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  91. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  92. def toString(): String
    Definition Classes
    FSDataInputStream → AnyRef → Any
  93. def transferTo(arg0: OutputStream): Long
    Definition Classes
    InputStream
    Annotations
    @throws(classOf[java.io.IOException])
  94. def unbuffer(): Unit
    Definition Classes
    FSDataInputStream → CanUnbuffer
  95. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  96. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  97. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  98. def withLogContext(context: Map[String, String])(body: => Unit): Unit
    Attributes
    protected
    Definition Classes
    Logging

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable]) @Deprecated
    Deprecated

    (Since version 9)

  2. final def readLine(): String
    Definition Classes
    DataInputStream → DataInput
    Annotations
    @throws(classOf[java.io.IOException]) @Deprecated
    Deprecated

Inherited from Logging

Inherited from FSDataInputStream

Inherited from IOStatisticsSource

Inherited from ByteBufferPositionedReadable

Inherited from StreamCapabilities

Inherited from CanUnbuffer

Inherited from HasEnhancedByteBufferAccess

Inherited from CanSetReadahead

Inherited from CanSetDropBehind

Inherited from HasFileDescriptor

Inherited from ByteBufferReadable

Inherited from PositionedReadable

Inherited from Seekable

Inherited from DataInputStream

Inherited from DataInput

Inherited from FilterInputStream

Inherited from InputStream

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped