Skip navigation links
C G N S T 

C

createInputStream(InputStream, Decompressor, long, long, SplittableCompressionCodec.READ_MODE) - Method in class nl.basjes.hadoop.io.compress.SplittableGzipCodec
 
createInputStream(InputStream, Decompressor) - Method in class nl.basjes.hadoop.io.compress.SplittableGzipCodec
 

G

getBytesRead() - Method in class nl.basjes.hadoop.io.compress.ThrottleableDecompressorStream
How many bytes have been read from disk so far?
getCompressedData() - Method in class nl.basjes.hadoop.io.compress.ThrottleableDecompressorStream
Reads the compressed input into the buffer.

N

nl.basjes.hadoop.io.compress - package nl.basjes.hadoop.io.compress
Provides the classes to make it possible to read and process a gzipped input file using multiple hadoop map tasks running concurrently.

S

setReadStep(int) - Method in class nl.basjes.hadoop.io.compress.ThrottleableDecompressorStream
The compressed data is read from disk in blocks the size of the allocated buffer.
SplittableGzipCodec - Class in nl.basjes.hadoop.io.compress
For each "split" the gzipped input file is read from the beginning of the file till the point where the split starts, thus reading, decompressing and discarding (wasting!) everything that is before the start of the split.

FACT: Files compressed with the Gzip codec are NOT SPLITTABLE.
SplittableGzipCodec() - Constructor for class nl.basjes.hadoop.io.compress.SplittableGzipCodec
 

T

ThrottleableDecompressorStream - Class in nl.basjes.hadoop.io.compress
A DecompressorStream that allows a dynamic reduction in block size of the file read from disk.
ThrottleableDecompressorStream(InputStream, Decompressor, int) - Constructor for class nl.basjes.hadoop.io.compress.ThrottleableDecompressorStream
 
ThrottleableDecompressorStream(InputStream, Decompressor) - Constructor for class nl.basjes.hadoop.io.compress.ThrottleableDecompressorStream
 
C G N S T 

Copyright © 2019. All Rights Reserved.