Package com.clickhouse.client

Class ClickHouseInputStream

java.lang.Object

java.io.InputStream

com.clickhouse.client.ClickHouseInputStream

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

AbstractByteArrayInputStream, AbstractByteBufferInputStream, EmptyInputStream, NonBlockingInputStream

public abstract class ClickHouseInputStream
extends InputStream

Extended input stream for read optimization. Methods like readBuffer(int), readByte(), readBytes(int), and readCustom(ClickHouseDataUpdater) are added to reduce object creation as well as closing the stream when it reaches end of stream. This class is also responsible for creating various input stream as needed.

Field Summary

Fields

Modifier and Type	Field	Description
protected ClickHouseByteBuffer	byteBuffer	Non-null reusable byte buffer.
protected boolean	closed
protected OutputStream	copyTo
static ByteBuffer	EMPTY_BUFFER	Deprecated. will be removed in v0.3.3, please use ClickHouseByteBuffer.EMPTY_BUFFER instead
static byte[]	EMPTY_BYTES	Deprecated. will be removed in v0.3.3, please use ClickHouseByteBuffer.EMPTY_BYTES instead
protected static String	ERROR_INCOMPLETE_READ
protected static String	ERROR_NULL_BYTES
protected static String	ERROR_REUSE_BUFFER
protected static String	ERROR_STREAM_CLOSED
protected ClickHouseFile	file	Underlying file.
protected Runnable	postCloseAction	Optional post close action.

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ClickHouseInputStream(ClickHouseFile file, OutputStream copyTo, Runnable postCloseAction)

Method Summary

Modifier and Type	Method	Description
void	close()
protected void	closeQuietly()	Closes the input stream quietly.
static ClickHouseInputStream	empty()	Gets an empty input stream that produces nothing and cannot be closed.
protected void	ensureOpen()	Checks whether the input stream has been closed and throw an exception if it is.
ClickHouseFile	getUnderlyingFile()	Gets underlying file.
boolean	isClosed()	Checks if the input stream has been closed or not.
static ClickHouseInputStream	of(byte[]... bytes)	Wraps the given byte arrays.
static ClickHouseInputStream	of(ClickHouseDeferredValue<InputStream> deferredInput, int bufferSize, Runnable postCloseAction)	Wraps the deferred input stream.
static ClickHouseInputStream	of(ClickHouseFile file, int bufferSize, Runnable postCloseAction)	Wraps the given file as input stream.
static ClickHouseInputStream	of(File... files)	Wraps the given files.
static ClickHouseInputStream	of(InputStream... inputs)	Wraps the given input streams.
static ClickHouseInputStream	of(InputStream input, int bufferSize)	Wraps the given input stream.
static ClickHouseInputStream	of(InputStream input, int bufferSize, ClickHouseCompression compression)	Wraps the given input stream.
static ClickHouseInputStream	of(InputStream input, int bufferSize, ClickHouseCompression compression, Runnable postCloseAction)	Wraps the given input stream.
static ClickHouseInputStream	of(InputStream input, int bufferSize, Runnable postCloseAction)	Wraps the given input stream.
static <T> ClickHouseInputStream	of(Iterable<T> source, Class<T> clazz, Function<T,byte[]> converter, Runnable postCloseAction)	Wraps the given iterable objects as byte array based binary input stream.
static ClickHouseInputStream	of(String... strings)	Wraps the given (UTF-8)strings.
static ClickHouseInputStream	of(URL... urls)	Wraps the given URLs.
static ClickHouseInputStream	of(ByteBuffer... buffers)	Wraps the given byte buffers.
static ClickHouseInputStream	of(BlockingQueue<ByteBuffer> queue, int timeout)	Wraps the given blocking queue.
static ClickHouseInputStream	of(BlockingQueue<ByteBuffer> queue, int timeout, Runnable postCloseAction)	Wraps the given blocking queue.
static <T> ClickHouseInputStream	of(T[] source, Class<T> clazz, Function<T,byte[]> converter, Runnable postCloseAction)	Wraps the given array of object as byte array based binary input stream.
abstract int	peek()	Peeks one byte.
abstract long	pipe(ClickHouseOutputStream output)	Reads all remaining bytes and write into given output stream.
static long	pipe(InputStream input, OutputStream output, byte[] buffer)	Transfers data from input stream to output stream.
static long	pipe(InputStream input, OutputStream output, int bufferSize)	Transfers data from input stream to output stream.
String	readAsciiString()	Reads ascii string from input stream.
String	readAsciiString(int byteLength)	Reads ascii string from input stream.
ClickHouseByteBuffer	readBuffer(int length)	Reads byte buffer from the input stream.
abstract byte	readByte()	Reads one single byte from the input stream.
byte[]	readBytes(int length)	Reads length bytes from the input stream.
abstract ClickHouseByteBuffer	readCustom(ClickHouseDataUpdater reader)	Reads bytes using custom reader.
String	readString(int byteLength, Charset charset)	Reads binary string from the input stream.
String	readString(Charset charset)	Reads binary string from the input stream.
String	readUnicodeString()	Reads unicode string from input stream.
String	readUnicodeString(int byteLength)	Reads unicode string from input stream.
int	readUnsignedByte()	Reads an unsigned byte from the input stream.
int	readVarInt()	Reads a varint from input stream.
long	readVarLong()	Reads 64-bit varint as long from input stream.
static File	save(File file, InputStream in, int bufferSize, int timeout, boolean deleteOnExit)
static File	save(InputStream in, int bufferSize, int timeout)
void	setCopyToTarget(OutputStream out)	Sets target output stream to copy bytes to.
static ClickHouseInputStream	wrap(ClickHouseFile file, InputStream input, int bufferSize, Runnable postCloseAction, ClickHouseCompression compression, int compressionLevel)	Wraps the given input stream.

Methods inherited from class java.io.InputStream

available, mark, markSupported, nullInputStream, read, read, read, readAllBytes, readNBytes, readNBytes, reset, skip, transferTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EMPTY_BYTES

@Deprecated
public static final byte[] EMPTY_BYTES

Deprecated.

will be removed in v0.3.3, please use ClickHouseByteBuffer.EMPTY_BYTES instead

Empty byte array.

EMPTY_BUFFER

@Deprecated
public static final ByteBuffer EMPTY_BUFFER

Deprecated.

will be removed in v0.3.3, please use ClickHouseByteBuffer.EMPTY_BUFFER instead

Empty and read-only byte buffer.

ERROR_INCOMPLETE_READ

protected static final String ERROR_INCOMPLETE_READ

See Also:

Constant Field Values

ERROR_NULL_BYTES

protected static final String ERROR_NULL_BYTES

See Also:

Constant Field Values

ERROR_REUSE_BUFFER

protected static final String ERROR_REUSE_BUFFER

See Also:

Constant Field Values

ERROR_STREAM_CLOSED

protected static final String ERROR_STREAM_CLOSED

See Also:

Constant Field Values

byteBuffer

protected final ClickHouseByteBuffer byteBuffer

Non-null reusable byte buffer.

file

protected final ClickHouseFile file

Underlying file.

postCloseAction

protected final Runnable postCloseAction

Optional post close action.

closed

protected boolean closed

copyTo

protected OutputStream copyTo

Constructor Detail

ClickHouseInputStream

protected ClickHouseInputStream(ClickHouseFile file,
                                OutputStream copyTo,
                                Runnable postCloseAction)

Method Detail

wrap

public static ClickHouseInputStream wrap(ClickHouseFile file,
                                         InputStream input,
                                         int bufferSize,
                                         Runnable postCloseAction,
                                         ClickHouseCompression compression,
                                         int compressionLevel)

Wraps the given input stream.

Parameters:

file - wrapped file, could be null

input - non-null input stream

bufferSize - buffer size

postCloseAction - custom action will be performed right after closing the wrapped input stream

compression - compression algorithm

compressionLevel - compression level

Returns:

non-null wrapped input stream

empty

public static ClickHouseInputStream empty()

Gets an empty input stream that produces nothing and cannot be closed.

Returns:

empty input stream

of

public static ClickHouseInputStream of(BlockingQueue<ByteBuffer> queue,
                                       int timeout)

Wraps the given blocking queue.

Parameters:

queue - non-null blocking queue

timeout - read timeout in milliseconds

Returns:

wrapped input

of

public static ClickHouseInputStream of(BlockingQueue<ByteBuffer> queue,
                                       int timeout,
                                       Runnable postCloseAction)

Wraps the given blocking queue.

Parameters:

queue - non-null blocking queue

timeout - read timeout in milliseconds

postCloseAction - custom action will be performed right after closing the input stream

Returns:

wrapped input

of

public static ClickHouseInputStream of(ClickHouseDeferredValue<InputStream> deferredInput,
                                       int bufferSize,
                                       Runnable postCloseAction)

Wraps the deferred input stream.

Parameters:

deferredInput - non-null deferred input stream

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

postCloseAction - custom action will be performed right after closing the input stream

Returns:

wrapped input

of

public static ClickHouseInputStream of(ClickHouseFile file,
                                       int bufferSize,
                                       Runnable postCloseAction)

Wraps the given file as input stream.

Parameters:

file - non-null file

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

postCloseAction - custom action will be performed right after closing the input stream

Returns:

wrapped input

of

public static ClickHouseInputStream of(InputStream input,
                                       int bufferSize)

Wraps the given input stream.

Parameters:

input - input stream

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

Returns:

wrapped input, or the same input if it's instance of ClickHouseInputStream

of

public static ClickHouseInputStream of(InputStream input,
                                       int bufferSize,
                                       Runnable postCloseAction)

Wraps the given input stream.

Parameters:

input - input stream

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

postCloseAction - custom action will be performed right after closing the input stream

Returns:

wrapped input, or the same input if it's instance of ClickHouseInputStream

of

public static ClickHouseInputStream of(InputStream input,
                                       int bufferSize,
                                       ClickHouseCompression compression)

Wraps the given input stream.

Parameters:

input - input stream

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

compression - compression algorithm, null or ClickHouseCompression.NONE means no compression

Returns:

wrapped input, or the same input if it's instance of ClickHouseInputStream

of

public static ClickHouseInputStream of(InputStream input,
                                       int bufferSize,
                                       ClickHouseCompression compression,
                                       Runnable postCloseAction)

Wraps the given input stream.

Parameters:

input - input stream

bufferSize - buffer size which is always greater than zero(usually 8192 or larger)

compression - compression algorithm, null or ClickHouseCompression.NONE means no compression

postCloseAction - custom action will be performed right after closing the input stream

Returns:

wrapped input, or the same input if it's instance of ClickHouseInputStream

of

public static ClickHouseInputStream of(byte[]... bytes)

Wraps the given byte arrays.

Parameters:

bytes - array of byte array

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static ClickHouseInputStream of(ByteBuffer... buffers)

Wraps the given byte buffers.

Parameters:

buffers - array of byte buffer

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static ClickHouseInputStream of(File... files)

Wraps the given files.

Parameters:

files - array of file

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static ClickHouseInputStream of(InputStream... inputs)

Wraps the given input streams.

Parameters:

inputs - array of input stream

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static ClickHouseInputStream of(String... strings)

Wraps the given (UTF-8)strings.

Parameters:

strings - array of string

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static ClickHouseInputStream of(URL... urls)

Wraps the given URLs.

Parameters:

urls - array of URL

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static <T> ClickHouseInputStream of(T[] source,
                                           Class<T> clazz,
                                           Function<T,byte[]> converter,
                                           Runnable postCloseAction)

Wraps the given array of object as byte array based binary input stream.

Type Parameters:

T - type of the object

Parameters:

source - array of object

clazz - class of the object

converter - optional transformer to convert each object into byte array

postCloseAction - custom action will be performed right after closing the input stream

Returns:

non-null input stream

See Also:

of(Iterable, Class, Function, Runnable)

of

public static <T> ClickHouseInputStream of(Iterable<T> source,
                                           Class<T> clazz,
                                           Function<T,byte[]> converter,
                                           Runnable postCloseAction)

Wraps the given iterable objects as byte array based binary input stream. byte[], ByteBuffer, InputStream, File, String, and URL are all supported by default.

Type Parameters:

T - type of the object

Parameters:

source - iterable objects(e.g. byte[], ByteBuffer, and String etc.)

clazz - class of the object

converter - optional transformer to convert each object into byte array

postCloseAction - custom action will be performed right after closing the input stream

Returns:

non-null input stream

pipe

public static long pipe(InputStream input,
                        OutputStream output,
                        int bufferSize)
                 throws IOException

Transfers data from input stream to output stream. Input stream will be closed but output stream will remain open. Please pay attention that you need to explictly call output.flush() before closing output stream.

Parameters:

input - non-null input stream, which will be closed

output - non-null output stream, which will remain open

bufferSize - buffer size, zero or negative number will be treated as ClickHouseClientOption.BUFFER_SIZE

Returns:

written bytes

Throws:

IOException - when error occured reading from input stream or writing data to output stream

pipe

public static long pipe(InputStream input,
                        OutputStream output,
                        byte[] buffer)
                 throws IOException

Transfers data from input stream to output stream. Input stream will be closed but output stream will remain open. Please pay attention that you need to explictly call output.flush() before closing output stream.

Parameters:

input - non-null input stream, which will be closed

output - non-null output stream, which will remain open

buffer - non-empty buffer

Returns:

written bytes

Throws:

IOException - when error occured reading from input stream or writing data to output stream

save

public static File save(InputStream in,
                        int bufferSize,
                        int timeout)

save

public static File save(File file,
                        InputStream in,
                        int bufferSize,
                        int timeout,
                        boolean deleteOnExit)

closeQuietly

protected void closeQuietly()

Closes the input stream quietly.

ensureOpen

protected void ensureOpen()
                   throws IOException

Checks whether the input stream has been closed and throw an exception if it is.

Throws:

IOException - when the input stream has been closed

getUnderlyingFile

public ClickHouseFile getUnderlyingFile()

Gets underlying file.

Returns:

non-null underlying file

peek

public abstract int peek()
                  throws IOException

Peeks one byte. It's similar as InputStream.read() except it never changes cursor.

Returns:

the next byte of data, or -1 if the end of the stream is reached

Throws:

IOException - when failed to read value from input stream or reached end of the stream

pipe

public abstract long pipe(ClickHouseOutputStream output)
                   throws IOException

Reads all remaining bytes and write into given output stream. Current input stream will be closed automatically at the end of writing, but output will remain open.

Parameters:

output - non-null output stream

Returns:

bytes being written into output stream

Throws:

IOException - when failed to read value from input stream or reached end of the stream

readUnsignedByte

public int readUnsignedByte()
                     throws IOException

Reads an unsigned byte from the input stream. Unlike InputStream.read(), it will throw IOException if the input stream has been closed.

Returns:

unsigned byte

Throws:

IOException - when failed to read value from input stream or reached end of the stream

readBuffer

public ClickHouseByteBuffer readBuffer(int length)
                                throws IOException

Reads byte buffer from the input stream.

Parameters:

length - byte length

Returns:

non-null byte buffer

Throws:

IOException - when failed to read bytes from input stream, not able to retrieve all bytes, or reached end of the stream

readByte

public abstract byte readByte()
                       throws IOException

Reads one single byte from the input stream. Unlike InputStream.read(), it will throw IOException if the input stream has been closed. In general, this method should be faster than InputStream.read(), especially when it's an input stream backed by byte[] or ByteBuffer.

Returns:

byte value if present

Throws:

IOException - when failed to read value from input stream or reached end of the stream

readBytes

public byte[] readBytes(int length)
                 throws IOException

Reads length bytes from the input stream. It behaves in the same way as DataInput.readFully(byte[]), except it will throw IOException when the input stream has been closed.

Parameters:

length - number of bytes to read

Returns:

byte array and its length should be length

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readCustom

public abstract ClickHouseByteBuffer readCustom(ClickHouseDataUpdater reader)
                                         throws IOException

Reads bytes using custom reader. Stream will be closed automatically when it reached end of stream. However, unlike readBuffer(int), this method will never throw EOFException.

Parameters:

reader - non-null data reader

Returns:

non-null byte buffer

Throws:

IOException - when failed to read bytes from input stream, not able to retrieve all bytes, or reached end of the stream

readString

public String readString(Charset charset)
                  throws IOException

Reads binary string from the input stream. readVarInt() will be first called automatically to understand byte length of the string.

Parameters:

charset - charset, null is treated as StandardCharsets.UTF_8

Returns:

non-null string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readString

public String readString(int byteLength,
                         Charset charset)
                  throws IOException

Reads binary string from the input stream. When byteLength is zero or negative number, this method will always return empty string.

Parameters:

byteLength - length in byte

charset - charset, null is treated as StandardCharsets.UTF_8

Returns:

non-null string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readAsciiString

public String readAsciiString()
                       throws IOException

Reads ascii string from input stream. readVarInt() will be first called automatically to understand byte length of the string.

Returns:

non-null ascii string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readAsciiString

public String readAsciiString(int byteLength)
                       throws IOException

Reads ascii string from input stream. Similar as readString(byteLength, StandardCharsets.US_ASCII).

Parameters:

byteLength - length in byte

Returns:

non-null ascii string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readUnicodeString

public String readUnicodeString()
                         throws IOException

Reads unicode string from input stream.

Returns:

non-null unicode string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readUnicodeString

public String readUnicodeString(int byteLength)
                         throws IOException

Reads unicode string from input stream. Similar as readString(byteLength, null).

Parameters:

byteLength - length in byte

Returns:

non-null unicode string

Throws:

IOException - when failed to read value from input stream, not able to retrieve all bytes, or reached end of the stream

readVarInt

public int readVarInt()
               throws IOException

Reads a varint from input stream.

Returns:

varint

Throws:

IOException - when failed to read value from input stream or reached end of the stream

readVarLong

public long readVarLong()
                 throws IOException

Reads 64-bit varint as long from input stream.

Returns:

64-bit varint

Throws:

IOException - when failed to read value from input stream or reached end of the stream

setCopyToTarget

public final void setCopyToTarget(OutputStream out)
                           throws IOException

Sets target output stream to copy bytes to. This is mainly used for testing, for example: dump input into a file while reading.

Parameters:

out - the output stream to write bytes to

Throws:

IOException - when failed to flush previous target or not able to write remaining bytes in buffer to the given output stream

isClosed

public boolean isClosed()

Checks if the input stream has been closed or not.

Returns:

true if the input stream has been closed; false otherwise

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class InputStream

Throws:

IOException