Package com.clickhouse.client

Class ClickHouseDataProcessor

java.lang.Object

com.clickhouse.client.ClickHouseDataProcessor

Direct Known Subclasses:

ClickHouseRowBinaryProcessor, ClickHouseTabSeparatedProcessor

public abstract class ClickHouseDataProcessor
extends Object

This defines a data processor for dealing with serialization and deserialization of one or multiple ClickHouseFormat. Unlike ClickHouseDeserializer and ClickHouseSerializer, which is for specific column or data type, data processor is a combination of both, and it can handle more scenarios like separator between columns and rows.

Field Summary

Fields

Modifier and Type	Field	Description
protected ClickHouseColumn[]	columns
protected ClickHouseConfig	config
protected ClickHouseRecord	currentRecord
static List<ClickHouseColumn>	DEFAULT_COLUMNS
protected static String	ERROR_FAILED_TO_READ
protected static String	ERROR_FAILED_TO_WRITE
protected static String	ERROR_REACHED_END_OF_STREAM
protected static String	ERROR_UNKNOWN_DATA_TYPE
protected ClickHouseInputStream	input
protected ClickHouseOutputStream	output
protected int	readPosition	Column index shared by read(ClickHouseValue, ClickHouseColumn), records(), and values().
protected Iterator<ClickHouseRecord>	records
protected Map<String,Object>	settings
protected ClickHouseValue[]	templates
protected Iterator<ClickHouseValue>	values

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	ClickHouseDataProcessor(ClickHouseConfig config, ClickHouseInputStream input, ClickHouseOutputStream output, List<ClickHouseColumn> columns, Map<String,Object> settings)	Default constructor.

Method Summary

Modifier and Type	Method	Description
protected static void	buildAggMappings(Map<ClickHouseAggregateFunction,ClickHouseDeserializer<ClickHouseValue>> deserializers, Map<ClickHouseAggregateFunction,ClickHouseSerializer<ClickHouseValue>> serializers, ClickHouseDeserializer<ClickHouseValue> d, ClickHouseSerializer<ClickHouseValue> s, ClickHouseAggregateFunction... types)
protected static <E extends Enum<E>,T extends ClickHouseValue> void	buildMappings(Map<E,ClickHouseDeserializer<? extends ClickHouseValue>> deserializers, Map<E,ClickHouseSerializer<? extends ClickHouseValue>> serializers, ClickHouseDeserializer<T> d, ClickHouseSerializer<T> s, E... types)
protected abstract ClickHouseRecord	createRecord()	Factory method to create a record.
List<ClickHouseColumn>	getColumns()	Gets list of columns to process.
protected Iterator<ClickHouseRecord>	initRecords()	Initializes iterator of ClickHouseRecord for reading values record by record.
protected Iterator<ClickHouseValue>	initValues()	Initializes iterator of ClickHouseValue for reading values one by one.
ClickHouseValue	read(ClickHouseValue value, ClickHouseColumn column)	Reads deserialized value directly from input stream.
protected void	readAndFill(ClickHouseRecord r)	Reads columns(starting from readPosition) from input stream and fill deserialized data into the given record.
protected abstract void	readAndFill(ClickHouseValue value, ClickHouseColumn column)	Reads column(at readPosition from input stream and fill deserialized data into the given value object.
protected abstract List<ClickHouseColumn>	readColumns()	Reads columns from input stream.
Iterable<ClickHouseRecord>	records()	Returns an iterable collection of records which can be walked through in a foreach-loop.
Iterable<ClickHouseValue>	values()	Returns an iterable collection of values which can be walked through in a foreach-loop.
abstract void	write(ClickHouseValue value, ClickHouseColumn column)	Writes serialized value to output stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_COLUMNS

public static final List<ClickHouseColumn> DEFAULT_COLUMNS

ERROR_FAILED_TO_READ

protected static final String ERROR_FAILED_TO_READ

See Also:

Constant Field Values

ERROR_FAILED_TO_WRITE

protected static final String ERROR_FAILED_TO_WRITE

See Also:

Constant Field Values

ERROR_REACHED_END_OF_STREAM

protected static final String ERROR_REACHED_END_OF_STREAM

See Also:

Constant Field Values

ERROR_UNKNOWN_DATA_TYPE

protected static final String ERROR_UNKNOWN_DATA_TYPE

See Also:

Constant Field Values

config

protected final ClickHouseConfig config

input

protected final ClickHouseInputStream input

output

protected final ClickHouseOutputStream output

columns

protected final ClickHouseColumn[] columns

currentRecord

protected final ClickHouseRecord currentRecord

templates

protected final ClickHouseValue[] templates

settings

protected final Map<String,Object> settings

records

protected final Iterator<ClickHouseRecord> records

values

protected final Iterator<ClickHouseValue> values

readPosition

protected int readPosition

Column index shared by read(ClickHouseValue, ClickHouseColumn), records(), and values().

Constructor Detail

ClickHouseDataProcessor

protected ClickHouseDataProcessor(ClickHouseConfig config,
                                  ClickHouseInputStream input,
                                  ClickHouseOutputStream output,
                                  List<ClickHouseColumn> columns,
                                  Map<String,Object> settings)
                           throws IOException

Default constructor.

Parameters:

config - non-null confinguration contains information like format

input - input stream for deserialization, can be null when output is available

output - outut stream for serialization, can be null when input is available

columns - nullable columns

settings - nullable settings

Throws:

IOException - when failed to read columns from input stream

Method Detail

buildAggMappings

protected static void buildAggMappings(Map<ClickHouseAggregateFunction,ClickHouseDeserializer<ClickHouseValue>> deserializers,
                                       Map<ClickHouseAggregateFunction,ClickHouseSerializer<ClickHouseValue>> serializers,
                                       ClickHouseDeserializer<ClickHouseValue> d,
                                       ClickHouseSerializer<ClickHouseValue> s,
                                       ClickHouseAggregateFunction... types)

buildMappings

protected static <E extends Enum<E>,T extends ClickHouseValue> void buildMappings(Map<E,ClickHouseDeserializer<? extends ClickHouseValue>> deserializers,
                                                                                        Map<E,ClickHouseSerializer<? extends ClickHouseValue>> serializers,
                                                                                        ClickHouseDeserializer<T> d,
                                                                                        ClickHouseSerializer<T> s,
                                                                                        E... types)

createRecord

protected abstract ClickHouseRecord createRecord()

Factory method to create a record.

Returns:

new record

initRecords

protected Iterator<ClickHouseRecord> initRecords()

Initializes iterator of ClickHouseRecord for reading values record by record. Usually this should be only called once during instantiation.

Returns:

non-null iterator of ClickHouseRecord

initValues

protected Iterator<ClickHouseValue> initValues()

Initializes iterator of ClickHouseValue for reading values one by one. Usually this should be only called once during instantiation.

Returns:

non-null iterator of ClickHouseValue

readAndFill

protected void readAndFill(ClickHouseRecord r)
                    throws IOException

Reads columns(starting from readPosition) from input stream and fill deserialized data into the given record. This method is only used when iterating through records().

Parameters:

r - non-null record to fill

Throws:

IOException - when failed to read columns from input stream

readAndFill

protected abstract void readAndFill(ClickHouseValue value,
                                    ClickHouseColumn column)
                             throws IOException

Reads column(at readPosition from input stream and fill deserialized data into the given value object. This method is mainly used when iterating through values(). In default implementation, it's also used in readAndFill(ClickHouseRecord) for simplicity.

Parameters:

value - non-null value object to fill

column - non-null type of the value

Throws:

IOException - when failed to read column from input stream

readColumns

protected abstract List<ClickHouseColumn> readColumns()
                                               throws IOException

Reads columns from input stream. Usually this will be only called once during instantiation.

Returns:

non-null list of columns

Throws:

IOException - when failed to read columns from input stream

getColumns

public final List<ClickHouseColumn> getColumns()

Gets list of columns to process.

Returns:

list of columns to process

records

public final Iterable<ClickHouseRecord> records()

Returns an iterable collection of records which can be walked through in a foreach-loop. Please pay attention that: 1) UncheckedIOException might be thrown when iterating through the collection; and 2) it's not supposed to be called for more than once because the input stream will be closed at the end of reading.

Returns:

non-null iterable records

Throws:

UncheckedIOException - when failed to access the input stream

values

public final Iterable<ClickHouseValue> values()

Returns an iterable collection of values which can be walked through in a foreach-loop. In general, this is slower than records(), because the latter reads data in bulk. However, it's particular useful when you're reading large values with limited memory - e.g. a binary field with a few GB bytes. Similarly, the input stream will be closed at the end of reading.

Returns:

non-null iterable values

Throws:

UncheckedIOException - when failed to access the input stream

read

public ClickHouseValue read(ClickHouseValue value,
                            ClickHouseColumn column)
                     throws IOException

Reads deserialized value directly from input stream. Unlike records(), which reads multiple values at a time, this method will only read one for each call.

Parameters:

value - value to update, could be null

column - type of the value, could be null

Returns:

updated value or a new ClickHouseValue when it is null

Throws:

IOException - when failed to read data from input stream

write

public abstract void write(ClickHouseValue value,
                           ClickHouseColumn column)
                    throws IOException

Writes serialized value to output stream.

Parameters:

value - non-null value to be serialized

column - non-null type information

Throws:

IOException - when failed to write data to output stream