Package org.apache.druid.data.input

Class IntermediateRowParsingReader<T>

java.lang.Object

org.apache.druid.data.input.IntermediateRowParsingReader<T>

Type Parameters:

T - type of intermediate row. For example, it can be String for text formats.

All Implemented Interfaces:

InputEntityReader

Direct Known Subclasses:

JsonNodeReader, JsonReader, TextReader

public abstract class IntermediateRowParsingReader<T>
extends Object
implements InputEntityReader

InputEntityReader that parses bytes into some intermediate rows first, and then into InputRows. For example, DelimitedValueReader parses bytes into string lines, and then parses those lines into InputRows.

Constructor Summary

Constructors

Constructor	Description
IntermediateRowParsingReader()

Method Summary

Modifier and Type	Method	Description
protected String	intermediateRowAsString(T row)	String representation of an intermediate row.
protected CloseableIterator<T>	intermediateRowIterator()	Creates an iterator of intermediate rows.
protected CloseableIteratorWithMetadata<T>	intermediateRowIteratorWithMetadata()	Same as intermediateRowIterator, but it also contains the metadata such as the line number to generate more informative ParseException.
protected abstract List<InputRow>	parseInputRows(T intermediateRow)	Parses the given intermediate row into a list of InputRows.
CloseableIterator<InputRow>	read()
CloseableIterator<InputRowListPlusRawValues>	sample()
protected InputEntity	source()
protected abstract List<Map<String,Object>>	toMap(T intermediateRow)	Converts the given intermediate row into a Map.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

IntermediateRowParsingReader

public IntermediateRowParsingReader()

Method Details

read

public CloseableIterator<InputRow> read()
                                 throws IOException

Specified by:

read in interface InputEntityReader

Throws:

IOException

sample

public CloseableIterator<InputRowListPlusRawValues> sample()
                                                    throws IOException

Specified by:

sample in interface InputEntityReader

Throws:

IOException

intermediateRowIterator

protected CloseableIterator<T> intermediateRowIterator()
                                                throws IOException

Creates an iterator of intermediate rows. The returned rows will be consumed by parseInputRows(T) and toMap(T). Either this or intermediateRowIteratorWithMetadata() should be implemented

Throws:

IOException

intermediateRowIteratorWithMetadata

protected CloseableIteratorWithMetadata<T> intermediateRowIteratorWithMetadata()
                                                                        throws IOException

Same as intermediateRowIterator, but it also contains the metadata such as the line number to generate more informative ParseException.

Throws:

IOException

intermediateRowAsString

protected String intermediateRowAsString(@Nullable
 T row)

String representation of an intermediate row. Used for error messages.

source

@Nullable
protected InputEntity source()

Returns:

InputEntity which the implementation is reading from. Useful in generating informative ParseExceptions. For example, in case of FileEntity, file name containing erroneous records or in case of HttpEntity, the endpoint containing the erroneous data can be attached to the error message

parseInputRows

protected abstract List<InputRow> parseInputRows(T intermediateRow)
                                          throws IOException,
ParseException

Parses the given intermediate row into a list of InputRows. This should return a non-empty list.

Throws:

ParseException - if it cannot parse the given intermediateRow properly

IOException

toMap

protected abstract List<Map<String,Object>> toMap(T intermediateRow)
                                           throws IOException

Converts the given intermediate row into a Map. The returned JSON will be used by InputSourceSampler. Implementations can use any method to convert the given row into a Map. This should return a non-empty list with the same size of the list returned by parseInputRows(T) or the returned objects will be discarded

Throws:

IOException