Package org.apache.druid.data.input.impl

Class JsonLineReader

java.lang.Object

org.apache.druid.data.input.IntermediateRowParsingReader<T>

org.apache.druid.data.input.TextReader<byte[]>

org.apache.druid.data.input.TextReader.Bytes

org.apache.druid.data.input.impl.JsonLineReader

All Implemented Interfaces:

InputEntityReader

public class JsonLineReader
extends TextReader.Bytes

JsonLineReader reads input text line by line and tries to convert each text line to an JSON object. Since each text line is processed indepdently, if any exception is thrown when parsing one text line, exception can be caught by callers to skip current line and continue to process next text line. This also means that each text line should be a well-formed JSON text, pretty-printed format is not allowed

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.druid.data.input.TextReader

TextReader.Bytes, TextReader.Strings

Method Summary

Modifier and Type	Method	Description
int	getNumHeaderLinesToSkip()	Returns the number of header lines to skip.
boolean	needsToProcessHeaderLine()	Returns true if the file format needs to process a header line.
List<InputRow>	parseInputRows(byte[] line)	Parses the given line into a list of InputRows.
void	processHeaderLine(byte[] line)	Processes a header line.
List<Map<String,Object>>	toMap(byte[] intermediateRow)	Converts the given intermediate row into a Map.

Methods inherited from class org.apache.druid.data.input.TextReader.Bytes

intermediateRowAsString, makeSourceIterator

Methods inherited from class org.apache.druid.data.input.TextReader

findOrCreateInputRowSignature, getInputRowSchema, intermediateRowIteratorWithMetadata, source

Methods inherited from class org.apache.druid.data.input.IntermediateRowParsingReader

intermediateRowIterator, read, sample

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

parseInputRows

public List<InputRow> parseInputRows(byte[] line)
                              throws IOException,
ParseException

Description copied from class: TextReader

Parses the given line into a list of InputRows. Note that some file formats can explode a single line of input into multiple inputRows. This method will be called after TextReader.getNumHeaderLinesToSkip() and TextReader.processHeaderLine(T).

Specified by:

parseInputRows in class TextReader<byte[]>

Throws:

ParseException - if it cannot parse the given intermediateRow properly

IOException

toMap

public List<Map<String,Object>> toMap(byte[] intermediateRow)
                               throws IOException

Description copied from class: IntermediateRowParsingReader

Converts the given intermediate row into a Map. The returned JSON will be used by InputSourceSampler. Implementations can use any method to convert the given row into a Map. This should return a non-empty list with the same size of the list returned by IntermediateRowParsingReader.parseInputRows(T) or the returned objects will be discarded

Specified by:

toMap in class IntermediateRowParsingReader<byte[]>

Throws:

IOException

getNumHeaderLinesToSkip

public int getNumHeaderLinesToSkip()

Description copied from class: TextReader

Returns the number of header lines to skip. TextReader.processHeaderLine(T) will be called as many times as the returned number.

Specified by:

getNumHeaderLinesToSkip in class TextReader<byte[]>

needsToProcessHeaderLine

public boolean needsToProcessHeaderLine()

Description copied from class: TextReader

Returns true if the file format needs to process a header line. This method will be called after skipping lines as many as TextReader.getNumHeaderLinesToSkip().

Specified by:

needsToProcessHeaderLine in class TextReader<byte[]>

processHeaderLine

public void processHeaderLine(byte[] line)

Description copied from class: TextReader

Processes a header line. This will be called if TextReader.needsToProcessHeaderLine() = true.

Specified by:

processHeaderLine in class TextReader<byte[]>