Package org.apache.druid.segment.nested

Class NestedDataColumnSerializer

java.lang.Object

org.apache.druid.segment.nested.NestedCommonFormatColumnSerializer

org.apache.druid.segment.nested.NestedDataColumnSerializer

All Implemented Interfaces:

GenericColumnSerializer<StructuredData>, Serializer

public class NestedDataColumnSerializer
extends NestedCommonFormatColumnSerializer

Serializer for NestedCommonFormatColumn which can store nested data. The serializer stores several components including: - a field list and associated type info - value dictionaries for string, long, double, and array values (where the arrays are stored as int[] that point to the string, long, and double values) - raw data is stored with a CompressedVariableSizedBlobColumnSerializer as blobs of SMILE encoded data - a null value bitmap to track which 'raw' rows are null For each nested field, a GlobalDictionaryEncodedFieldColumnWriter will write a sub-column to specialize fast reading and filtering of that path.

See Also:

• - single type double field
• - single type long field
• - single type string field
• - single type array of string, long, and double field
• - mixed type field

Field Summary

Fields inherited from class org.apache.druid.segment.nested.NestedCommonFormatColumnSerializer

NESTED_FIELD_PREFIX, RAW_FILE_NAME, V0

Constructor Summary

Constructors

Constructor	Description
NestedDataColumnSerializer(String name, NestedCommonFormatColumnFormatSpec columnFormatSpec, SegmentWriteOutMedium segmentWriteOutMedium, Closer closer)

Method Summary

Modifier and Type	Method	Description
String	getColumnName()
DictionaryIdLookup	getDictionaryIdLookup()
long	getSerializedSize()	Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher) call.
boolean	hasNulls()
void	open()
void	openDictionaryWriter(File segmentBaseDir)
void	serialize(ColumnValueSelector<? extends StructuredData> selector)
void	serialize(StructuredData data)
void	serializeDictionaries(Iterable<String> strings, Iterable<Long> longs, Iterable<Double> doubles, Iterable<int[]> arrays)
void	serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields)
void	setDictionaryIdLookup(DictionaryIdLookup dictionaryIdLookup)
void	setFieldsAndOpenWriters(NestedDataColumnSerializer serializer)
void	writeTo(WritableByteChannel channel, FileSmoosher smoosher)	Writes the serialized form of this object.

Methods inherited from class org.apache.druid.segment.nested.NestedCommonFormatColumnSerializer

computeFilenameBytes, copyFromTempSmoosh, writeInternal, writeV0Header

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

NestedDataColumnSerializer

public NestedDataColumnSerializer(String name,
 NestedCommonFormatColumnFormatSpec columnFormatSpec,
 SegmentWriteOutMedium segmentWriteOutMedium,
 Closer closer)

Method Details

getColumnName

public String getColumnName()

Specified by:

getColumnName in class NestedCommonFormatColumnSerializer

getDictionaryIdLookup

public DictionaryIdLookup getDictionaryIdLookup()

Specified by:

getDictionaryIdLookup in class NestedCommonFormatColumnSerializer

hasNulls

public boolean hasNulls()

Specified by:

hasNulls in class NestedCommonFormatColumnSerializer

openDictionaryWriter

public void openDictionaryWriter(File segmentBaseDir)
                          throws IOException

Specified by:

openDictionaryWriter in class NestedCommonFormatColumnSerializer

Throws:

IOException

open

public void open()
          throws IOException

Throws:

IOException

serializeFields

public void serializeFields(SortedMap<String,FieldTypeInfo.MutableTypeSet> fields)
                     throws IOException

Overrides:

serializeFields in class NestedCommonFormatColumnSerializer

Throws:

IOException

serializeDictionaries

public void serializeDictionaries(Iterable<String> strings,
 Iterable<Long> longs,
 Iterable<Double> doubles,
 Iterable<int[]> arrays)
                           throws IOException

Specified by:

serializeDictionaries in class NestedCommonFormatColumnSerializer

Throws:

IOException

serialize

public void serialize(ColumnValueSelector<? extends StructuredData> selector)
               throws IOException

Throws:

IOException

serialize

public void serialize(StructuredData data)
               throws IOException

Throws:

IOException

getSerializedSize

public long getSerializedSize()
                       throws IOException

Description copied from interface: Serializer

Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher) call.

Throws:

IOException

writeTo

public void writeTo(WritableByteChannel channel,
 FileSmoosher smoosher)
             throws IOException

Description copied from interface: Serializer

Writes the serialized form of this object. The entire object may be written to the provided channel, or the object may be split over the provided channel and files added to the

invalid @link

{@link FileSmoosher],

where additional channels can be created via FileSmoosher.addWithSmooshedWriter(String, long). The latter approach is useful when the serialized form of the object is too large for a single smoosh container. At the time this javadoc was written, the max smoosh container size is limit to the max ByteBuffer size.

Throws:

IOException

setDictionaryIdLookup

public void setDictionaryIdLookup(DictionaryIdLookup dictionaryIdLookup)

Specified by:

setDictionaryIdLookup in class NestedCommonFormatColumnSerializer

setFieldsAndOpenWriters

public void setFieldsAndOpenWriters(NestedDataColumnSerializer serializer)
                             throws IOException

Throws:

IOException