Package io.delta.kernel.data

Class FilteredColumnarBatch

Object

io.delta.kernel.data.FilteredColumnarBatch

@Evolving
public class FilteredColumnarBatch
extends Object

Represents a filtered version of ColumnarBatch. Contains original ColumnarBatch with an optional selection vector to select only a subset of rows for the original columnar batch.

The selection vector is of type boolean and has the same size as the data in the corresponding ColumnarBatch. For each row index, a value of true in the selection vector indicates the row at the same index in the data ColumnarBatch is valid; a value of false indicates the row should be ignored. If there is no selection vector then all the rows are valid.

Since:

3.0.0

Constructor Summary

Constructors

Constructor	Description
FilteredColumnarBatch(ColumnarBatch data, Optional<ColumnVector> selectionVector)
FilteredColumnarBatch(ColumnarBatch data, Optional<ColumnVector> selectionVector, String filePath, int preComputedNumSelectedRows)

Method Summary

Modifier and Type	Method	Description
ColumnarBatch	getData()	Return the data as ColumnarBatch.
Optional<String>	getFilePath()	Returns the file path from which the data originates, if available.
Optional<Integer>	getPreComputedNumSelectedRows()
CloseableIterator<Row>	getRows()	Iterator of rows that survived the filter.
Optional<ColumnVector>	getSelectionVector()	Optional selection vector containing one entry for each row in data indicating whether a row is selected or not selected.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

FilteredColumnarBatch

public FilteredColumnarBatch(ColumnarBatch data,
 Optional<ColumnVector> selectionVector)

FilteredColumnarBatch

public FilteredColumnarBatch(ColumnarBatch data,
 Optional<ColumnVector> selectionVector,
 String filePath,
 int preComputedNumSelectedRows)

Method Details

getData

public ColumnarBatch getData()

Return the data as ColumnarBatch. Not all rows in the data are valid for this result. An optional selectionVector determines which rows are selected. If there is no selection vector that means all rows in this columnar batch are valid for this result.

Returns:

all the data read from the file

getFilePath

public Optional<String> getFilePath()

Returns the file path from which the data originates, if available.

Note: The file path may not be present. It is only set if explicitly provided in the constructor.

Returns:

an Optional containing the file path if available, otherwise an empty Optional

getSelectionVector

public Optional<ColumnVector> getSelectionVector()

Optional selection vector containing one entry for each row in data indicating whether a row is selected or not selected. If there is no selection vector then all the rows are valid.

Returns:

an optional ColumnVector indicating which rows are valid

getRows

public CloseableIterator<Row> getRows()

Iterator of rows that survived the filter.

Returns:

Closeable iterator of rows that survived the filter. It is responsibility of the caller to the close the iterator.

getPreComputedNumSelectedRows

public Optional<Integer> getPreComputedNumSelectedRows()

Returns:

an Optional containing the pre-computed number of selected rows, if available.

If present, this value was computed ahead of time and can be used without incurring any additional cost. This occurs in two cases:

• When the selection vector is absent, which implies that all rows are selected — in this case, the number of selected rows is equal to the batch size.
• When the number of selected rows was explicitly pre-computed and passed in.

If empty, the caller must compute the number of selected rows manually from the selection vector.