com.lancedb.lance

Class Fragment

java.lang.Object

com.lancedb.lance.Fragment

public class Fragment
extends Object

Fragment operations.

Constructor Summary

Constructors

Constructor and Description
Fragment(Dataset dataset, FragmentMetadata fragmentMetadata)
Fragment(Dataset dataset, int fragmentId)

Method Summary

Modifier and Type	Method and Description
int	countRows()
static List<FragmentMetadata>	create(String datasetUri, org.apache.arrow.c.ArrowArrayStream stream, WriteParams params) Create a fragment from the given arrow stream.
static List<FragmentMetadata>	create(String datasetUri, org.apache.arrow.memory.BufferAllocator allocator, org.apache.arrow.vector.VectorSchemaRoot root, WriteParams params) Create a fragment from the given data.
FragmentMetadata	deleteRows(List<Integer> rowIndexes) Delete rows by row indexes.
int	getId()
FragmentMergeResult	mergeColumns(org.apache.arrow.c.ArrowArrayStream stream, String leftOn, String rightOn) Merge the new columns into this Fragment, will return the new fragment with the same FragmentId.
FragmentMetadata	metadata()
LanceScanner	newScan() Create a new Dataset Scanner.
LanceScanner	newScan(long batchSize) Create a new Dataset Scanner.
LanceScanner	newScan(ScanOptions options) Create a new Dataset Scanner.
FragmentUpdateResult	updateColumns(org.apache.arrow.c.ArrowArrayStream stream)
FragmentUpdateResult	updateColumns(org.apache.arrow.c.ArrowArrayStream stream, String leftOn, String rightOn) Update existed columns into this Fragment, will return the new fragment with the same FragmentId and updated field ids.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Fragment

public Fragment(Dataset dataset,
                int fragmentId)

Fragment

public Fragment(Dataset dataset,
                FragmentMetadata fragmentMetadata)

Method Detail

metadata

public FragmentMetadata metadata()

newScan

public LanceScanner newScan()

Create a new Dataset Scanner.

Returns:

a dataset scanner

newScan

public LanceScanner newScan(long batchSize)

Create a new Dataset Scanner.

Parameters:

batchSize - scan batch size

Returns:

a dataset scanner

newScan

public LanceScanner newScan(ScanOptions options)

Create a new Dataset Scanner.

Parameters:

options - the scan options

Returns:

a dataset scanner

deleteRows

public FragmentMetadata deleteRows(List<Integer> rowIndexes)

Delete rows by row indexes.

Parameters:

rowIndexes - The row indexes to delete.

Returns:

The fragment metadata after deletion. If all rows are deleted, return Null. Otherwise, returns a new fragment with the updated deletion vector.

getId

public int getId()

countRows

public int countRows()

Returns:

row counts in this Fragment

mergeColumns

public FragmentMergeResult mergeColumns(org.apache.arrow.c.ArrowArrayStream stream,
                                        String leftOn,
                                        String rightOn)

Merge the new columns into this Fragment, will return the new fragment with the same FragmentId. This operation will perform a left-join with the right table (new data in stream) on the column specified by leftOn and rightOn. For every row in current fragment, the new column value is:

1. if no matched row on the right side, null value.
2. if there is exactly one corresponding row on the right side, column value of the matching row.
3. if there are multiple corresponding rows, column value of a random row.

The returned Result will be further committed.

Parameters:

stream - the input data stream

leftOn - column name of current fragment to be joined on.

rightOn - column name of new data to be joined on.

Returns:

the fragment metadata and new schema.

updateColumns

public FragmentUpdateResult updateColumns(org.apache.arrow.c.ArrowArrayStream stream,
                                          String leftOn,
                                          String rightOn)

Update existed columns into this Fragment, will return the new fragment with the same FragmentId and updated field ids. This operation will perform a left-outer-hash-join with the right table (new data in stream) on the column specified by leftOn and rightOn. For every row in current fragment, the updated column value is:

1. if no matched row on the right side, column value of the left side row.
2. if there is exactly one corresponding row on the right side, column value of the matching row.
3. if there are multiple corresponding rows, column value of a random row.

The returned Result will be further committed.

Parameters:

stream - the input data stream

leftOn - column name of current fragment to be joined on.

rightOn - column name of new data to be joined on.

Returns:

the fragment metadata.

updateColumns

public FragmentUpdateResult updateColumns(org.apache.arrow.c.ArrowArrayStream stream)

create

public static List<FragmentMetadata> create(String datasetUri,
                                            org.apache.arrow.memory.BufferAllocator allocator,
                                            org.apache.arrow.vector.VectorSchemaRoot root,
                                            WriteParams params)

Create a fragment from the given data.

Parameters:

datasetUri - the dataset uri

allocator - the buffer allocator

root - the vector schema root

params - the write params

Returns:

the fragment metadata

create

public static List<FragmentMetadata> create(String datasetUri,
                                            org.apache.arrow.c.ArrowArrayStream stream,
                                            WriteParams params)

Create a fragment from the given arrow stream.

Parameters:

datasetUri - the dataset uri

stream - the arrow stream

params - the write params

Returns:

the fragment metadata