Package io.delta.kernel

Interface PaginatedScanFilesIterator

All Superinterfaces:
AutoCloseable, Closeable, CloseableIterator<FilteredColumnarBatch>, Iterator<FilteredColumnarBatch>
All Known Implementing Classes:
PaginatedScanFilesIteratorImpl
public interface PaginatedScanFilesIterator extends CloseableIterator<FilteredColumnarBatch>
An iterator over FilteredColumnarBatch, each representing a batch of Scan Files in a paginated scan. This iterator also exposes the page token that can be used to resume the scan from the exact position current page ends in a subsequent request.

This interface extends CloseableIterator and should be closed when the iteration is complete.

  • Method Details

    • getCurrentPageToken

      Optional<Row> getCurrentPageToken()
      Returns an optional page token representing the starting position of next page. This token is used to resume the scan from the exact position current page ends in a subsequent request. Page token also contains metadata for validation purpose, such as detecting changes in query parameters or the underlying log files.

      The page token represents the position of current iterator at the time it's called. If the iterator is only partially consumed, the returned token will always point to the beginning of the next unconsumed FilteredColumnarBatch. This method will return Option.empty() if all data in the Scan is consumed (no more non-empty pages remain).