Package org.apache.druid.segment.metadata

Class SqlIndexingStateStorage

java.lang.Object

org.apache.druid.segment.metadata.SqlIndexingStateStorage

All Implemented Interfaces:

IndexingStateStorage

public class SqlIndexingStateStorage
extends Object
implements IndexingStateStorage

Database-backed implementation of IndexingStateStorage.

Manages the persistence and retrieval of CompactionState (AKA IndexingState) objects in the metadata storage. Indexing states are uniquely identified by their fingerprints, which are SHA-256 hashes of their content.

This implementation is designed to be called from a single thread and relies on database constraints and the retry mechanism to handle any conflicts. Operations are idempotent - concurrent upserts for the same fingerprint will either succeed or fail with a constraint violation that is safely ignored.

Constructor Summary

Constructors

Constructor	Description
SqlIndexingStateStorage(MetadataStorageTablesConfig dbTables, com.fasterxml.jackson.databind.ObjectMapper jsonMapper, SQLMetadataConnector connector)

Method Summary

Modifier and Type	Method	Description
int	deletePendingIndexingStatesOlderThan(long timestamp)	Deletes pending indexing states older than the given timestamp.
int	deleteUnusedIndexingStatesOlderThan(long timestamp)	Deletes unused indexing states older than the given timestamp.
List<String>	findReferencedIndexingStateMarkedAsUnused()	Finds all indexing state fingerprints which have been marked as unused but are still referenced by some used segments.
Boolean	isIndexingStatePending(String fingerprint)	Checks if the indexing state for the given fingerprint is pending.
int	markIndexingStatesAsActive(List<String> stateFingerprints)	Marks indexing states as active
int	markIndexingStatesAsUsed(List<String> stateFingerprints)	Marks indexing states as used.
int	markUnreferencedIndexingStatesAsUnused()	Marks indexing states as unused if they are not referenced by any used segments.
void	upsertIndexingState(@NotEmpty String dataSource, @NotEmpty String fingerprint, CompactionState indexingState, org.joda.time.DateTime updateTime)	Upserts an indexing state to storage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

SqlIndexingStateStorage

@Inject
public SqlIndexingStateStorage(MetadataStorageTablesConfig dbTables,
 com.fasterxml.jackson.databind.ObjectMapper jsonMapper,
 SQLMetadataConnector connector)

Method Details

upsertIndexingState

public void upsertIndexingState(@NotEmpty
 @NotEmpty String dataSource,
 @NotEmpty
 @NotEmpty String fingerprint,
 @Nonnull
 CompactionState indexingState,
 @Nonnull
 org.joda.time.DateTime updateTime)

Description copied from interface: IndexingStateStorage

Upserts an indexing state to storage.

If a fingerprint already exists, update to reflect proper used state and timestamp. If a fingerprint doesn't exist, inserts a new row with the full state payload.

Specified by:

upsertIndexingState in interface IndexingStateStorage

Parameters:

dataSource - The datasource name

fingerprint - The fingerprint of the indexing state

indexingState - The indexing state to upsert

updateTime - The timestamp for this update

markUnreferencedIndexingStatesAsUnused

public int markUnreferencedIndexingStatesAsUnused()

Description copied from interface: IndexingStateStorage

Marks indexing states as unused if they are not referenced by any used segments.

This is used for cleanup operations.

Specified by:

markUnreferencedIndexingStatesAsUnused in interface IndexingStateStorage

Returns:

Number of rows updated, or 0 if not applicable

findReferencedIndexingStateMarkedAsUnused

public List<String> findReferencedIndexingStateMarkedAsUnused()

Description copied from interface: IndexingStateStorage

Finds all indexing state fingerprints which have been marked as unused but are still referenced by some used segments. This is used for validation/reconciliation. Implementations may return an empty list if not applicable.

Specified by:

findReferencedIndexingStateMarkedAsUnused in interface IndexingStateStorage

Returns:

List of fingerprints, or empty list

markIndexingStatesAsUsed

public int markIndexingStatesAsUsed(List<String> stateFingerprints)

Description copied from interface: IndexingStateStorage

Marks indexing states as used.

This is used for reconciliation operations to avoid deleting states that are still in use.

Specified by:

markIndexingStatesAsUsed in interface IndexingStateStorage

Parameters:

stateFingerprints - List of fingerprints to mark as used

Returns:

Number of rows updated, or 0 if not applicable

markIndexingStatesAsActive

public int markIndexingStatesAsActive(List<String> stateFingerprints)

Description copied from interface: IndexingStateStorage

Marks indexing states as active

Specified by:

markIndexingStatesAsActive in interface IndexingStateStorage

Parameters:

stateFingerprints - List of fingerprints to mark as active

Returns:

Number of rows updated, or 0 if not applicable

deleteUnusedIndexingStatesOlderThan

public int deleteUnusedIndexingStatesOlderThan(long timestamp)

Description copied from interface: IndexingStateStorage

Deletes unused indexing states older than the given timestamp.

This is used for cleanup operations.

Specified by:

deleteUnusedIndexingStatesOlderThan in interface IndexingStateStorage

Parameters:

timestamp - The cutoff timestamp in milliseconds

Returns:

Number of rows deleted, or 0 if not applicable

deletePendingIndexingStatesOlderThan

public int deletePendingIndexingStatesOlderThan(long timestamp)

Description copied from interface: IndexingStateStorage

Deletes pending indexing states older than the given timestamp.

Specified by:

deletePendingIndexingStatesOlderThan in interface IndexingStateStorage

Parameters:

timestamp - The cutoff timestamp in milliseconds

Returns:

Number of rows deleted, or 0 if not applicable

isIndexingStatePending

@Nullable
public Boolean isIndexingStatePending(String fingerprint)

Checks if the indexing state for the given fingerprint is pending.

Useful for testing purposes to verify the pending status of an indexing state.