software.amazon.kinesis.coordinator.migration

Class TableMigrationStateMachineImpl

java.lang.Object

software.amazon.kinesis.coordinator.migration.TableMigrationStateMachineImpl

All Implemented Interfaces:

TableMigrationStateMachine

@ThreadSafe
public class TableMigrationStateMachineImpl
extends Object
implements TableMigrationStateMachine

Drives the table migration state machine that consolidates CoordinatorState and WorkerMetricStats from separate DDB tables into the Lease table.

This state machine always uses the legacy coordinator DAO for reads and writes of the TableMigrationState. The only exception is the final PENDING → COMPLETE transition where the migration state is written to both the legacy table and the lease table as part of copying all entries.

For the logic on how the TableMigrationStatus affects how CoordinatorState and WorkerMetricStats are read or written, refer to the respective DAO java doc.

State transitions:

 (no state in DDB + no legacy table)
      |
      +---> COMPLETE   [2-to-3 migration short-circuit, no state written]

 (no state in DDB + legacy table exists + config=true)
      |
      +---> InvalidStateException  [Phase 2 cannot start without Phase 1]

 (no state in DDB + legacy table exists + config=false)
      |
      +---> (local INIT, no DDB write yet)
               |
               | leader: writes INIT to DDB when min support code is met (sets steadySinceEpoch)
               | leader: after bake time with min support code still met transition to DEPLOYED
               v
            DEPLOYED   [Phase 1 complete, safe steady state]
               |
               | leader (config=true): legacy worker metrics empty
               | leader: starts async move of coordinator state entries to lease table
               | leader: after async move completes + legacy table verified empty
               | leader: writes PENDING to DDB (starts bake timer)
               | leader: after bake time with min support code still met transition to PENDING
               v
            PENDING ----------------------------+
               |                                |
               | leader: legacy still empty     | leader: legacy non-empty
               | + bake time elapsed            |   (Phase 2 rollback detected)
               v                                v
            COMPLETE                         DEPLOYED
 

DEPLOYED → PENDING transition:

When all workers are emitting metrics only to the lease table (DDB TableMigrationStatus=DEPLOYED), the leader:

1. Asynchronously moves all coordinator state entries (except the lock and migration state) from the legacy table to the lease table using transactional batches with leader fencing.
2. After the async move completes, verifies the legacy table is empty (except lock and migration state entries).
3. Writes PENDING to the legacy DDB table, starting the bake timer.

PENDING → COMPLETE transition:

When the bake time elapses in PENDING state (validating that reads from the lease table work correctly), the leader:

1. Creates the TableMigrationState with COMPLETE status conditionally in both the legacy table and the lease table.
2. Updates the TableMigrationStatusProvider to COMPLETE.
3. Throws InvalidStateException to signal the caller to release the current leader lock (held on legacy table) so the next cycle acquires the lock from the lease table.

Worker behavior (config-driven):

Workers determine their effective local status from DDB state + config. Only COMPLETE from DDB changes behavior unconditionally. All other states are overridden by config:

Refer to determineStatusForInitialization() javadoc for more details.

Rollback handling:

Phase 2 to Phase 1 (PENDING to DEPLOYED):

Rolled-back workers (config=false) resume writing to legacy table. The leader in PENDING detects non-empty legacy metrics and writes DEPLOYED back to DDB. From DEPLOYED, the leader re-evaluates once all workers converge before re-entering PENDING.

Phase 1 to 3.4 (no guaranteed reset):

The 3.4 worker has no table migration code. On rollback, DDB state remains as-is. On rollforward, Phase 1 code evaluates from whatever state was left. Customers should not initiate a second deployment until the first has fully succeeded across all workers.

ThreadSafety: This class's methods are primarily called from the LeaderDecider on the Scheduler thread and from the LAM thread via the LAMDataManager.

See Also:

TableMigrationStatus, CoordinatorConfig#migrateAllEntitiesToLeaseTable(), CoordinatorConfig#tableMigrationCompleteBakeTimeSeconds()

Constructor Summary

Constructors

Constructor and Description
TableMigrationStateMachineImpl(TableMigrationStatusProvider statusProvider, CoordinatorStateDAO coordinatorStateDAO, String workerId, CoordinatorConfig coordinatorConfig, ExecutorService migrationExecutor)

Method Summary

Modifier and Type	Method and Description
void	handleLeaderLockResult(boolean isLeader) Called after leader election to handle table migration state transitions that require leader ownership.
void	initialize() Initialize TableMigrationStateMachine, there can be dependency exception when updating TableMigrationState in DDB or InvalidStateException when 2 phase deployment is not followed.
void	shutdown() Idempotent shutdown of the table migration state machine.
void	updateMigrationSummary(TableMigrationSummary summary) Consumer method to receive migration summary from the LAMDataProvider.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TableMigrationStateMachineImpl

public TableMigrationStateMachineImpl(TableMigrationStatusProvider statusProvider,
                                      CoordinatorStateDAO coordinatorStateDAO,
                                      String workerId,
                                      CoordinatorConfig coordinatorConfig,
                                      ExecutorService migrationExecutor)

Method Detail

initialize

public void initialize()
                throws DependencyException,
                       InvalidStateException

Initialize TableMigrationStateMachine, there can be dependency exception when updating TableMigrationState in DDB or InvalidStateException when 2 phase deployment is not followed. In both case retry will be attempted from Scheduler.initialize and finally fail the application from starting after max retries.

Specified by:

initialize in interface TableMigrationStateMachine

Throws:

DependencyException - if DDB operations fail

InvalidStateException - if the state is inconsistent

handleLeaderLockResult

public void handleLeaderLockResult(boolean isLeader)
                            throws DependencyException,
                                   InvalidStateException

Description copied from interface: TableMigrationStateMachine

Called after leader election to handle table migration state transitions that require leader ownership. The leader drives transitions between states.

When the PENDING → COMPLETE transition completes (async copy done, status updated), this method throws InvalidStateException to signal the caller to release the current leader lock (held on the legacy table). The next isLeader check will use the updated TableMigrationStatusProvider (now COMPLETE) to acquire the lock from the lease table instead.

Specified by:

handleLeaderLockResult in interface TableMigrationStateMachine

Parameters:

isLeader - whether the current worker holds the leader lock

Throws:

DependencyException - if DDB read/write fails — caller should treat as transient and retry on the next cycle (e.g., isLeader returns false for this cycle)

InvalidStateException - when the migration has just completed and the caller must release the current lock so the correct lock can be acquired

shutdown

public void shutdown()

Idempotent shutdown of the table migration state machine. Cancels any in-flight async copy operation and shuts down the dedicated executor used for migration tasks.

After this method returns, no further DDB mutations will be performed by the state machine. This should be called during scheduler shutdown to ensure a leader shutting down mid-copy does not continue mutating DDB tables after the worker considers itself stopped.

Calling this method multiple times is safe and has no additional effect.

Cancels any in-flight async copy and shuts down the dedicated migration executor. After this returns, no new DDB mutations will be initiated by this state machine.

Specified by:

shutdown in interface TableMigrationStateMachine

updateMigrationSummary

public void updateMigrationSummary(TableMigrationSummary summary)

Consumer method to receive migration summary from the LAMDataProvider. Called on the LAM thread each time data is loaded. Thread-safe via synchronized.

This is the method reference passed as Consumer<TableMigrationSummary> when constructing the MigrationAwareLAMDataManager.