| Modifier and Type | Class and Description |
|---|---|
class |
DoesNothingPreparedCheckpointer
A special PreparedCheckpointer that does nothing, which can be used when preparing a checkpoint at the current
checkpoint sequence number where it is never necessary to do another checkpoint.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBCheckpointer |
class |
DynamoDBCheckpointFactory |
| Modifier and Type | Class and Description |
|---|---|
class |
DiagnosticUtils |
class |
KinesisRequestsBuilder |
| Modifier and Type | Class and Description |
|---|---|
class |
CoordinatorState
DataModel for CoordinatorState, this data model is used to store various state information required
for coordination across the KCL worker fleet.
|
class |
CoordinatorStateDAO
Data Access Object that routes
CoordinatorState operations to the appropriate
DDB table based on the current TableMigrationStatus. |
class |
DynamicMigrationComponentsInitializer
This class is responsible for initializing the KCL components that supports
3-phase upgrade from v2.x to v3.x.
|
class |
DynamoDbAsyncToSyncClientAdapter
DDB Lock client depends on DynamoDbClient and KCL only has DynamoDbAsyncClient configured.
|
class |
ExecutorStateEvent |
class |
MigrationAdaptiveLeaseAssignmentModeProvider
Provides the lease assignment mode KCL must operate in during migration
from 2.x to 3.x.
|
class |
Scheduler |
class |
SchedulerCoordinatorFactory |
class |
StreamInfoManager |
| Modifier and Type | Interface and Description |
|---|---|
interface |
LAMDataManager
Manages data loading, validation, and cleanup for
LeaseAssignmentManager. |
class |
LAMDataSnapshot
Immutable snapshot of the data loaded by
LAMDataManager for a single
LeaseAssignmentManager assignment cycle. |
interface |
LeaseAssignmentDecider |
class |
LeaseAssignmentManager
Performs the LeaseAssignment for the application.
|
class |
LeaseCountBasedLeaseAssignmentDecider
A lease assignment decider that distributes leases based on lease count balancing.
|
class |
MigrationAwareLAMDataManager
Migration-aware implementation of
LAMDataManager that encapsulates:
Reading leases and worker metrics from the lease table (via EntityDAO) and
optionally the legacy table (via WorkerMetricStatsDAO's legacy delegate)
Filtering out invalid worker metrics (logging + emitting metrics)
Filtering to only valid, non-expired worker metrics
Async deletion of stale worker metrics (using the correct delegate based on origin)
Computing and publishing TableMigrationSummary for migration state decisions
|
class |
VarianceBasedLeaseAssignmentDecider
VarianceBasedLeaseAssignmentDecider
This implementation of LeaseAssignmentDecider performs lease assignment by considering the WorkerMetricStats values of workers
with respect to fleet level average of that WorkerMetricStats.
|
| Modifier and Type | Class and Description |
|---|---|
class |
CoordinatorStateDAODelegate
Data Access Object to abstract accessing
CoordinatorState from
the DDB table it is stored in. |
class |
LeaseTableCoordinatorStateDAODelegate |
class |
LegacyTableCoordinatorStateDAODelegate
Delegate for the legacy (separate) CoordinatorState DDB table.
|
| Modifier and Type | Class and Description |
|---|---|
class |
AbstractMigrationClientVersionState
Abstract base class for
MigrationClientVersionState |
class |
ClientVersionChangeMonitor
Change monitor for MigrationState.clientVersion to notify a callback if the value
changes from a given value.
|
class |
MigrationClientVersion2xState
State for CLIENT_VERSION_2X.
|
class |
MigrationClientVersion3xState
State for CLIENT_VERSION_3X which enables KCL to run 3.x algorithms on new KCLv3.x application
or successfully upgraded application which upgraded from v2.x.
|
class |
MigrationClientVersion3xWithRollbackState
State for CLIENT_VERSION_3X_WITH_ROLLBACK which enables KCL to run its 3.x compliant algorithms
during the upgrade process after all KCL workers in the fleet are 3.x complaint.
|
class |
MigrationClientVersionInitState
State representing the INIT / Phase 1 passive state of the migration state machine.
|
interface |
MigrationClientVersionState
Interface of a state implementation for the MigrationStateMachine
|
class |
MigrationClientVersionStateInitializer
Initializer to determine start state of the state machine which identifies the
state to initialize KCL when it is starting up.
|
class |
MigrationClientVersionUpgradeFrom2xState
State for CLIENT_VERSION_UPGRADE_FROM_2X.
|
class |
MigrationReadyMonitor
Monitor for KCL workers 3.x readiness.
|
class |
MigrationState
Data model of the Migration state.
|
interface |
MigrationStateMachine
State machine that provides:
1.
|
class |
MigrationStateMachineImpl
Implementation of
MigrationStateMachine |
class |
TableMigrationState
Data model of the TableMigration state.
|
interface |
TableMigrationStateMachine
Table State machine that provides:
1.
|
class |
TableMigrationStateMachineImpl
Drives the table migration state machine that consolidates CoordinatorState and WorkerMetricStats
from separate DDB tables into the Lease table.
|
class |
TableMigrationStatus
TableMigration support during upgrade from KCLv3.4 or lower to KCLv3.5 or higher.
|
class |
TableMigrationStatusProvider
Provides the table migration status using which dependent classes
will decide whether to read from Lease Table or Legacy table or
both and merge the results.
|
class |
TableMigrationSummary
Summary computed by the
MigrationAwareLAMDataManager
during data loading, and pushed to a registered consumer for use by the table migration
state machine. |
| Modifier and Type | Class and Description |
|---|---|
class |
StreamIdCache |
class |
StreamIdCacheManager |
class |
StreamInfo
Data model of the StreamInfo state.
|
class |
StreamInfoDAO |
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBLockBasedLeaderDecider
Implementation for LeaderDecider to elect leader using lock on dynamo db table.
|
class |
LeaderLock
Data model of the KCL Lock.
|
class |
MigrationAdaptiveLeaderDecider
MigrationAdaptiveLeaderDecider that wraps around the actual LeaderDecider which can dynamically
change based on the MigrationStateMachine.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoUtils
Static utility functions used by our LeaseSerializers.
|
interface |
EntityDAO
DAO that reads all entity types from the lease table in a single DDB scan.
|
class |
EntityType
Global enum representing all entity types that can be stored in the DynamoDB tables
used by KCL (Lease table, CoordinatorState table).
|
class |
HierarchicalShardSyncer
Helper class to sync leases with shards of the Kinesis stream.
|
class |
KinesisShardDetector |
interface |
LeaseDiscoverer |
class |
LeaseStatsRecorder
This class records the stats for the leases.
|
static class |
LeaseStatsRecorder.LeaseStats |
class |
ShardSyncTask
This task syncs leases/activies with shards of the stream.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBLeaseCoordinator
LeaseCoordinator abstracts away LeaseTaker and LeaseRenewer from the application code that's using leasing.
|
class |
DynamoDBLeaseDiscoverer
An implementation of
LeaseDiscoverer, it uses LeaseRefresher to query
DynamoDBLeaseRefresher.LEASE_OWNER_TO_LEASE_KEY_INDEX_NAME and find the leases assigned
to current worker and then filter and returns the leases that have not started processing (looks at
LeaseRenewer.getCurrentlyHeldLeases() to find out which leases are currently held leases). |
class |
DynamoDBLeaseManagementFactory |
class |
DynamoDBLeaseRefresher
An implementation of
LeaseRefresher that uses DynamoDB. |
class |
DynamoDBLeaseRenewer
An implementation of
LeaseRenewer that uses DynamoDB via LeaseRefresher. |
class |
DynamoDBLeaseSerializer
An implementation of ILeaseSerializer for basic Lease objects.
|
class |
DynamoDBLeaseTableDao
DynamoDB implementation of
EntityDAO that performs a parallel scan of the lease table
and groups all entities by their EntityType. |
class |
DynamoDBLeaseTaker
An implementation of
LeaseTaker that uses DynamoDB via LeaseRefresher. |
| Modifier and Type | Class and Description |
|---|---|
class |
BlockOnParentShardTask
Task to block until processing of all data records in the parent shard(s) is completed.
|
interface |
ConsumerTaskFactory |
class |
InitializeTask
Task for initializing shard position and invoking the ShardRecordProcessor initialize() API.
|
class |
KinesisConsumerTaskFactory |
class |
LeaseGracefulShutdownHandler
This class handles the graceful shutdown of shard consumers.
|
class |
ProcessTask
Task for fetching data records and invoking processRecords() on the record processor instance.
|
class |
ShardConsumer
Responsible for consuming data records of a (specified) shard.
|
class |
ShardConsumerArgument |
class |
ShardConsumerNotifyingSubscriber |
class |
ShardConsumerShutdownNotification
Contains callbacks for completion of stages in a requested record processor shutdown.
|
class |
ShutdownNotificationTask
Notifies record processor of incoming shutdown request, and gives them a chance to checkpoint.
|
class |
ShutdownTask
Task for invoking the ShardRecordProcessor shutdown() callback.
|
| Modifier and Type | Class and Description |
|---|---|
class |
MetricsCollectingTaskDecorator
Decorates an ConsumerTask and reports metrics about its timing and success/failure.
|
| Modifier and Type | Class and Description |
|---|---|
class |
AWSExceptionManager
Traverses a
Throwable class inheritance in search of a mapping
function which will convert that throwable into a RuntimeException. |
class |
BatchUniqueIdentifier |
interface |
GetRecordsResponseAdapter |
class |
IteratorBuilder |
class |
KinesisGetRecordsResponseAdapter |
| Modifier and Type | Class and Description |
|---|---|
class |
FanOutConsumerRegistration |
class |
FanOutRecordsPublisher |
class |
FanOutRetrievalFactory |
| Modifier and Type | Class and Description |
|---|---|
class |
AsynchronousGetRecordsRetrievalStrategy |
class |
BlockingRecordsPublisher
This is the BlockingRecordsPublisher class.
|
class |
KinesisDataFetcher
Used to get data from Amazon Kinesis.
|
class |
KinesisSleepTimeController |
class |
PrefetchRecordsPublisher
This is the prefetch caching class, this class spins up a thread if prefetching is enabled.
|
class |
SimpleRecordsFetcherFactory |
interface |
SleepTimeController |
class |
SleepTimeControllerConfig
Configuration for the
SleepTimeController. |
class |
SynchronousBlockingRetrievalFactory |
class |
SynchronousGetRecordsRetrievalStrategy |
| Modifier and Type | Class and Description |
|---|---|
class |
Cgroup |
class |
DdbUtil |
class |
ExponentialMovingAverage
|
class |
Statistics |
| Modifier and Type | Class and Description |
|---|---|
class |
WorkerMetricsSelector
Class to select appropriate WorkerMetricStats based on the operating range provider that is available on the instance.
|
| Modifier and Type | Class and Description |
|---|---|
class |
Cgroupv1CpuWorkerMetric
Utilizes Linux Control Groups by reading cpu time and available cpu from cgroup directory.This works for Elastic
Kubernetes Service (EKS) containers running on Linux instances which use cgroupv1.
|
class |
Cgroupv2CpuWorkerMetric
Utilizes Linux Control Groups by reading cpu time and available cpu from cgroup directory.
|
class |
EcsCpuWorkerMetric
Queries the Amazon ECS task metadata endpoint version 4 to get CPU metric stats as well as allocated CPU to the ECS task and
containers to calculate percent CPU utilization.
|
| Modifier and Type | Class and Description |
|---|---|
class |
HeapMemoryAfterGCWorkerMetric
Memory WorkerMetricStats that reads the heap memory after GC.
|
| Modifier and Type | Class and Description |
|---|---|
class |
LinuxCpuWorkerMetric
Reads CPU usage statistics out of /proc/stat file that is present on the EC2 instances.
|
class |
LinuxNetworkInWorkerMetric
Ref java doc for
LinuxNetworkWorkerMetricBase |
class |
LinuxNetworkOutWorkerMetric
Ref java doc for
LinuxNetworkWorkerMetricBase |
class |
LinuxNetworkWorkerMetricBase
Base class for EC2NetworkWorkerMetrics, this reads and parses /proc/net/dev file and look for the specific
interface and reads received and transmitted bytes.
|
| Modifier and Type | Class and Description |
|---|---|
class |
WorkerMetricStats
DataModel for a WorkerMetric, this data model is used to store the current state of a Worker in terms of relevant
WorkerMetric(CPU, Memory, Network).
|
class |
WorkerMetricStatsDAO
Data Access Object that routes
WorkerMetricStats operations to the appropriate
DDB table based on the TableMigrationStatus. |
class |
WorkerMetricStatsManager
WorkerMetricStatsManager is a class that manages the collection of raw WorkerMetricStats values for the list of WorkerMetricStats
periodically and store in a bounded in-memory queue.
|
class |
WorkerMetricStatsReporter
Reporter that is periodically executed to report WorkerMetricStats.
|
| Modifier and Type | Class and Description |
|---|---|
class |
LeaseTableWorkerMetricStatsDAODelegate
Delegate for WorkerMetricStats stored in the unified lease table.
|
class |
LegacyTableWorkerMetricStatsDAODelegate
Delegate for the legacy (separate) WorkerMetricStats DDB table.
|
class |
WorkerMetricStatsDAODelegate<T extends WorkerMetricStats>
Data Access Object to abstract accessing
WorkerMetricStats from
the DDB table it is stored in. |
| Modifier and Type | Class and Description |
|---|---|
class |
Ec2Resource
Provides resource metadata for EC2.
|
class |
EcsResource
Provides resource metadata for ECS.
|
class |
EksResource
Provides resource metadata for EKS.
|
class |
OperatingRangeDataProvider
Enum representing the different operating range metadata providers.
|
interface |
ResourceMetadataProvider
Interface for providing resource metadata for worker.
|
Copyright © 2026. All rights reserved.