Package org.apache.flink.runtime.state.memory

Class MemoryBackendCheckpointStorageAccess

    • Constructor Detail

      • MemoryBackendCheckpointStorageAccess

        public MemoryBackendCheckpointStorageAccess​(org.apache.flink.api.common.JobID jobId,
                                            @Nullable
                                            org.apache.flink.core.fs.Path checkpointsBaseDirectory,
                                            @Nullable
                                            org.apache.flink.core.fs.Path defaultSavepointLocation,
                                            boolean createCheckpointSubDirs,
                                            int maxStateSize)
                                     throws IOException
        Creates a new MemoryBackendCheckpointStorage.
        Parameters:
        jobId - The ID of the job writing the checkpoints.
        checkpointsBaseDirectory - The directory to write checkpoints to. May be null, in which case this storage does not support durable persistence.
        defaultSavepointLocation - The default savepoint directory, or null, if none is set.
        createCheckpointSubDirs - Whether to create sub-directory with name of jobId.
        maxStateSize - The maximum size of each individual piece of state.
        Throws:
        IOException - Thrown if a checkpoint base directory is given configured and the checkpoint directory cannot be created within that directory.

    • Method Detail

      • getMaxStateSize

        public int getMaxStateSize()
        Gets the size (in bytes) that a individual chunk of state may have at most.

      • supportsHighlyAvailableStorage

        public boolean supportsHighlyAvailableStorage()
        Description copied from interface: CheckpointStorageCoordinatorView
        Checks whether this backend supports highly available storage of data.

        Some state backends may not support highly-available durable storage, with default settings, which makes them suitable for zero-config prototyping, but not for actual production setups.

      • initializeLocationForCheckpoint

        public CheckpointStorageLocation initializeLocationForCheckpoint​(long checkpointId)
                                                          throws IOException
        Description copied from interface: CheckpointStorageCoordinatorView
        Initializes a storage location for new checkpoint with the given ID.

        The returned storage location can be used to write the checkpoint data and metadata to and to obtain the pointers for the location(s) where the actual checkpoint data should be stored.

        Parameters:
        checkpointId - The ID (logical timestamp) of the checkpoint that should be persisted.
        Returns:
        A storage location for the data and metadata of the given checkpoint.
        Throws:
        IOException - Thrown if the storage location cannot be initialized due to an I/O exception.

      • resolveCheckpointStorageLocation

        public CheckpointStreamFactory resolveCheckpointStorageLocation​(long checkpointId,
                                                                CheckpointStorageLocationReference reference)
        Description copied from interface: CheckpointStorageWorkerView
        Resolves a storage location reference into a CheckpointStreamFactory.

        The reference may be the default reference, in which case the method should return the default location, taking existing configuration and checkpoint ID into account.

        Parameters:
        checkpointId - The ID of the checkpoint that the location is initialized for.
        reference - The checkpoint location reference.
        Returns:
        A checkpoint storage location reflecting the reference and checkpoint ID.

      • createTaskOwnedStateStream

        public CheckpointStateOutputStream createTaskOwnedStateStream()
        Description copied from interface: CheckpointStorageWorkerView
        Opens a stream to persist checkpoint state data that is owned strictly by tasks and not attached to the life cycle of a specific checkpoint.

        This method should be used when the persisted data cannot be immediately dropped once the checkpoint that created it is dropped. Examples are write-ahead-logs. For those, the state can only be dropped once the data has been moved to the target system, which may sometimes take longer than one checkpoint (if the target system is temporarily unable to keep up).

        The fact that the job manager does not own the life cycle of this type of state means also that it is strictly the responsibility of the tasks to handle the cleanup of this data.

        Developer note: In the future, we may be able to make this a special case of "shared state", where the task re-emits the shared state reference as long as it needs to hold onto the persisted state data.

        Returns:
        A checkpoint state stream to the location for state owned by tasks.