org.apache.hudi.common.table.timeline

Class HoodieArchivedTimeline

java.lang.Object

org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

org.apache.hudi.common.table.timeline.HoodieArchivedTimeline

All Implemented Interfaces:

Serializable, HoodieTimeline

public class HoodieArchivedTimeline
extends HoodieDefaultTimeline

Represents the Archived Timeline for the Hoodie table. Instants for the last 12 hours (configurable) is in the ActiveTimeline and the rest are in ArchivedTimeline.

Instants are read from the archive file during initialization and never refreshed. To refresh, clients need to call reload()

This class can be serialized and de-serialized and on de-serialization the FileSystem is re-initialized.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HoodieArchivedTimeline.ArchiveFileVersionComparator Sort files by reverse order of version suffix in file name.

Field Summary

Fields

Modifier and Type	Field and Description
static String	MERGE_ARCHIVE_PLAN_NAME

Fields inherited from class org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

details

Fields inherited from interface org.apache.hudi.common.table.timeline.HoodieTimeline

CLEAN_ACTION, CLEAN_EXTENSION, COMMIT_ACTION, COMMIT_EXTENSION, COMPACTION_ACTION, COMPLETED_COMPACTION_SUFFIX, COMPLETED_EXTENSION, DELTA_COMMIT_ACTION, DELTA_COMMIT_EXTENSION, EQUALS, FULL_BOOTSTRAP_INSTANT_TS, GREATER_THAN, GREATER_THAN_OR_EQUALS, INDEX_COMMIT_EXTENSION, INDEXING_ACTION, INFLIGHT_CLEAN_EXTENSION, INFLIGHT_COMMIT_EXTENSION, INFLIGHT_COMPACTION_EXTENSION, INFLIGHT_DELTA_COMMIT_EXTENSION, INFLIGHT_EXTENSION, INFLIGHT_INDEX_COMMIT_EXTENSION, INFLIGHT_LOG_COMPACTION_EXTENSION, INFLIGHT_REPLACE_COMMIT_EXTENSION, INFLIGHT_RESTORE_EXTENSION, INFLIGHT_ROLLBACK_EXTENSION, INFLIGHT_SAVE_SCHEMA_ACTION_EXTENSION, INFLIGHT_SAVEPOINT_EXTENSION, INIT_INSTANT_TS, INVALID_INSTANT_TS, LESSER_THAN, LESSER_THAN_OR_EQUALS, LOG_COMPACTION_ACTION, METADATA_BOOTSTRAP_INSTANT_TS, REPLACE_COMMIT_ACTION, REPLACE_COMMIT_EXTENSION, REQUESTED_CLEAN_EXTENSION, REQUESTED_COMMIT_EXTENSION, REQUESTED_COMPACTION_EXTENSION, REQUESTED_COMPACTION_SUFFIX, REQUESTED_DELTA_COMMIT_EXTENSION, REQUESTED_EXTENSION, REQUESTED_INDEX_COMMIT_EXTENSION, REQUESTED_LOG_COMPACTION_EXTENSION, REQUESTED_LOG_COMPACTION_SUFFIX, REQUESTED_REPLACE_COMMIT_EXTENSION, REQUESTED_RESTORE_EXTENSION, REQUESTED_ROLLBACK_EXTENSION, REQUESTED_SAVE_SCHEMA_ACTION_EXTENSION, RESTORE_ACTION, RESTORE_EXTENSION, ROLLBACK_ACTION, ROLLBACK_EXTENSION, SAVE_SCHEMA_ACTION_EXTENSION, SAVEPOINT_ACTION, SAVEPOINT_EXTENSION, SCHEMA_COMMIT_ACTION, VALID_ACTIONS_IN_TIMELINE

Constructor Summary

Constructors

Constructor and Description
HoodieArchivedTimeline() Deprecated.
HoodieArchivedTimeline(HoodieTableMetaClient metaClient) Loads all the archived instants.
HoodieArchivedTimeline(HoodieTableMetaClient metaClient, String startTs) Loads completed instants from startTs(inclusive).

Method Summary

Modifier and Type	Method and Description
void	clearInstantDetailsFromMemory(String instantTime)
void	clearInstantDetailsFromMemory(String startTs, String endTs)
static StoragePath	getArchiveLogPath(String archiveFolder)
Option<byte[]>	getInstantDetails(HoodieInstant instant) Read the completed instant details.
HoodieDefaultTimeline	getWriteTimeline() Timeline to just include commits (commit/deltacommit), compaction and replace actions.
void	loadCompactionDetailsInMemory(String compactionInstantTime)
void	loadCompactionDetailsInMemory(String startTs, String endTs)
void	loadCompletedInstantDetailsInMemory()
void	loadInstantDetailsInMemory(String startTs, String endTs)
HoodieArchivedTimeline	reload()

Methods inherited from class org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

containsInstant, containsInstant, containsOrBeforeTimelineStarts, countInstants, empty, filter, filterCompletedAndCompactionInstants, filterCompletedIndexTimeline, filterCompletedInstants, filterCompletedInstantsOrRewriteTimeline, filterCompletedOrMajorOrMinorCompactionInstants, filterInflights, filterInflightsAndRequested, filterInstantsByAction, filterPendingCompactionTimeline, filterPendingExcludingCompaction, filterPendingExcludingLogCompaction, filterPendingExcludingMajorAndMinorCompaction, filterPendingIndexTimeline, filterPendingLogCompactionTimeline, filterPendingMajorOrMinorCompactionTimeline, filterPendingReplaceTimeline, filterPendingRollbackTimeline, filterRequestedRollbackTimeline, findInstantBefore, findInstantsAfter, findInstantsAfter, findInstantsAfterOrEquals, findInstantsBefore, findInstantsBeforeOrEquals, findInstantsInClosedRange, findInstantsInRange, findInstantsInRangeByStateTransitionTime, findInstantsModifiedAfterByStateTransitionTime, firstInstant, firstInstant, getAllCommitsTimeline, getCleanerTimeline, getCommitAndReplaceTimeline, getCommitsAndCompactionTimeline, getCommitsTimeline, getCommitTimeline, getCompletedReplaceTimeline, getContiguousCompletedWriteTimeline, getDeltaCommitTimeline, getFirstNonSavepointCommit, getFirstPendingClusterInstant, getInstants, getInstantsAsStream, getInstantsOrderedByStateTransitionTime, getLastClusteringInstant, getLastPendingClusterInstant, getRestoreTimeline, getReverseOrderedInstants, getRollbackAndRestoreTimeline, getRollbackTimeline, getSavePointTimeline, getTimelineHash, getTimelineOfActions, isBeforeTimelineStarts, isEmpty, isPendingClusterInstant, lastInstant, mergeTimeline, nthFromLastInstant, nthInstant, setInstants, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.table.timeline.HoodieTimeline

compareTimestamps, getCleanInflightInstant, getCleanRequestedInstant, getCommitFromCommitFile, getCompactionInflightInstant, getCompactionRequestedInstant, getCompletedInstant, getIndexInflightInstant, getIndexRequestedInstant, getInflightInstant, getLogCompactionInflightInstant, getLogCompactionRequestedInstant, getReplaceCommitInflightInstant, getReplaceCommitRequestedInstant, getRequestedInstant, getRestoreRequestedInstant, getRollbackRequestedInstant, isInClosedRange, isInRange, makeCleanerFileName, makeCommitFileName, makeDeltaFileName, makeFileNameAsComplete, makeFileNameAsInflight, makeIndexCommitFileName, makeInflightCleanerFileName, makeInflightCommitFileName, makeInflightCompactionFileName, makeInflightDeltaFileName, makeInflightIndexFileName, makeInflightLogCompactionFileName, makeInflightReplaceFileName, makeInflightRestoreFileName, makeInflightRollbackFileName, makeInflightSavePointFileName, makeInflightSchemaFileName, makeReplaceFileName, makeRequestedCleanerFileName, makeRequestedCommitFileName, makeRequestedCompactionFileName, makeRequestedDeltaFileName, makeRequestedIndexFileName, makeRequestedLogCompactionFileName, makeRequestedReplaceFileName, makeRequestedRestoreFileName, makeRequestedRollbackFileName, makeRequestSchemaFileName, makeRestoreFileName, makeRollbackFileName, makeSavePointFileName, makeSchemaFileName

Field Detail

MERGE_ARCHIVE_PLAN_NAME

public static final String MERGE_ARCHIVE_PLAN_NAME

See Also:

Constant Field Values

Constructor Detail

HoodieArchivedTimeline

public HoodieArchivedTimeline(HoodieTableMetaClient metaClient)

Loads all the archived instants. Note that there is no lazy loading, so this may not work if the archived timeline range is really long. TBD: Should we enforce maximum time range?

HoodieArchivedTimeline

public HoodieArchivedTimeline(HoodieTableMetaClient metaClient,
                              String startTs)

Loads completed instants from startTs(inclusive). Note that there is no lazy loading, so this may not work if really early startTs is specified.

HoodieArchivedTimeline

public HoodieArchivedTimeline()

Deprecated.

For serialization and de-serialization only.

Method Detail

getArchiveLogPath

public static StoragePath getArchiveLogPath(String archiveFolder)

loadInstantDetailsInMemory

public void loadInstantDetailsInMemory(String startTs,
                                       String endTs)

loadCompletedInstantDetailsInMemory

public void loadCompletedInstantDetailsInMemory()

loadCompactionDetailsInMemory

public void loadCompactionDetailsInMemory(String compactionInstantTime)

loadCompactionDetailsInMemory

public void loadCompactionDetailsInMemory(String startTs,
                                          String endTs)

clearInstantDetailsFromMemory

public void clearInstantDetailsFromMemory(String instantTime)

clearInstantDetailsFromMemory

public void clearInstantDetailsFromMemory(String startTs,
                                          String endTs)

getInstantDetails

public Option<byte[]> getInstantDetails(HoodieInstant instant)

Description copied from interface: HoodieTimeline

Read the completed instant details.

Specified by:

getInstantDetails in interface HoodieTimeline

Overrides:

getInstantDetails in class HoodieDefaultTimeline

reload

public HoodieArchivedTimeline reload()

getWriteTimeline

public HoodieDefaultTimeline getWriteTimeline()

Description copied from interface: HoodieTimeline

Timeline to just include commits (commit/deltacommit), compaction and replace actions.

Specified by:

getWriteTimeline in interface HoodieTimeline

Overrides:

getWriteTimeline in class HoodieDefaultTimeline

Returns: