HoodieTableMetaClient (hudi-common 0.15.0 API)

java.lang.Object
- org.apache.hudi.common.table.HoodieTableMetaClient

All Implemented Interfaces:

Serializable
```
public class HoodieTableMetaClient
extends Object
implements Serializable
```
HoodieTableMetaClient allows to access meta-data about a hoodie table It returns meta-data about commits, savepoints, compactions, cleanups as a HoodieTimeline Create an instance of the HoodieTableMetaClient with FileSystem and basePath to start getting the meta-data.
All the timelines are computed lazily, once computed the timeline is cached and never refreshed. Use the HoodieTimeline.reload() to refresh timelines.

Since:

0.3.0

See Also:

HoodieTimeline, Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`HoodieTableMetaClient.Builder` Builder for `HoodieTableMetaClient`.
`static class`	`HoodieTableMetaClient.PropertyBuilder` Builder for `Properties`.

Field Summary

Fields
Modifier and Type	Field and Description
`protected HoodieActiveTimeline`	`activeTimeline`
`static String`	`AUXILIARYFOLDER_NAME`
`protected StoragePath`	`basePath`
`static String`	`BOOTSTRAP_INDEX_BY_FILE_ID_FOLDER_PATH`
`static String`	`BOOTSTRAP_INDEX_BY_PARTITION_FOLDER_PATH`
`static String`	`BOOTSTRAP_INDEX_ROOT_FOLDER_PATH`
`static String`	`HASHING_METADATA_FOLDER_NAME`
`static String`	`HEARTBEAT_FOLDER_NAME`
`static String`	`MARKER_EXTN`
`static String`	`METADATA_TABLE_FOLDER_PATH`
`static String`	`METAFOLDER_NAME`
`protected StoragePath`	`metaPath`
`protected HoodieMetaserverConfig`	`metaserverConfig`
`static String`	`SAMPLE_WRITES_FOLDER_PATH`
`static String`	`SCHEMA_FOLDER_NAME`
`protected StorageConfiguration<?>`	`storageConf`
`protected HoodieTableConfig`	`tableConfig`
`static String`	`TEMPFOLDER_NAME`

Constructor Summary

Constructors
Modifier	Constructor and Description
	`HoodieTableMetaClient()` Deprecated.
`protected`	`HoodieTableMetaClient(HoodieStorage storage, String basePath, boolean loadActiveTimelineOnLoad, ConsistencyGuardConfig consistencyGuardConfig, Option<TimelineLayoutVersion> layoutVersion, String payloadClassName, String recordMergerStrategy, FileSystemRetryConfig fileSystemRetryConfig)` Instantiate HoodieTableMetaClient.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`static HoodieTableMetaClient.Builder`	`builder()`
`boolean`	`equals(Object o)`
`HoodieActiveTimeline`	`getActiveTimeline()` Get the active instants as a timeline.
`HoodieArchivedTimeline`	`getArchivedTimeline()` Get the archived commits as a timeline.
`HoodieArchivedTimeline`	`getArchivedTimeline(String startTs)` Returns the cached archived timeline from startTs (inclusive).
`HoodieArchivedTimeline`	`getArchivedTimeline(String startTs, boolean useCache)` Returns the cached archived timeline if using in-memory cache or a fresh new archived timeline if not using cache, from startTs (inclusive).
`String`	`getArchivePath()`
`String`	`getBasePath()` Deprecated. please use `getBasePathV2()`
`StoragePath`	`getBasePathV2()` Returns base path of the table
`String`	`getBootstrapIndexByFileIdFolderNameFolderPath()`
`String`	`getBootstrapIndexByPartitionFolderPath()`
`String`	`getCommitActionType()` Gets the commit action type.
`HoodieTimeline`	`getCommitsAndCompactionTimeline()` Get the commit + pending-compaction timeline visible for this table.
`HoodieTimeline`	`getCommitsTimeline()` Get the commit timeline visible for this table.
`HoodieTimeline`	`getCommitTimeline()` Get the compacted commit timeline visible for this table.
`ConsistencyGuardConfig`	`getConsistencyGuardConfig()`
`FileSystemRetryConfig`	`getFileSystemRetryConfig()`
`String`	`getHashingMetadataPath()`
`static String`	`getHeartbeatFolderPath(String basePath)`
`String`	`getMarkerFolderPath(String instantTs)` Returns Marker folder path.
`String`	`getMetaAuxiliaryPath()`
`StoragePath`	`getMetaPath()`
`HoodieMetaserverConfig`	`getMetaserverConfig()`
`HoodieStorage`	`getRawHoodieStorage()`
`String`	`getSchemaFolderName()`
`HoodieStorage`	`getStorage()`
`StorageConfiguration<?>`	`getStorageConf()`
`HoodieTableConfig`	`getTableConfig()`
`HoodieTableType`	`getTableType()`
`String`	`getTempFolderPath()`
`TimelineLayoutVersion`	`getTimelineLayoutVersion()`
`int`	`hashCode()`
`void`	`initializeBootstrapDirsIfNotExists()`
`static void`	`initializeBootstrapDirsIfNotExists(String basePath, HoodieStorage storage)`
`static HoodieTableMetaClient`	`initTableAndGetMetaClient(StorageConfiguration<?> storageConf, String basePath, Properties props)` Helper method to initialize a given path as a hoodie table with configs passed in as Properties.
`boolean`	`isTimelineNonEmpty()`
`static HoodieTableMetaClient`	`reload(HoodieTableMetaClient oldMetaClient)`
`HoodieActiveTimeline`	`reloadActiveTimeline()` Reload ActiveTimeline and cache.
`static List<StoragePathInfo>`	`scanFiles(HoodieStorage storage, StoragePath metaPath, StoragePathFilter nameFilter)` Helper method to scan all hoodie-instant metafiles.
`List<HoodieInstant>`	`scanHoodieInstantsFromFileSystem(Set<String> includedExtensions, boolean applyLayoutVersionFilters)` Helper method to scan all hoodie-instant metafiles and construct HoodieInstant objects.
`List<HoodieInstant>`	`scanHoodieInstantsFromFileSystem(StoragePath timelinePath, Set<String> includedExtensions, boolean applyLayoutVersionFilters)` Helper method to scan all hoodie-instant metafiles and construct HoodieInstant objects.
`void`	`setHoodieStorage(HoodieStorage storage)`
`String`	`toString()`
`void`	`validateTableProperties(Properties properties)` Validate table properties.
`static HoodieTableMetaClient.PropertyBuilder`	`withPropertyBuilder()`

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

METAFOLDER_NAME

public static final String METAFOLDER_NAME

See Also:: Constant Field Values

TEMPFOLDER_NAME

public static final String TEMPFOLDER_NAME

See Also:: Constant Field Values

AUXILIARYFOLDER_NAME

public static final String AUXILIARYFOLDER_NAME

See Also:: Constant Field Values

BOOTSTRAP_INDEX_ROOT_FOLDER_PATH

public static final String BOOTSTRAP_INDEX_ROOT_FOLDER_PATH

See Also:: Constant Field Values

SAMPLE_WRITES_FOLDER_PATH

public static final String SAMPLE_WRITES_FOLDER_PATH

See Also:: Constant Field Values

HEARTBEAT_FOLDER_NAME

public static final String HEARTBEAT_FOLDER_NAME

See Also:: Constant Field Values

METADATA_TABLE_FOLDER_PATH

public static final String METADATA_TABLE_FOLDER_PATH

See Also:: Constant Field Values

HASHING_METADATA_FOLDER_NAME

public static final String HASHING_METADATA_FOLDER_NAME

See Also:: Constant Field Values

BOOTSTRAP_INDEX_BY_PARTITION_FOLDER_PATH

public static final String BOOTSTRAP_INDEX_BY_PARTITION_FOLDER_PATH

See Also:: Constant Field Values

BOOTSTRAP_INDEX_BY_FILE_ID_FOLDER_PATH

public static final String BOOTSTRAP_INDEX_BY_FILE_ID_FOLDER_PATH

See Also:: Constant Field Values

SCHEMA_FOLDER_NAME

public static final String SCHEMA_FOLDER_NAME

See Also:: Constant Field Values

MARKER_EXTN
```
public static final String MARKER_EXTN
```
See Also:

Constant Field Values

basePath
```
protected StoragePath basePath
```

metaPath
```
protected StoragePath metaPath
```

storageConf

protected StorageConfiguration<?> storageConf

tableConfig

protected HoodieTableConfig tableConfig

activeTimeline

protected HoodieActiveTimeline activeTimeline

metaserverConfig

protected HoodieMetaserverConfig metaserverConfig

Constructor Detail

HoodieTableMetaClient

protected HoodieTableMetaClient(HoodieStorage storage,
                                String basePath,
                                boolean loadActiveTimelineOnLoad,
                                ConsistencyGuardConfig consistencyGuardConfig,
                                Option<TimelineLayoutVersion> layoutVersion,
                                String payloadClassName,
                                String recordMergerStrategy,
                                FileSystemRetryConfig fileSystemRetryConfig)

Instantiate HoodieTableMetaClient. Can only be called if table already exists

HoodieTableMetaClient
```
public HoodieTableMetaClient()
```
Deprecated.

For serializing and de-serializing.

Method Detail
- reload
```
public static HoodieTableMetaClient reload(HoodieTableMetaClient oldMetaClient)
```
- getBasePathV2
```
public StoragePath getBasePathV2()
```
  Returns base path of the table
- getBasePath
```
@Deprecated
public String getBasePath()
```
  Deprecated. please use getBasePathV2()
  
  Returns:
  
  Base path
- getTableType
```
public HoodieTableType getTableType()
```
  Returns:
  
  Hoodie Table Type
- getMetaPath
```
public StoragePath getMetaPath()
```
  Returns:
  
  Meta path
- getSchemaFolderName
```
public String getSchemaFolderName()
```
  Returns:
  
  schema folder path
- getHashingMetadataPath
```
public String getHashingMetadataPath()
```
  Returns:
  
  Hashing metadata base path
- getTempFolderPath
```
public String getTempFolderPath()
```
  Returns:
  
  Temp Folder path
- getMarkerFolderPath
```
public String getMarkerFolderPath(String instantTs)
```
  Returns Marker folder path.
  
  Parameters:
  
  instantTs - Instant Timestamp
  
  Returns:
- getMetaAuxiliaryPath
```
public String getMetaAuxiliaryPath()
```
  Returns:
  
  Auxiliary Meta path
- getHeartbeatFolderPath
```
public static String getHeartbeatFolderPath(String basePath)
```
  Returns:
  
  Heartbeat folder path.
- getBootstrapIndexByPartitionFolderPath
```
public String getBootstrapIndexByPartitionFolderPath()
```
  Returns:
  
  Bootstrap Index By Partition Folder
- getBootstrapIndexByFileIdFolderNameFolderPath
```
public String getBootstrapIndexByFileIdFolderNameFolderPath()
```
  Returns:
  
  Bootstrap Index By Hudi File Id Folder
- getArchivePath
```
public String getArchivePath()
```
  Returns:
  
  path where archived timeline is stored
- getTableConfig
```
public HoodieTableConfig getTableConfig()
```
  Returns:
  
  Table Config
- getTimelineLayoutVersion
```
public TimelineLayoutVersion getTimelineLayoutVersion()
```
- getStorage
```
public HoodieStorage getStorage()
```
- setHoodieStorage
```
public void setHoodieStorage(HoodieStorage storage)
```
- getRawHoodieStorage
```
public HoodieStorage getRawHoodieStorage()
```
- getStorageConf
```
public StorageConfiguration<?> getStorageConf()
```
- getActiveTimeline
```
public HoodieActiveTimeline getActiveTimeline()
```
  Get the active instants as a timeline.
  
  Returns:
  
  Active instants timeline
- reloadActiveTimeline
```
public HoodieActiveTimeline reloadActiveTimeline()
```
  Reload ActiveTimeline and cache.
  
  Returns:
  
  Active instants timeline
- getConsistencyGuardConfig
```
public ConsistencyGuardConfig getConsistencyGuardConfig()
```
- getFileSystemRetryConfig
```
public FileSystemRetryConfig getFileSystemRetryConfig()
```
- getArchivedTimeline
```
public HoodieArchivedTimeline getArchivedTimeline()
```
  Get the archived commits as a timeline. This is costly operation, as all data from the archived files are read. This should not be used, unless for historical debugging purposes.
  
  Returns:
  
  Archived commit timeline
- getMetaserverConfig
```
public HoodieMetaserverConfig getMetaserverConfig()
```
- getArchivedTimeline
```
public HoodieArchivedTimeline getArchivedTimeline(String startTs)
```
  Returns the cached archived timeline from startTs (inclusive).
  
  Parameters:
  
  startTs - The start instant time (inclusive) of the archived timeline.
  
  Returns:
  
  the archived timeline.
- getArchivedTimeline
```
public HoodieArchivedTimeline getArchivedTimeline(String startTs,
                                                  boolean useCache)
```
  Returns the cached archived timeline if using in-memory cache or a fresh new archived timeline if not using cache, from startTs (inclusive).
  Instantiating an archived timeline is costly operation if really early startTs is specified.
  This method is not thread safe.
  
  Parameters:
  
  startTs - The start instant time (inclusive) of the archived timeline.
  
  useCache - Whether to use in-memory cache.
  
  Returns:
  
  the archived timeline based on the arguments.
- validateTableProperties
```
public void validateTableProperties(Properties properties)
```
  Validate table properties.
  
  Parameters:
  
  properties - Properties from writeConfig.
- initTableAndGetMetaClient
```
public static HoodieTableMetaClient initTableAndGetMetaClient(StorageConfiguration<?> storageConf,
                                                              String basePath,
                                                              Properties props)
                                                       throws IOException
```
  Helper method to initialize a given path as a hoodie table with configs passed in as Properties.
  
  Returns:
  
  Instance of HoodieTableMetaClient
  
  Throws:
  
  IOException
- initializeBootstrapDirsIfNotExists
```
public static void initializeBootstrapDirsIfNotExists(String basePath,
                                                      HoodieStorage storage)
                                               throws IOException
```
  Throws:
  
  IOException
- scanFiles
```
public static List<StoragePathInfo> scanFiles(HoodieStorage storage,
                                              StoragePath metaPath,
                                              StoragePathFilter nameFilter)
                                       throws IOException
```
  Helper method to scan all hoodie-instant metafiles.
  
  Parameters:
  
  storage - The file system implementation for this table
  
  metaPath - The meta path where meta files are stored
  
  nameFilter - The name filter to filter meta files
  
  Returns:
  
  An array of meta FileStatus
  
  Throws:
  
  IOException - In case of failure
- isTimelineNonEmpty
```
public boolean isTimelineNonEmpty()
```
  Returns:
  
  true if any commits are found, else false.
- getCommitsTimeline
```
public HoodieTimeline getCommitsTimeline()
```
  Get the commit timeline visible for this table.
- getCommitsAndCompactionTimeline
```
public HoodieTimeline getCommitsAndCompactionTimeline()
```
  Get the commit + pending-compaction timeline visible for this table. A RT filesystem view is constructed with this timeline so that file-slice after pending compaction-requested instant-time is also considered valid. A RT file-system view for reading must then merge the file-slices before and after pending compaction instant so that all delta-commits are read.
- getCommitTimeline
```
public HoodieTimeline getCommitTimeline()
```
  Get the compacted commit timeline visible for this table.
- getCommitActionType
```
public String getCommitActionType()
```
  Gets the commit action type.
- scanHoodieInstantsFromFileSystem
```
public List<HoodieInstant> scanHoodieInstantsFromFileSystem(Set<String> includedExtensions,
                                                            boolean applyLayoutVersionFilters)
                                                     throws IOException
```
  Helper method to scan all hoodie-instant metafiles and construct HoodieInstant objects.
  
  Parameters:
  
  includedExtensions - Included hoodie extensions
  
  applyLayoutVersionFilters - Depending on Timeline layout version, if there are multiple states for the same action instant, only include the highest state
  
  Returns:
  
  List of Hoodie Instants generated
  
  Throws:
  
  IOException - in case of failure
- scanHoodieInstantsFromFileSystem
```
public List<HoodieInstant> scanHoodieInstantsFromFileSystem(StoragePath timelinePath,
                                                            Set<String> includedExtensions,
                                                            boolean applyLayoutVersionFilters)
                                                     throws IOException
```
  Helper method to scan all hoodie-instant metafiles and construct HoodieInstant objects.
  
  Parameters:
  
  timelinePath - MetaPath where instant files are stored
  
  includedExtensions - Included hoodie extensions
  
  applyLayoutVersionFilters - Depending on Timeline layout version, if there are multiple states for the same action instant, only include the highest state
  
  Returns:
  
  List of Hoodie Instants generated
  
  Throws:
  
  IOException - in case of failure
- equals
```
public boolean equals(Object o)
```
  Overrides:
  
  equals in class Object
- hashCode
```
public int hashCode()
```
  Overrides:
  
  hashCode in class Object
- toString
```
public String toString()
```
  Overrides:
  
  toString in class Object
- initializeBootstrapDirsIfNotExists
```
public void initializeBootstrapDirsIfNotExists()
                                        throws IOException
```
  Throws:
  
  IOException
- builder
```
public static HoodieTableMetaClient.Builder builder()
```
- withPropertyBuilder
```
public static HoodieTableMetaClient.PropertyBuilder withPropertyBuilder()
```

Class HoodieTableMetaClient

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

METAFOLDER_NAME

TEMPFOLDER_NAME

AUXILIARYFOLDER_NAME

BOOTSTRAP_INDEX_ROOT_FOLDER_PATH

SAMPLE_WRITES_FOLDER_PATH

HEARTBEAT_FOLDER_NAME

METADATA_TABLE_FOLDER_PATH

HASHING_METADATA_FOLDER_NAME

BOOTSTRAP_INDEX_BY_PARTITION_FOLDER_PATH

BOOTSTRAP_INDEX_BY_FILE_ID_FOLDER_PATH

SCHEMA_FOLDER_NAME

MARKER_EXTN

basePath

metaPath

storageConf

tableConfig

activeTimeline

metaserverConfig

Constructor Detail

HoodieTableMetaClient

HoodieTableMetaClient

Method Detail

reload

getBasePathV2

getBasePath

getTableType

getMetaPath

getSchemaFolderName

getHashingMetadataPath

getTempFolderPath

getMarkerFolderPath

getMetaAuxiliaryPath

getHeartbeatFolderPath

getBootstrapIndexByPartitionFolderPath

getBootstrapIndexByFileIdFolderNameFolderPath

getArchivePath

getTableConfig

getTimelineLayoutVersion

getStorage

setHoodieStorage

getRawHoodieStorage

getStorageConf

getActiveTimeline

reloadActiveTimeline

getConsistencyGuardConfig

getFileSystemRetryConfig

getArchivedTimeline

getMetaserverConfig

getArchivedTimeline

getArchivedTimeline

validateTableProperties

initTableAndGetMetaClient

initializeBootstrapDirsIfNotExists

scanFiles

isTimelineNonEmpty

getCommitsTimeline

getCommitsAndCompactionTimeline

getCommitTimeline

getCommitActionType

scanHoodieInstantsFromFileSystem

scanHoodieInstantsFromFileSystem

equals

hashCode

toString

initializeBootstrapDirsIfNotExists

builder

withPropertyBuilder