org.apache.hudi.common.conflict.detection

Class DirectMarkerBasedDetectionStrategy

java.lang.Object

org.apache.hudi.common.conflict.detection.DirectMarkerBasedDetectionStrategy

All Implemented Interfaces:

EarlyConflictDetectionStrategy

@PublicAPIClass(maturity=EVOLVING)
public abstract class DirectMarkerBasedDetectionStrategy
extends Object
implements EarlyConflictDetectionStrategy

This abstract strategy is used for direct marker writers, trying to do early conflict detection.

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieActiveTimeline	activeTimeline
protected HoodieConfig	config
protected String	fileId
protected String	instantTime
protected String	partitionPath
protected HoodieStorage	storage

Constructor Summary

Constructors

Constructor and Description
DirectMarkerBasedDetectionStrategy(HoodieStorage storage, String partitionPath, String fileId, String instantTime, HoodieActiveTimeline activeTimeline, HoodieConfig config)

Method Summary

Modifier and Type	Method and Description
boolean	checkMarkerConflict(String basePath, long maxAllowableHeartbeatIntervalInMs) We need to do list operation here.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.conflict.detection.EarlyConflictDetectionStrategy

detectAndResolveConflictIfNecessary, hasMarkerConflict, resolveMarkerConflict

Field Detail

storage

protected final HoodieStorage storage

partitionPath

protected final String partitionPath

fileId

protected final String fileId

instantTime

protected final String instantTime

activeTimeline

protected final HoodieActiveTimeline activeTimeline

config

protected final HoodieConfig config

Constructor Detail

DirectMarkerBasedDetectionStrategy

public DirectMarkerBasedDetectionStrategy(HoodieStorage storage,
                                          String partitionPath,
                                          String fileId,
                                          String instantTime,
                                          HoodieActiveTimeline activeTimeline,
                                          HoodieConfig config)

Method Detail

checkMarkerConflict

public boolean checkMarkerConflict(String basePath,
                                   long maxAllowableHeartbeatIntervalInMs)
                            throws IOException

We need to do list operation here. In order to reduce the list pressure as much as possible, first we build path prefix in advance: '$base_path/.temp/instant_time/partition_path', and only list these specific partition_paths we need instead of list all the '$base_path/.temp/'

Parameters:

basePath - Base path of the table.

maxAllowableHeartbeatIntervalInMs - Heartbeat timeout.

Returns:

true if current fileID is already existed under .temp/instant_time/partition_path/..

Throws:

IOException - upon errors.