Package org.apache.druid.frame.util

Class DurableStorageUtils

java.lang.Object

org.apache.druid.frame.util.DurableStorageUtils

public class DurableStorageUtils
extends Object

Helper class that fetches the directory and file names corresponding to file location

Field Summary

Fields

Modifier and Type	Field	Description
static final String	QUERY_RESULTS_DIR
static final com.google.common.base.Splitter	SPLITTER
static final String	SUCCESS_MARKER_FILENAME

Constructor Summary

Constructors

Constructor	Description
DurableStorageUtils()

Method Summary

Modifier and Type	Method	Description
static String	getControllerDirectory(String controllerTaskId)
static String	getNextDirNameWithPrefixFromPath(String path)	Tries to parse out the most top level directory from the path.
static String	getOutputsFileNameForPath(String controllerTaskId, int stageNumber, int workerNumber, String taskId, String path)	Fetches the file location where a particular worker writes the data corresponding to a particular stage and a custom path name
static String	getPartitionOutputsFileNameWithPathForPartition(String controllerTaskId, int stageNumber, int workerNumber, String taskId, int partitionNumber)	Fetches the file location where a particular worker writes the data corresponding to a particular stage and partition
static String	getQueryResultsFileNameWithPathForPartition(String controllerTaskId, int stageNumber, int workerNumber, String taskId, int partitionNumber)
static String	getQueryResultsForTaskIdFolderName(String controllerTaskId, int stageNumber, int workerNumber, String taskId)
static String	getQueryResultsSuccessFilePath(String controllerTaskId, int stageNumber, int workerNumber)
static String	getTaskIdOutputsFolderName(String controllerTaskId, int stageNumber, int workerNumber, String taskId)	Fetches the directory location where a particular worker will store the partition files corresponding to the stage number, and it's task id
static String	getWorkerOutputSuccessFilePath(String controllerTaskId, int stageNumber, int workerNumber)
static boolean	isQueryResultFileActive(String path, Set<String> recentlyCompletedTaskIds)	Checks if a query result file should be retained by checking if its task ID is recently completed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SUCCESS_MARKER_FILENAME

public static final String SUCCESS_MARKER_FILENAME

See Also:

• Constant Field Values

SPLITTER

public static final com.google.common.base.Splitter SPLITTER

QUERY_RESULTS_DIR

public static final String QUERY_RESULTS_DIR

See Also:

• Constant Field Values

Constructor Details

DurableStorageUtils

public DurableStorageUtils()

Method Details

getControllerDirectory

public static String getControllerDirectory(String controllerTaskId)

getWorkerOutputSuccessFilePath

public static String getWorkerOutputSuccessFilePath(String controllerTaskId,
 int stageNumber,
 int workerNumber)

getQueryResultsSuccessFilePath

public static String getQueryResultsSuccessFilePath(String controllerTaskId,
 int stageNumber,
 int workerNumber)

getTaskIdOutputsFolderName

public static String getTaskIdOutputsFolderName(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId)

Fetches the directory location where a particular worker will store the partition files corresponding to the stage number, and it's task id

getQueryResultsForTaskIdFolderName

public static String getQueryResultsForTaskIdFolderName(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId)

getPartitionOutputsFileNameWithPathForPartition

public static String getPartitionOutputsFileNameWithPathForPartition(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 int partitionNumber)

Fetches the file location where a particular worker writes the data corresponding to a particular stage and partition

getQueryResultsFileNameWithPathForPartition

public static String getQueryResultsFileNameWithPathForPartition(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 int partitionNumber)

getOutputsFileNameForPath

public static String getOutputsFileNameForPath(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 String path)

Fetches the file location where a particular worker writes the data corresponding to a particular stage and a custom path name

getNextDirNameWithPrefixFromPath

@Nullable
public static String getNextDirNameWithPrefixFromPath(String path)

Tries to parse out the most top level directory from the path. Returns null if there is no such directory.

For eg:

• for input path controller_query_id/task/123 the function will return controller_query_id
• for input path abcd, the function will return abcd
• for input path null, the function will return null

isQueryResultFileActive

public static boolean isQueryResultFileActive(String path,
 Set<String> recentlyCompletedTaskIds)

Checks if a query result file should be retained by checking if its task ID is recently completed. Parses the controller task ID from paths under QUERY_RESULTS_DIR and checks membership in the provided set.

Parameters:

path - the file path to check

recentlyCompletedTaskIds - set of task IDs that completed recently and whose files should be retained

Returns:

true if the file belongs to a recently completed task; false otherwise