Package org.apache.druid.frame.util

Class DurableStorageUtils

java.lang.Object

org.apache.druid.frame.util.DurableStorageUtils

public class DurableStorageUtils
extends Object

Helper class that fetches the directory and file names corresponding to file location

Field Summary

Fields

Modifier and Type	Field	Description
static final String	QUERY_RESULTS_DIR
static final com.google.common.base.Splitter	SPLITTER
static final String	SUCCESS_MARKER_FILENAME

Constructor Summary

Constructors

Constructor	Description
DurableStorageUtils()

Method Summary

Modifier and Type	Method	Description
static String	getControllerDirectory(String controllerTaskId)
static String	getNextDirNameWithPrefixFromPath(String path)	Tries to parse out the most top level directory from the path.
static String	getOutputsFileNameForPath(String controllerTaskId, int stageNumber, int workerNumber, String taskId, String path)	Fetches the file location where a particular worker writes the data corresponding to a particular stage and a custom path name
static String	getPartitionOutputsFileNameWithPathForPartition(String controllerTaskId, int stageNumber, int workerNumber, String taskId, int partitionNumber)	Fetches the file location where a particular worker writes the data corresponding to a particular stage and partition
static String	getQueryResultsFileNameWithPathForPartition(String controllerTaskId, int stageNumber, int workerNumber, String taskId, int partitionNumber)
static String	getQueryResultsForTaskIdFolderName(String controllerTaskId, int stageNumber, int workerNumber, String taskId)
static String	getQueryResultsSuccessFilePath(String controllerTaskId, int stageNumber, int workerNumber)
static String	getTaskIdOutputsFolderName(String controllerTaskId, int stageNumber, int workerNumber, String taskId)	Fetches the directory location where a particular worker will store the partition files corresponding to the stage number, and it's task id
static String	getWorkerOutputSuccessFilePath(String controllerTaskId, int stageNumber, int workerNumber)
static boolean	isQueryResultFileActive(String path, Set<String> knownTasks)	Tries to parse out the controller taskID from the query results path, and checks if the taskID is present in the set of known tasks.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

SUCCESS_MARKER_FILENAME

public static final String SUCCESS_MARKER_FILENAME

See Also:

• Constant Field Values

SPLITTER

public static final com.google.common.base.Splitter SPLITTER

QUERY_RESULTS_DIR

public static final String QUERY_RESULTS_DIR

See Also:

• Constant Field Values

Constructor Details

DurableStorageUtils

public DurableStorageUtils()

Method Details

getControllerDirectory

public static String getControllerDirectory(String controllerTaskId)

getWorkerOutputSuccessFilePath

public static String getWorkerOutputSuccessFilePath(String controllerTaskId,
 int stageNumber,
 int workerNumber)

getQueryResultsSuccessFilePath

public static String getQueryResultsSuccessFilePath(String controllerTaskId,
 int stageNumber,
 int workerNumber)

getTaskIdOutputsFolderName

public static String getTaskIdOutputsFolderName(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId)

Fetches the directory location where a particular worker will store the partition files corresponding to the stage number, and it's task id

getQueryResultsForTaskIdFolderName

public static String getQueryResultsForTaskIdFolderName(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId)

getPartitionOutputsFileNameWithPathForPartition

public static String getPartitionOutputsFileNameWithPathForPartition(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 int partitionNumber)

Fetches the file location where a particular worker writes the data corresponding to a particular stage and partition

getQueryResultsFileNameWithPathForPartition

public static String getQueryResultsFileNameWithPathForPartition(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 int partitionNumber)

getOutputsFileNameForPath

public static String getOutputsFileNameForPath(String controllerTaskId,
 int stageNumber,
 int workerNumber,
 String taskId,
 String path)

Fetches the file location where a particular worker writes the data corresponding to a particular stage and a custom path name

getNextDirNameWithPrefixFromPath

@Nullable
public static String getNextDirNameWithPrefixFromPath(String path)

Tries to parse out the most top level directory from the path. Returns null if there is no such directory.

For eg:

• for input path controller_query_id/task/123 the function will return controller_query_id
• for input path abcd, the function will return abcd
• for input path null, the function will return null

isQueryResultFileActive

public static boolean isQueryResultFileActive(String path,
 Set<String> knownTasks)

Tries to parse out the controller taskID from the query results path, and checks if the taskID is present in the set of known tasks. Returns true if the set contains the taskId. Returns false if taskId could not be parsed or if the set does not contain the taskId.

For eg:

• for path controller_query_id/task/123 the function will return false
• for path query-result/controller_query_id/results.json, the function will return true
if the controller_query_id is in known tasks
• for path query-result/controller_query_id/results.json, the function will return false
if the controller_query_id is not in known tasks
• for path null, the function will return false