public class CompactionRecordCountProvider extends RecordCountProvider
RecordCountProvider, which provides record count from file path.
The file name should follow the pattern: {Prefix}{RecordCount}.{SystemCurrentTimeInMills}.{RandomInteger}{SUFFIX}.
The prefix should be either M_OUTPUT_FILE_PREFIX or MR_OUTPUT_FILE_PREFIX.
For example, given a file path: "/a/b/c/part-m-123.1444437036.12345.avro", the record count will be 123.| Modifier and Type | Field and Description |
|---|---|
static String |
M_OUTPUT_FILE_PREFIX |
static String |
MR_OUTPUT_FILE_PREFIX |
| Constructor and Description |
|---|
CompactionRecordCountProvider() |
| Modifier and Type | Method and Description |
|---|---|
static String |
constructFileName(String filenamePrefix,
long recordCount)
Deprecated.
discouraged since default behavior is not obvious from API itself.
|
static String |
constructFileName(String filenamePrefix,
String extension,
long recordCount)
Construct the file name as {filenamePrefix}{recordCount}.{SystemCurrentTimeInMills}.{RandomInteger}{extension}.
|
org.apache.hadoop.fs.Path |
convertPath(org.apache.hadoop.fs.Path path,
String extension,
RecordCountProvider src)
This method currently supports converting the given
Path from IngestionRecordCountProvider. |
long |
getRecordCount(org.apache.hadoop.fs.Path filepath)
Get the record count through filename.
|
getNotImplementedException, getRecordCountpublic static final String MR_OUTPUT_FILE_PREFIX
public static final String M_OUTPUT_FILE_PREFIX
@Deprecated public static String constructFileName(String filenamePrefix, long recordCount)
public static String constructFileName(String filenamePrefix, String extension, long recordCount)
public long getRecordCount(org.apache.hadoop.fs.Path filepath)
getRecordCount in class RecordCountProviderpublic org.apache.hadoop.fs.Path convertPath(org.apache.hadoop.fs.Path path,
String extension,
RecordCountProvider src)
Path from IngestionRecordCountProvider.
The converted Path will start with M_OUTPUT_FILE_PREFIX.convertPath in class RecordCountProvider