org.apache.gobblin.util.recordcount

Class IngestionRecordCountProvider

java.lang.Object

org.apache.gobblin.util.RecordCountProvider

org.apache.gobblin.util.recordcount.IngestionRecordCountProvider

public class IngestionRecordCountProvider
extends RecordCountProvider

Implementation of RecordCountProvider, which provides record count from file path. The file path should follow the pattern: {Filename}.{RecordCount}.{Extension}. For example, given a file path: "/a/b/c/file.123.avro", the record count will be 123.

Constructor Summary

Constructors

Constructor and Description
IngestionRecordCountProvider()

Method Summary

Modifier and Type	Method and Description
static String	constructFilePath(String oldFilePath, long recordCounts) Construct a new file path by appending record count to the filename of the given file path, separated by SEPARATOR.
long	getRecordCount(org.apache.hadoop.fs.Path filepath) The record count should be the last component before the filename extension.

Methods inherited from class org.apache.gobblin.util.RecordCountProvider

convertPath, getNotImplementedException, getRecordCount

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IngestionRecordCountProvider

public IngestionRecordCountProvider()

Method Detail

constructFilePath

public static String constructFilePath(String oldFilePath,
                                       long recordCounts)

Construct a new file path by appending record count to the filename of the given file path, separated by SEPARATOR. For example, given path: "/a/b/c/file.avro" and record count: 123, the new path returned will be: "/a/b/c/file.123.avro"

getRecordCount

public long getRecordCount(org.apache.hadoop.fs.Path filepath)

The record count should be the last component before the filename extension.

Specified by:

getRecordCount in class RecordCountProvider