com.google.cloud.bigtable.mapreduce.validation

Class BigtableSyncTableJob

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.hbase.mapreduce.SyncTable

com.google.cloud.bigtable.mapreduce.validation.BigtableSyncTableJob

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class BigtableSyncTableJob
extends SyncTable

Bigtable SyncTable Job

Base source authored by SyncTable.

SyncTable is part 2 of 2 stages used for byte level validation between a source and target table only. BigtableSyncTableJob extends functionality to provide connectivity to Bigtable as a source or target and relies on SyncTable for core functionality. BigtableSyncTableJob overrides dryrun configuration as true by default to perform a read-only validation report only.

SyncTable is a Mapreduce job that generates hashes on the target table and compares these hashes with the output from . For diverging hashes, cell-level comparison is performed.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hbase.mapreduce.SyncTable

SyncTable.SyncMapper

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.mapreduce.Counters	counters
static String	SOURCE_BT_APP_PROFILE_CONF_KEY
static String	SOURCE_BT_INSTANCE_CONF_KEY
static String	SOURCE_BT_PROJECTID_CONF_KEY
static String	TARGET_BT_APP_PROFILE_CONF_KEY
static String	TARGET_BT_INSTANCE_CONF_KEY
static String	TARGET_BT_PROJECTID_CONF_KEY

Constructor Summary

Constructors

Constructor and Description
BigtableSyncTableJob(org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
protected boolean	doCommandLine(SyncTable syncTable, String[] args) SyncTable command line parsing utility
String	getSourceBigtableAppProfile()
String	getSourceBigtableInstance()
String	getSourceBigtableProjectId()
String	getTargetBigtableAppProfile()
String	getTargetBigtableInstance()
String	getTargetBigtableProjectId()
static void	main(String[] args) Main entry point.
static void	printUsage(String errorMsg, String[] args)
int	run(String[] args)
boolean	verifyRequiredArgsSet(SyncTable syncTable, String[] args, String sourceBigtableProjectId, String sourceBigtableInstance, String targetBigtableProjectId, String targetBigtableInstance) verify required options and args are correctly set

Methods inherited from class org.apache.hadoop.hbase.mapreduce.SyncTable

createSubmittableJob

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

SOURCE_BT_PROJECTID_CONF_KEY

public static final String SOURCE_BT_PROJECTID_CONF_KEY

See Also:

Constant Field Values

SOURCE_BT_INSTANCE_CONF_KEY

public static final String SOURCE_BT_INSTANCE_CONF_KEY

See Also:

Constant Field Values

SOURCE_BT_APP_PROFILE_CONF_KEY

public static final String SOURCE_BT_APP_PROFILE_CONF_KEY

See Also:

Constant Field Values

TARGET_BT_PROJECTID_CONF_KEY

public static final String TARGET_BT_PROJECTID_CONF_KEY

See Also:

Constant Field Values

TARGET_BT_INSTANCE_CONF_KEY

public static final String TARGET_BT_INSTANCE_CONF_KEY

See Also:

Constant Field Values

TARGET_BT_APP_PROFILE_CONF_KEY

public static final String TARGET_BT_APP_PROFILE_CONF_KEY

See Also:

Constant Field Values

counters

protected org.apache.hadoop.mapreduce.Counters counters

Constructor Detail

BigtableSyncTableJob

public BigtableSyncTableJob(org.apache.hadoop.conf.Configuration conf)

Method Detail

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Overrides:

run in class SyncTable

Throws:

Exception

doCommandLine

protected boolean doCommandLine(SyncTable syncTable,
                                String[] args)

SyncTable command line parsing utility

Parameters:

args -

Returns:

verifyRequiredArgsSet

public boolean verifyRequiredArgsSet(SyncTable syncTable,
                                     String[] args,
                                     String sourceBigtableProjectId,
                                     String sourceBigtableInstance,
                                     String targetBigtableProjectId,
                                     String targetBigtableInstance)

verify required options and args are correctly set

Parameters:

args -

sourceBigtableProjectId -

sourceBigtableInstance -

targetBigtableProjectId -

targetBigtableInstance -

Returns:

boolean state if required args are set

printUsage

public static void printUsage(String errorMsg,
                              String[] args)

getSourceBigtableProjectId

public String getSourceBigtableProjectId()

getSourceBigtableInstance

public String getSourceBigtableInstance()

getSourceBigtableAppProfile

public String getSourceBigtableAppProfile()

getTargetBigtableProjectId

public String getTargetBigtableProjectId()

getTargetBigtableInstance

public String getTargetBigtableInstance()

getTargetBigtableAppProfile

public String getTargetBigtableAppProfile()

main

public static void main(String[] args)
                 throws Exception

Main entry point.

Throws:

Exception