com.google.cloud.bigtable.mapreduce

Class Import

java.lang.Object

org.apache.hadoop.conf.Configured

com.google.cloud.bigtable.mapreduce.Import

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class Import
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

Import data written by Export.

Version:

$Id: $Id

Author:

sduskis

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Import.Importer Write table content out to files.
static class	Import.KeyValueImporter A mapper that just writes out KeyValues.

Field Summary

Fields

Modifier and Type	Field and Description
static String	BULK_OUTPUT_CONF_KEY Constant BULK_OUTPUT_CONF_KEY="import.bulk.output"
static String	CF_RENAME_PROP Constant CF_RENAME_PROP="HBASE_IMPORTER_RENAME_CFS"
static String	FILTER_ARGS_CONF_KEY Constant FILTER_ARGS_CONF_KEY="import.filter.args"
static String	FILTER_CLASS_CONF_KEY Constant FILTER_CLASS_CONF_KEY="import.filter.class"
static String	TABLE_NAME Constant TABLE_NAME="import.table.name"

Constructor Summary

Constructors

Constructor and Description
Import()

Method Summary

Modifier and Type	Method and Description
static void	addFilterAndArguments(org.apache.hadoop.conf.Configuration conf, Class<? extends Filter> clazz, List<String> filterArgs) Add a Filter to be instantiated on import
static void	configureCfRenaming(org.apache.hadoop.conf.Configuration conf, Map<String,String> renameMap) Sets a configuration property with key CF_RENAME_PROP in conf that tells the mapper how to rename column families.
static org.apache.hadoop.mapreduce.Job	createSubmittableJob(org.apache.hadoop.conf.Configuration conf, String[] args) Sets up the actual job.
static Cell	filterKv(Filter filter, Cell kv) Attempt to filter out the keyvalue
static Filter	instantiateFilter(org.apache.hadoop.conf.Configuration conf) Create a Filter to apply to all incoming keys (KeyValues) to optionally not include in the job output
static void	main(String[] args) Main entry point.
int	run(String[] args)

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Field Detail

CF_RENAME_PROP

public static final String CF_RENAME_PROP

Constant CF_RENAME_PROP="HBASE_IMPORTER_RENAME_CFS"

See Also:

Constant Field Values

BULK_OUTPUT_CONF_KEY

public static final String BULK_OUTPUT_CONF_KEY

Constant BULK_OUTPUT_CONF_KEY="import.bulk.output"

See Also:

Constant Field Values

FILTER_CLASS_CONF_KEY

public static final String FILTER_CLASS_CONF_KEY

Constant FILTER_CLASS_CONF_KEY="import.filter.class"

See Also:

Constant Field Values

FILTER_ARGS_CONF_KEY

public static final String FILTER_ARGS_CONF_KEY

Constant FILTER_ARGS_CONF_KEY="import.filter.args"

See Also:

Constant Field Values

TABLE_NAME

public static final String TABLE_NAME

Constant TABLE_NAME="import.table.name"

See Also:

Constant Field Values

Constructor Detail

Import

public Import()

Method Detail

instantiateFilter

public static Filter instantiateFilter(org.apache.hadoop.conf.Configuration conf)

Create a Filter to apply to all incoming keys (KeyValues) to optionally not include in the job output

Parameters:

conf - Configuration from which to load the filter

Returns:

the filter to use for the task, or null if no filter to should be used

Throws:

IllegalArgumentException - if the filter is misconfigured

filterKv

public static Cell filterKv(Filter filter,
                            Cell kv)
                     throws IOException

Attempt to filter out the keyvalue

Parameters:

kv - KeyValue on which to apply the filter

filter - a Filter object.

Returns:

null if the key should not be written, otherwise returns the original KeyValue

Throws:

IOException - if any.

configureCfRenaming

public static void configureCfRenaming(org.apache.hadoop.conf.Configuration conf,
                                       Map<String,String> renameMap)

Sets a configuration property with key CF_RENAME_PROP in conf that tells the mapper how to rename column families.

Alternately, instead of calling this function, you could set the configuration key CF_RENAME_PROP yourself. The value should look like

srcCf1:destCf1,srcCf2:destCf2,....

. This would have the same effect on the mapper behavior.

Parameters:

conf - the Configuration in which the CF_RENAME_PROP key will be set

renameMap - a mapping from source CF names to destination CF names

addFilterAndArguments

public static void addFilterAndArguments(org.apache.hadoop.conf.Configuration conf,
                                         Class<? extends Filter> clazz,
                                         List<String> filterArgs)

Add a Filter to be instantiated on import

Parameters:

conf - Configuration to update (will be passed to the job)

clazz - Filter subclass to instantiate on the server.

filterArgs - List of arguments to pass to the filter on instantiation

createSubmittableJob

public static org.apache.hadoop.mapreduce.Job createSubmittableJob(org.apache.hadoop.conf.Configuration conf,
                                                                   String[] args)
                                                            throws IOException

Sets up the actual job.

Parameters:

conf - The current configuration.

args - The command line parameters.

Returns:

The newly created job.

Throws:

IOException - When setting up the job fails.

run

public int run(String[] args)
        throws Exception

Specified by:

run in interface org.apache.hadoop.util.Tool

Throws:

Exception

main

public static void main(String[] args)
                 throws Exception

Main entry point.

Parameters:

args - The command line parameters.

Throws:

Exception - When running the job fails.