org.apache.flink.connector.hbase1.source

类 AbstractTableInputFormat<T>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>

org.apache.flink.connector.hbase1.source.AbstractTableInputFormat<T>

所有已实现的接口:

Serializable, org.apache.flink.api.common.io.InputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.connector.hbase.source.TableInputSplit>

直接已知子类:

HBaseRowDataInputFormat

@Internal
public abstract class AbstractTableInputFormat<T>
extends org.apache.flink.api.common.io.RichInputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>

Abstract InputFormat to read data from HBase tables.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected org.apache.hadoop.hbase.client.Connection	connection
protected byte[]	currentRow
protected boolean	endReached
protected static org.slf4j.Logger	LOG
protected org.apache.hadoop.hbase.client.ResultScanner	resultScanner HBase iterator wrapper.
protected org.apache.hadoop.hbase.client.Scan	scan
protected long	scannedRows
protected byte[]	serializedConfig
protected org.apache.hadoop.hbase.client.HTable	table

构造器概要

构造器

构造器和说明
AbstractTableInputFormat(org.apache.hadoop.conf.Configuration hConf)

方法概要

限定符和类型	方法和说明
void	close()
void	closeTable()
void	configure(org.apache.flink.configuration.Configuration parameters)
org.apache.flink.connector.hbase.source.TableInputSplit[]	createInputSplits(int minNumSplits)
org.apache.hadoop.hbase.client.Connection	getConnection()
protected org.apache.hadoop.conf.Configuration	getHadoopConfiguration()
org.apache.flink.core.io.InputSplitAssigner	getInputSplitAssigner(org.apache.flink.connector.hbase.source.TableInputSplit[] inputSplits)
protected abstract org.apache.hadoop.hbase.client.Scan	getScanner() Returns an instance of Scan that retrieves the required subset of records from the HBase table.
org.apache.flink.api.common.io.statistics.BaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
protected abstract String	getTableName() What table is to be read.
protected boolean	includeRegionInScan(byte[] startKey, byte[] endKey) Test if the given region is to be included in the scan while splitting the regions of a table.
protected abstract void	initTable() Creates a Scan object and opens the HTable connection to initialize the HBase table.
protected abstract T	mapResultToOutType(org.apache.hadoop.hbase.client.Result r) HBase returns an instance of Result.
T	nextRecord(T reuse)
void	open(org.apache.flink.connector.hbase.source.TableInputSplit split) Creates a Scan object and opens the HTable connection.
boolean	reachedEnd()

从类继承的方法 org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

protected static final org.slf4j.Logger LOG

endReached

protected boolean endReached

connection

protected transient org.apache.hadoop.hbase.client.Connection connection

table

protected transient org.apache.hadoop.hbase.client.HTable table

scan

protected transient org.apache.hadoop.hbase.client.Scan scan

resultScanner

protected org.apache.hadoop.hbase.client.ResultScanner resultScanner

HBase iterator wrapper.

currentRow

protected byte[] currentRow

scannedRows

protected long scannedRows

serializedConfig

protected byte[] serializedConfig

构造器详细资料

AbstractTableInputFormat

public AbstractTableInputFormat(org.apache.hadoop.conf.Configuration hConf)

方法详细资料

initTable

protected abstract void initTable()
                           throws IOException

Creates a Scan object and opens the HTable connection to initialize the HBase table.

抛出:

IOException - Thrown, if the connection could not be opened due to an I/O problem.

getScanner

protected abstract org.apache.hadoop.hbase.client.Scan getScanner()

Returns an instance of Scan that retrieves the required subset of records from the HBase table.

返回:

The appropriate instance of Scan for this use case.

getTableName

protected abstract String getTableName()

What table is to be read.

Per instance of a TableInputFormat derivative only a single table name is possible.

返回:

The name of the table

mapResultToOutType

protected abstract T mapResultToOutType(org.apache.hadoop.hbase.client.Result r)

HBase returns an instance of Result.

This method maps the returned Result instance into the output type T.

参数:

r - The Result instance from HBase that needs to be converted

返回:

The appropriate instance of T that contains the data of Result.

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

getHadoopConfiguration

protected org.apache.hadoop.conf.Configuration getHadoopConfiguration()

open

public void open(org.apache.flink.connector.hbase.source.TableInputSplit split)
          throws IOException

Creates a Scan object and opens the HTable connection. The connection is opened in this method and closed in close().

参数:

split - The split to be opened.

抛出:

IOException - Thrown, if the spit could not be opened due to an I/O problem.

nextRecord

public T nextRecord(T reuse)
             throws IOException

抛出:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

抛出:

IOException

close

public void close()
           throws IOException

抛出:

IOException

closeTable

public void closeTable()

createInputSplits

public org.apache.flink.connector.hbase.source.TableInputSplit[] createInputSplits(int minNumSplits)
                                                                            throws IOException

抛出:

IOException

includeRegionInScan

protected boolean includeRegionInScan(byte[] startKey,
                                      byte[] endKey)

Test if the given region is to be included in the scan while splitting the regions of a table.

参数:

startKey - Start key of the region

endKey - End key of the region

返回:

true, if this region needs to be included as part of the input (default).

getInputSplitAssigner

public org.apache.flink.core.io.InputSplitAssigner getInputSplitAssigner(org.apache.flink.connector.hbase.source.TableInputSplit[] inputSplits)

getStatistics

public org.apache.flink.api.common.io.statistics.BaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)

getConnection

@VisibleForTesting
public org.apache.hadoop.hbase.client.Connection getConnection()