org.apache.flink.connector.jdbc

类 JdbcInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

org.apache.flink.connector.jdbc.JdbcInputFormat

所有已实现的接口:

Serializable, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>, org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.io.InputSplit>

@Experimental
public class JdbcInputFormat
extends org.apache.flink.api.common.io.RichInputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>
implements org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

InputFormat to read data from a database and generate Rows. The InputFormat has to be configured using the supplied InputFormatBuilder. A valid RowTypeInfo must be properly configured in the builder, e.g.:

 TypeInformation[] fieldTypes = new TypeInformation[] {
        BasicTypeInfo.INT_TYPE_INFO,
        BasicTypeInfo.STRING_TYPE_INFO,
        BasicTypeInfo.STRING_TYPE_INFO,
        BasicTypeInfo.DOUBLE_TYPE_INFO,
        BasicTypeInfo.INT_TYPE_INFO
 };

 RowTypeInfo rowTypeInfo = new RowTypeInfo(fieldTypes);

 JdbcInputFormat jdbcInputFormat = JdbcInputFormat.buildJdbcInputFormat()
                        .setDrivername("org.apache.derby.jdbc.EmbeddedDriver")
                        .setDBUrl("jdbc:derby:memory:ebookshop")
                        .setQuery("select * from books")
                        .setRowTypeInfo(rowTypeInfo)
                        .finish();
 

In order to query the JDBC source in parallel, you need to provide a parameterized query template (i.e. a valid PreparedStatement) and a JdbcParameterValuesProvider which provides binding values for the query parameters. E.g.:

 Serializable[][] queryParameters = new String[2][1];
 queryParameters[0] = new String[]{"Kumar"};
 queryParameters[1] = new String[]{"Tan Ah Teck"};

 JdbcInputFormat jdbcInputFormat = JdbcInputFormat.buildJdbcInputFormat()
                        .setDrivername("org.apache.derby.jdbc.EmbeddedDriver")
                        .setDBUrl("jdbc:derby:memory:ebookshop")
                        .setQuery("select * from books WHERE author = ?")
                        .setRowTypeInfo(rowTypeInfo)
                        .setParametersProvider(new JdbcGenericParameterValuesProvider(queryParameters))
                        .finish();
 

另请参阅:

Row, JdbcParameterValuesProvider, PreparedStatement, DriverManager, 序列化表格

嵌套类概要

嵌套类

限定符和类型	类和说明
static class	JdbcInputFormat.JdbcInputFormatBuilder Builder for JdbcInputFormat.

字段概要

字段

限定符和类型	字段和说明
protected Boolean	autoCommit
protected JdbcConnectionProvider	connectionProvider
protected int	fetchSize
protected boolean	hasNext
protected static org.slf4j.Logger	LOG
protected Object[][]	parameterValues
protected String	queryTemplate
protected ResultSet	resultSet
protected int	resultSetConcurrency
protected int	resultSetType
protected org.apache.flink.api.java.typeutils.RowTypeInfo	rowTypeInfo
protected static long	serialVersionUID
protected PreparedStatement	statement

构造器概要

构造器

构造器和说明
JdbcInputFormat()

方法概要

限定符和类型	方法和说明
static JdbcInputFormat.JdbcInputFormatBuilder	buildJdbcInputFormat() A builder used to set parameters to the output format's configuration in a fluent way.
void	close() Closes all resources used.
void	closeInputFormat()
void	configure(org.apache.flink.configuration.Configuration parameters)
org.apache.flink.core.io.InputSplit[]	createInputSplits(int minNumSplits)
protected Connection	getDbConn()
org.apache.flink.core.io.InputSplitAssigner	getInputSplitAssigner(org.apache.flink.core.io.InputSplit[] inputSplits)
org.apache.flink.api.java.typeutils.RowTypeInfo	getProducedType()
protected PreparedStatement	getStatement()
org.apache.flink.api.common.io.statistics.BaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
org.apache.flink.types.Row	nextRecord(org.apache.flink.types.Row reuse) Stores the next resultSet row in a tuple.
void	open(org.apache.flink.core.io.InputSplit inputSplit) Connects to the source database and executes the query in a parallel fashion if this InputFormat is built using a parameterized query (i.e. using a PreparedStatement) and a proper JdbcParameterValuesProvider, in a non-parallel fashion otherwise.
void	openInputFormat()
boolean	reachedEnd() Checks whether all data has been read.

从类继承的方法 org.apache.flink.api.common.io.RichInputFormat

getRuntimeContext, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

serialVersionUID

protected static final long serialVersionUID

另请参阅:

常量字段值

LOG

protected static final org.slf4j.Logger LOG

connectionProvider

protected JdbcConnectionProvider connectionProvider

queryTemplate

protected String queryTemplate

resultSetType

protected int resultSetType

resultSetConcurrency

protected int resultSetConcurrency

rowTypeInfo

protected org.apache.flink.api.java.typeutils.RowTypeInfo rowTypeInfo

statement

protected transient PreparedStatement statement

resultSet

protected transient ResultSet resultSet

fetchSize

protected int fetchSize

autoCommit

protected Boolean autoCommit

hasNext

protected boolean hasNext

parameterValues

protected Object[][] parameterValues

构造器详细资料

JdbcInputFormat

public JdbcInputFormat()

方法详细资料

getProducedType

public org.apache.flink.api.java.typeutils.RowTypeInfo getProducedType()

指定者:

getProducedType 在接口中 org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

指定者:

configure 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

openInputFormat

public void openInputFormat()

覆盖:

openInputFormat 在类中 org.apache.flink.api.common.io.RichInputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

closeInputFormat

public void closeInputFormat()

覆盖:

closeInputFormat 在类中 org.apache.flink.api.common.io.RichInputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

open

public void open(org.apache.flink.core.io.InputSplit inputSplit)
          throws IOException

Connects to the source database and executes the query in a parallel fashion if this InputFormat is built using a parameterized query (i.e. using a PreparedStatement) and a proper JdbcParameterValuesProvider, in a non-parallel fashion otherwise.

指定者:

open 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

参数:

inputSplit - which is ignored if this InputFormat is executed as a non-parallel source, a "hook" to the query parameters otherwise (using its splitNumber)

抛出:

IOException - if there's an error during the execution of the query

close

public void close()
           throws IOException

Closes all resources used.

指定者:

close 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

抛出:

IOException - Indicates that a resource could not be closed.

reachedEnd

public boolean reachedEnd()
                   throws IOException

Checks whether all data has been read.

指定者:

reachedEnd 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

返回:

boolean value indication whether all data has been read.

抛出:

IOException

nextRecord

public org.apache.flink.types.Row nextRecord(org.apache.flink.types.Row reuse)
                                      throws IOException

Stores the next resultSet row in a tuple.

指定者:

nextRecord 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

参数:

reuse - row to be reused.

返回:

row containing next Row

抛出:

IOException

getStatistics

public org.apache.flink.api.common.io.statistics.BaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
                                                                       throws IOException

指定者:

getStatistics 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

抛出:

IOException

createInputSplits

public org.apache.flink.core.io.InputSplit[] createInputSplits(int minNumSplits)
                                                        throws IOException

指定者:

createInputSplits 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

指定者:

createInputSplits 在接口中 org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.io.InputSplit>

抛出:

IOException

getInputSplitAssigner

public org.apache.flink.core.io.InputSplitAssigner getInputSplitAssigner(org.apache.flink.core.io.InputSplit[] inputSplits)

指定者:

getInputSplitAssigner 在接口中 org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.io.InputSplit>

指定者:

getInputSplitAssigner 在接口中 org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.io.InputSplit>

getStatement

@VisibleForTesting
protected PreparedStatement getStatement()

getDbConn

@VisibleForTesting
protected Connection getDbConn()

buildJdbcInputFormat

public static JdbcInputFormat.JdbcInputFormatBuilder buildJdbcInputFormat()

A builder used to set parameters to the output format's configuration in a fluent way.

返回:

builder