org.apache.flink.formats.csv

Class RowCsvInputFormat

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<OT,org.apache.flink.core.fs.FileInputSplit>

org.apache.flink.api.common.io.FileInputFormat<T>

org.apache.flink.formats.csv.AbstractCsvInputFormat<org.apache.flink.types.Row>

org.apache.flink.formats.csv.RowCsvInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

public class RowCsvInputFormat
extends AbstractCsvInputFormat<org.apache.flink.types.Row>

Input format that reads csv into Row.

Different from old csv org.apache.flink.api.java.io.RowCsvInputFormat: 1.New csv will emit this row (Fill null the remaining fields) when row is too short. But Old csv will skip this too short row. 2.New csv, escape char will be removed. But old csv will keep the escape char.

These can be continuously improved in new csv input format: 1.New csv not support configure comment char. The comment char is "#". 2.New csv not support configure multi chars field delimiter. 3.New csv not support read first N, it will throw exception. 4.Only support configure line delimiter: "\r" or "\n" or "\r\n".

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RowCsvInputFormat.Builder A builder for creating a RowCsvInputFormat.

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.formats.csv.AbstractCsvInputFormat

csvInputStream, csvSchema

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Method Summary

Modifier and Type	Method and Description
static RowCsvInputFormat.Builder	builder(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo, org.apache.flink.core.fs.Path... filePaths) Create a builder.
org.apache.flink.types.Row	nextRecord(org.apache.flink.types.Row record)
void	open(org.apache.flink.core.fs.FileInputSplit split)
boolean	reachedEnd()

Methods inherited from class org.apache.flink.formats.csv.AbstractCsvInputFormat

supportsMultiPaths

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, close, configure, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFilePaths, getFileStats, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNestedFileEnumeration, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, registerInflaterInputStreamFactory, setFilePath, setFilePath, setFilePaths, setFilePaths, setFilesFilter, setMinSplitSize, setNestedFileEnumeration, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

open

public void open(org.apache.flink.core.fs.FileInputSplit split)
          throws IOException

Specified by:

open in interface org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,org.apache.flink.core.fs.FileInputSplit>

Overrides:

open in class AbstractCsvInputFormat<org.apache.flink.types.Row>

Throws:

IOException

reachedEnd

public boolean reachedEnd()

nextRecord

public org.apache.flink.types.Row nextRecord(org.apache.flink.types.Row record)
                                      throws IOException

Throws:

IOException

builder

public static RowCsvInputFormat.Builder builder(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo,
                                                org.apache.flink.core.fs.Path... filePaths)

Create a builder.