Package org.apache.flink.connector.file.src.reader

Class TextLineInputFormat

java.lang.Object

org.apache.flink.connector.file.src.reader.SimpleStreamFormat<String>

org.apache.flink.connector.file.src.reader.TextLineInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.java.typeutils.ResultTypeQueryable<String>, StreamFormat<String>

@PublicEvolving
public class TextLineInputFormat
extends SimpleStreamFormat<String>

A reader format that text lines from a file.

The reader uses Java's built-in InputStreamReader to decode the byte stream using various supported charset encodings.

This format does not support optimized recovery from checkpoints. On recovery, it will re-read and discard the number of lined that were processed before the last checkpoint. That is due to the fact that the offsets of lines in the file cannot be tracked through the charset decoders with their internal buffering of stream input and charset decoder state.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TextLineInputFormat.Reader	The actual reader for the TextLineInputFormat.

Field Summary

Fields

Modifier and Type	Field	Description
static String	DEFAULT_CHARSET_NAME

Fields inherited from interface org.apache.flink.connector.file.src.reader.StreamFormat

FETCH_IO_SIZE

Constructor Summary

Constructors

Constructor	Description
TextLineInputFormat()
TextLineInputFormat(String charsetName)

Method Summary

Modifier and Type	Method	Description
TextLineInputFormat.Reader	createReader(org.apache.flink.configuration.Configuration config, org.apache.flink.core.fs.FSDataInputStream stream)	Creates a new reader.
org.apache.flink.api.common.typeinfo.TypeInformation<String>	getProducedType()	Gets the type produced by this format.

Methods inherited from class org.apache.flink.connector.file.src.reader.SimpleStreamFormat

createReader, isSplittable, restoreReader

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_CHARSET_NAME

public static final String DEFAULT_CHARSET_NAME

See Also:

Constant Field Values

Constructor Detail

TextLineInputFormat

public TextLineInputFormat()

TextLineInputFormat

public TextLineInputFormat(String charsetName)

Method Detail

createReader

public TextLineInputFormat.Reader createReader(org.apache.flink.configuration.Configuration config,
                                               org.apache.flink.core.fs.FSDataInputStream stream)
                                        throws IOException

Description copied from class: SimpleStreamFormat

Creates a new reader. This method is called both for the creation of new reader (from the beginning of a file) and for restoring checkpointed readers.

If the reader previously checkpointed an offset, then the input stream will be positioned to that particular offset. Readers checkpoint an offset by returning a value from the method StreamFormat.Reader.getCheckpointedPosition() method with an offset other than CheckpointedPosition.NO_OFFSET).

Specified by:

createReader in class SimpleStreamFormat<String>

Throws:

IOException

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<String> getProducedType()

Description copied from class: SimpleStreamFormat

Gets the type produced by this format. This type will be the type produced by the file source as a whole.

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<String>

Specified by:

getProducedType in interface StreamFormat<String>

Specified by:

getProducedType in class SimpleStreamFormat<String>