alluxio.job.plan.transform.format.csv

Class CsvSchema

java.lang.Object

alluxio.job.plan.transform.format.csv.CsvSchema

All Implemented Interfaces:

TableSchema

public final class CsvSchema
extends java.lang.Object
implements TableSchema

CSV table schema in Avro format.

Constructor Summary

Constructors

Constructor and Description
CsvSchema(@NotNull java.util.ArrayList<FieldSchema> schema) CsvReader uses AvroCSVReader to read records from CSV.

Method Summary

Modifier and Type	Method and Description
boolean	equals(java.lang.Object o)
java.util.ArrayList<FieldSchema>	getAlluxioSchema()
org.apache.avro.Schema	getReadSchema()
org.apache.avro.Schema	getWriteSchema()
int	hashCode()
ParquetSchema	toParquet()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CsvSchema

public CsvSchema(@NotNull
                 @NotNull java.util.ArrayList<FieldSchema> schema)
          throws java.io.IOException

CsvReader uses AvroCSVReader to read records from CSV. AvroCSVReader internally uses RecordBuilder, which does not support BYTES, and some logical types such as DECIMAL. So we use readSchema to let AvroCSVReader be able to read the record, then when writing out the record, we interpret the record according to writeSchema. For example, if a column is of type DECIMAL, in readSchema, the type will be STRING, but in writeSchema, it's logical type DECIMAL backed by BYTES.

Parameters:

schema - the schema from Alluxio table master

Throws:

java.io.IOException - when failed to initialize schema

Method Detail

getAlluxioSchema

public java.util.ArrayList<FieldSchema> getAlluxioSchema()

Returns:

the schema from Alluxio table master

getReadSchema

public org.apache.avro.Schema getReadSchema()

Returns:

the schema for reading from CSV

getWriteSchema

public org.apache.avro.Schema getWriteSchema()

Returns:

the schema for writing

toParquet

public ParquetSchema toParquet()

Specified by:

toParquet in interface TableSchema

Returns:

the schema in parquet representation

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object