Package org.datavec.local.transforms

Class AnalyzeLocal

java.lang.Object

org.datavec.local.transforms.AnalyzeLocal

public class AnalyzeLocal
extends Object

Constructor Summary

Constructors

Constructor	Description
AnalyzeLocal()

Method Summary

Modifier and Type	Method	Description
static DataAnalysis	analyze(Schema schema, RecordReader rr)	Analyse the specified data - returns a DataAnalysis object with summary information about each column
static DataAnalysis	analyze(Schema schema, RecordReader rr, int maxHistogramBuckets)	Analyse the specified data - returns a DataAnalysis object with summary information about each column
static DataQualityAnalysis	analyzeQuality(Schema schema, RecordReader data)	Analyze the data quality of data - provides a report on missing values, values that don't comply with schema, etc
static DataQualityAnalysis	analyzeQualitySequence(Schema schema, SequenceRecordReader data)	Analyze the data quality of sequence data - provides a report on missing values, values that don't comply with schema, etc
static Set<Writable>	getUnique(String columnName, Schema schema, RecordReader data)	Get a list of unique values from the specified columns.
static Map<String,Set<Writable>>	getUnique(List<String> columnNames, Schema schema, RecordReader data)	Get a list of unique values from the specified columns.
static Set<Writable>	getUniqueSequence(String columnName, Schema schema, SequenceRecordReader sequenceData)	Get a list of unique values from the specified column of a sequence
static Map<String,Set<Writable>>	getUniqueSequence(List<String> columnNames, Schema schema, SequenceRecordReader sequenceData)	Get a list of unique values from the specified columns of a sequence

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AnalyzeLocal

public AnalyzeLocal()

Method Detail

analyze

public static DataAnalysis analyze(Schema schema,
                                   RecordReader rr)

Analyse the specified data - returns a DataAnalysis object with summary information about each column

Parameters:

schema - Schema for data

rr - Data to analyze

Returns:

DataAnalysis for data

analyze

public static DataAnalysis analyze(Schema schema,
                                   RecordReader rr,
                                   int maxHistogramBuckets)

Analyse the specified data - returns a DataAnalysis object with summary information about each column

Parameters:

schema - Schema for data

rr - Data to analyze

Returns:

DataAnalysis for data

analyzeQualitySequence

public static DataQualityAnalysis analyzeQualitySequence(Schema schema,
                                                         SequenceRecordReader data)

Analyze the data quality of sequence data - provides a report on missing values, values that don't comply with schema, etc

Parameters:

schema - Schema for data

data - Data to analyze

Returns:

DataQualityAnalysis object

analyzeQuality

public static DataQualityAnalysis analyzeQuality(Schema schema,
                                                 RecordReader data)

Analyze the data quality of data - provides a report on missing values, values that don't comply with schema, etc

Parameters:

schema - Schema for data

data - Data to analyze

Returns:

DataQualityAnalysis object

getUnique

public static Set<Writable> getUnique(String columnName,
                                      Schema schema,
                                      RecordReader data)

Get a list of unique values from the specified columns. For sequence data, use getUniqueSequence(List, Schema, SequenceRecordReader)

Parameters:

columnName - Name of the column to get unique values from

schema - Data schema

data - Data to get unique values from

Returns:

List of unique values

getUnique

public static Map<String,Set<Writable>> getUnique(List<String> columnNames,
                                                        Schema schema,
                                                        RecordReader data)

Get a list of unique values from the specified columns. For sequence data, use getUniqueSequence(String, Schema, SequenceRecordReader)

Parameters:

columnNames - Names of the column to get unique values from

schema - Data schema

data - Data to get unique values from

Returns:

List of unique values, for each of the specified columns

getUniqueSequence

public static Set<Writable> getUniqueSequence(String columnName,
                                              Schema schema,
                                              SequenceRecordReader sequenceData)

Get a list of unique values from the specified column of a sequence

Parameters:

columnName - Name of the column to get unique values from

schema - Data schema

sequenceData - Sequence data to get unique values from

Returns:

getUniqueSequence

public static Map<String,Set<Writable>> getUniqueSequence(List<String> columnNames,
                                                                Schema schema,
                                                                SequenceRecordReader sequenceData)

Get a list of unique values from the specified columns of a sequence

Parameters:

columnNames - Name of the columns to get unique values from

schema - Data schema

sequenceData - Sequence data to get unique values from

Returns: