Package weka.core.converters

Class TextDirectoryLoader

java.lang.Object

weka.core.converters.AbstractLoader

weka.core.converters.TextDirectoryLoader

All Implemented Interfaces:

Serializable, CommandlineRunnable, BatchConverter, IncrementalConverter, Loader, OptionHandler, RevisionHandler

public class TextDirectoryLoader
extends AbstractLoader
implements BatchConverter, IncrementalConverter, OptionHandler, CommandlineRunnable

Loads all text files in a directory and uses the subdirectory names as class labels. The content of the text files will be stored in a String attribute, the filename can be stored as well.

Valid options are:

 -D
  Enables debug output.
  (default: off)
 

 -F
  Stores the filename in an additional attribute.
  (default: off)
 

 -dir <directory>
  The directory to work on.
  (default: current directory)
 

 -charset <charset name>
  The character set to use, e.g UTF-8.
  (default: use the default character set)
 

 -R
  Retain all string attribute values when reading incrementally.
 

Based on code from the TextDirectoryToArff tool. See the

Wiki article

Version:

$Revision$

Author:

Ashraf M. Kibriya (amk14 at cs.waikato.ac.nz), Richard Kirkby (rkirkby at cs.waikato.ac.nz), fracpete (fracpete at waikato dot ac dot nz)

See Also:

Loader, Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface weka.core.converters.Loader

Loader.StructureNotReadyException

Field Summary

Fields inherited from interface weka.core.converters.Loader

BATCH, INCREMENTAL, NONE

Constructor Summary

Constructors

Constructor	Description
TextDirectoryLoader()	default constructor

Method Summary

Modifier and Type	Method	Description
String	charSetTipText()	the tip text for this property
String	debugTipText()	the tip text for this property
String	getCharSet()	Get the character set to use when reading text files.
Instances	getDataSet()	Return the full data set.
boolean	getDebug()	Gets whether additional debug information is printed.
File	getDirectory()	get the Dir specified as the source
String	getFileDescription()	Returns a description of the file type, actually it's directories.
Instance	getNextInstance(Instances structure)	Process input directories/files incrementally.
String[]	getOptions()	Gets the setting
boolean	getOutputFilename()	Gets whether the filename will be stored as an extra attribute.
String	getRevision()	Returns the revision string.
Instances	getStructure()	Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.
String	globalInfo()	Returns a string describing this loader
Enumeration<Option>	listOptions()	Lists the available options
static void	main(String[] args)	Main method.
String	outputFilenameTipText()	the tip text for this property
void	postExecution()	Perform any teardown stuff that might need to happen after execution.
void	preExecution()	Perform any setup stuff that might need to happen before commandline execution.
void	reset()	Resets the loader ready to read a new data set
void	run(Object toRun, String[] args)	Execute the supplied object.
void	setCharSet(String charSet)	Set the character set to use when reading text files (an empty string indicates that the default character set will be used).
void	setDebug(boolean value)	Sets whether to print some debug information.
void	setDirectory(File dir)	sets the source directory
void	setOptions(String[] options)	Parses a given list of options.
void	setOutputFilename(boolean value)	Sets whether the filename will be stored as an extra attribute.
void	setSource(File dir)	Resets the Loader object and sets the source of the data set to be the supplied File object.

Methods inherited from class weka.core.converters.AbstractLoader

setRetrieval, setSource

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextDirectoryLoader

public TextDirectoryLoader()

default constructor

Method Detail

globalInfo

public String globalInfo()

Returns a string describing this loader

Returns:

a description of the evaluator suitable for displaying in the explorer/experimenter gui

listOptions

public Enumeration<Option> listOptions()

Lists the available options

Specified by:

listOptions in interface OptionHandler

Returns:

an enumeration of the available options

setOptions

public void setOptions(String[] options)
                throws Exception

Parses a given list of options.

Valid options are:

 -D
  Enables debug output.
  (default: off)
 

 -F
  Stores the filename in an additional attribute.
  (default: off)
 

 -dir <directory>
  The directory to work on.
  (default: current directory)
 

 -charset <charset name>
  The character set to use, e.g UTF-8.
  (default: use the default character set)
 

Specified by:

setOptions in interface OptionHandler

Parameters:

options - the options

Throws:

Exception - if options cannot be set

getOptions

public String[] getOptions()

Gets the setting

Specified by:

getOptions in interface OptionHandler

Returns:

the current setting

charSetTipText

public String charSetTipText()

the tip text for this property

Returns:

the tip text

setCharSet

public void setCharSet(String charSet)

Set the character set to use when reading text files (an empty string indicates that the default character set will be used).

Parameters:

charSet - the character set to use.

getCharSet

public String getCharSet()

Get the character set to use when reading text files. An empty string indicates that the default character set will be used.

Returns:

the character set name to use (or empty string to indicate that the default character set will be used).

setDebug

public void setDebug(boolean value)

Sets whether to print some debug information.

Parameters:

value - if true additional debug information will be printed.

getDebug

public boolean getDebug()

Gets whether additional debug information is printed.

Returns:

true if additional debug information is printed

debugTipText

public String debugTipText()

the tip text for this property

Returns:

the tip text

setOutputFilename

public void setOutputFilename(boolean value)

Sets whether the filename will be stored as an extra attribute.

Parameters:

value - if true the filename will be stored in an extra attribute

getOutputFilename

public boolean getOutputFilename()

Gets whether the filename will be stored as an extra attribute.

Returns:

true if the filename is stored in an extra attribute

outputFilenameTipText

public String outputFilenameTipText()

the tip text for this property

Returns:

the tip text

getFileDescription

public String getFileDescription()

Returns a description of the file type, actually it's directories.

Returns:

a short file description

getDirectory

public File getDirectory()

get the Dir specified as the source

Returns:

the source directory

setDirectory

public void setDirectory(File dir)
                  throws IOException

sets the source directory

Parameters:

dir - the source directory

Throws:

IOException - if an error occurs

reset

public void reset()

Resets the loader ready to read a new data set

Specified by:

reset in interface Loader

Overrides:

reset in class AbstractLoader

setSource

public void setSource(File dir)
               throws IOException

Resets the Loader object and sets the source of the data set to be the supplied File object.

Specified by:

setSource in interface Loader

Overrides:

setSource in class AbstractLoader

Parameters:

dir - the source directory.

Throws:

IOException - if an error occurs

getStructure

public Instances getStructure()
                       throws IOException

Determines and returns (if possible) the structure (internally the header) of the data set as an empty set of instances.

Specified by:

getStructure in interface Loader

Specified by:

getStructure in class AbstractLoader

Returns:

the structure of the data set as an empty set of Instances

Throws:

IOException - if an error occurs

getDataSet

public Instances getDataSet()
                     throws IOException

Return the full data set. If the structure hasn't yet been determined by a call to getStructure then method should do so before processing the rest of the data set.

Specified by:

getDataSet in interface Loader

Specified by:

getDataSet in class AbstractLoader

Returns:

the structure of the data set as an empty set of Instances

Throws:

IOException - if there is no source or parsing fails

getNextInstance

public Instance getNextInstance(Instances structure)
                         throws IOException

Process input directories/files incrementally.

Specified by:

getNextInstance in interface Loader

Specified by:

getNextInstance in class AbstractLoader

Parameters:

structure - ignored

Returns:

never returns without throwing an exception

Throws:

IOException - if a problem occurs

getRevision

public String getRevision()

Returns the revision string.

Specified by:

getRevision in interface RevisionHandler

Returns:

the revision

main

public static void main(String[] args)

Main method.

Parameters:

args - should contain the name of an input file.

preExecution

public void preExecution()
                  throws Exception

Perform any setup stuff that might need to happen before commandline execution. Subclasses should override if they need to do something here

Specified by:

preExecution in interface CommandlineRunnable

Throws:

Exception - if a problem occurs during setup

postExecution

public void postExecution()
                   throws Exception

Perform any teardown stuff that might need to happen after execution. Subclasses should override if they need to do something here

Specified by:

postExecution in interface CommandlineRunnable

Throws:

Exception - if a problem occurs during teardown

run

public void run(Object toRun,
                String[] args)
         throws IllegalArgumentException

Description copied from interface: CommandlineRunnable

Execute the supplied object.

Specified by:

run in interface CommandlineRunnable

Parameters:

toRun - the object to execute

args - any options to pass to the object

Throws:

IllegalArgumentException