org.apache.hadoop.streaming

Class StreamXmlRecordReader

java.lang.Object

org.apache.hadoop.streaming.StreamBaseRecordReader

org.apache.hadoop.streaming.StreamXmlRecordReader

All Implemented Interfaces:

org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

public class StreamXmlRecordReader
extends StreamBaseRecordReader

A way to interpret XML fragments as Mapper input records. Values are XML subtrees delimited by configurable tags. Keys could be the value of a certain attribute in the XML subtree, but this is left to the stream processor application. The name-value properties that StreamXmlRecordReader understands are: String begin (chars marking beginning of record) String end (chars marking end of record) int maxrec (maximum record size) int lookahead(maximum lookahead to sync CDATA) boolean slowmatch

Field Summary

Fields inherited from class org.apache.hadoop.streaming.StreamBaseRecordReader

LOG

Constructor Summary

Constructors

Constructor and Description
StreamXmlRecordReader(org.apache.hadoop.fs.FSDataInputStream in, org.apache.hadoop.mapred.FileSplit split, org.apache.hadoop.mapred.Reporter reporter, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.fs.FileSystem fs)

Method Summary

Methods

Modifier and Type	Method and Description
void	init()
boolean	next(org.apache.hadoop.io.Text key, org.apache.hadoop.io.Text value) Read a record.
void	seekNextRecordBoundary() Implementation should seek forward in_ to the first byte of the next record.

Methods inherited from class org.apache.hadoop.streaming.StreamBaseRecordReader

close, createKey, createValue, getPos, getProgress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamXmlRecordReader

public StreamXmlRecordReader(org.apache.hadoop.fs.FSDataInputStream in,
                     org.apache.hadoop.mapred.FileSplit split,
                     org.apache.hadoop.mapred.Reporter reporter,
                     org.apache.hadoop.mapred.JobConf job,
                     org.apache.hadoop.fs.FileSystem fs)
                      throws IOException

Throws:

IOException

Method Detail

init

public void init()
          throws IOException

Throws:

IOException

next

public boolean next(org.apache.hadoop.io.Text key,
           org.apache.hadoop.io.Text value)
             throws IOException

Description copied from class: StreamBaseRecordReader

Read a record. Implementation should call numRecStats at the end

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Specified by:

next in class StreamBaseRecordReader

Throws:

IOException

seekNextRecordBoundary

public void seekNextRecordBoundary()
                            throws IOException

Description copied from class: StreamBaseRecordReader

Implementation should seek forward in_ to the first byte of the next record. The initial byte offset in the stream is arbitrary.

Specified by:

seekNextRecordBoundary in class StreamBaseRecordReader

Throws:

IOException