datafu.hourglass.avro

Class AvroKeyValueWithMetadataRecordWriter<K,V>

java.lang.Object

org.apache.hadoop.mapreduce.RecordWriter<K,V>

datafu.hourglass.avro.AvroKeyValueWithMetadataRecordWriter<K,V>

Type Parameters:

K - The type of key to write.

V - The type of value to write.

public class AvroKeyValueWithMetadataRecordWriter<K,V>
extends org.apache.hadoop.mapreduce.RecordWriter<K,V>

Writes key/value pairs to an Avro container file.

Each entry in the Avro container file will be a generic record with two fields, named 'key' and 'value'. The input types may be basic Writable objects like Text or IntWritable, or they may be AvroWrapper subclasses (AvroKey or AvroValue). Writable objects will be converted to their corresponding Avro types when written to the generic record key/value pair.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	TEXT_PREFIX The configuration key prefix for a text output metadata.

Constructor Summary

Constructors

Constructor and Description
AvroKeyValueWithMetadataRecordWriter(org.apache.avro.hadoop.io.AvroDatumConverter<K,?> keyConverter, org.apache.avro.hadoop.io.AvroDatumConverter<V,?> valueConverter, org.apache.avro.file.CodecFactory compressionCodec, java.io.OutputStream outputStream, org.apache.hadoop.conf.Configuration conf)

Method Summary

Methods

Modifier and Type	Method and Description
void	close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
org.apache.avro.Schema	getWriterSchema() Gets the writer schema for the key/value pair generic record.
void	write(K key, V value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TEXT_PREFIX

public static final java.lang.String TEXT_PREFIX

The configuration key prefix for a text output metadata.

See Also:

Constant Field Values

Constructor Detail

AvroKeyValueWithMetadataRecordWriter

public AvroKeyValueWithMetadataRecordWriter(org.apache.avro.hadoop.io.AvroDatumConverter<K,?> keyConverter,
                                    org.apache.avro.hadoop.io.AvroDatumConverter<V,?> valueConverter,
                                    org.apache.avro.file.CodecFactory compressionCodec,
                                    java.io.OutputStream outputStream,
                                    org.apache.hadoop.conf.Configuration conf)
                                     throws java.io.IOException

Throws:

java.io.IOException

Method Detail

getWriterSchema

public org.apache.avro.Schema getWriterSchema()

Gets the writer schema for the key/value pair generic record.

Returns:

The writer schema used for entries of the Avro container file.

write

public void write(K key,
         V value)
           throws java.io.IOException

Specified by:

write in class org.apache.hadoop.mapreduce.RecordWriter<K,V>

Throws:

java.io.IOException

close

public void close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
           throws java.io.IOException

Specified by:

close in class org.apache.hadoop.mapreduce.RecordWriter<K,V>

Throws:

java.io.IOException