datafu.hourglass.avro
Class AvroKeyValueWithMetadataOutputFormat<K,V>

java.lang.Object
  org.apache.hadoop.mapreduce.OutputFormat<K,V>
      org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>
          org.apache.avro.mapreduce.AvroOutputFormatBase<K,V>
              datafu.hourglass.avro.AvroKeyValueWithMetadataOutputFormat<K,V>

Type Parameters:

K - The type of key. If an Avro type, it must be wrapped in an AvroKey.

V - The type of value. If an Avro type, it must be wrapped in an AvroValue.

public class AvroKeyValueWithMetadataOutputFormat<K,V>
extends org.apache.avro.mapreduce.AvroOutputFormatBase<K,V>

FileOutputFormat for writing Avro container files of key/value pairs.

Since Avro container files can only contain records (not key/value pairs), this output format puts the key and value into an Avro generic record with two fields, named 'key' and 'value'.

The keys and values given to this output format may be Avro objects wrapped in AvroKey or AvroValue objects. The basic Writable types are also supported (e.g., IntWritable, Text); they will be converted to their corresponding Avro types.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.Counter

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, PART

Constructor Summary

AvroKeyValueWithMetadataOutputFormat()

Method Summary

org.apache.hadoop.mapreduce.RecordWriter<K,V>

getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class org.apache.avro.mapreduce.AvroOutputFormatBase

getAvroFileOutputStream, getCompressionCodec

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroKeyValueWithMetadataOutputFormat

public AvroKeyValueWithMetadataOutputFormat()

Method Detail

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<K,V> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                              throws java.io.IOException

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<K,V>

Throws:

java.io.IOException