datafu.hourglass.avro
Class AvroKeyValueWithMetadataRecordWriter<K,V>
java.lang.Object
org.apache.hadoop.mapreduce.RecordWriter<K,V>
datafu.hourglass.avro.AvroKeyValueWithMetadataRecordWriter<K,V>
- Type Parameters:
K
- The type of key to write.V
- The type of value to write.
public class AvroKeyValueWithMetadataRecordWriter<K,V>
- extends org.apache.hadoop.mapreduce.RecordWriter<K,V>
Writes key/value pairs to an Avro container file.
Each entry in the Avro container file will be a generic record with two fields,
named 'key' and 'value'. The input types may be basic Writable objects like Text or
IntWritable, or they may be AvroWrapper subclasses (AvroKey or AvroValue). Writable
objects will be converted to their corresponding Avro types when written to the generic
record key/value pair.
Field Summary |
static java.lang.String |
TEXT_PREFIX
The configuration key prefix for a text output metadata. |
Constructor Summary |
AvroKeyValueWithMetadataRecordWriter(org.apache.avro.hadoop.io.AvroDatumConverter<K,?> keyConverter,
org.apache.avro.hadoop.io.AvroDatumConverter<V,?> valueConverter,
org.apache.avro.file.CodecFactory compressionCodec,
java.io.OutputStream outputStream,
org.apache.hadoop.conf.Configuration conf)
|
Method Summary |
void |
close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
|
org.apache.avro.Schema |
getWriterSchema()
Gets the writer schema for the key/value pair generic record. |
void |
write(K key,
V value)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
TEXT_PREFIX
public static final java.lang.String TEXT_PREFIX
- The configuration key prefix for a text output metadata.
- See Also:
- Constant Field Values
AvroKeyValueWithMetadataRecordWriter
public AvroKeyValueWithMetadataRecordWriter(org.apache.avro.hadoop.io.AvroDatumConverter<K,?> keyConverter,
org.apache.avro.hadoop.io.AvroDatumConverter<V,?> valueConverter,
org.apache.avro.file.CodecFactory compressionCodec,
java.io.OutputStream outputStream,
org.apache.hadoop.conf.Configuration conf)
throws java.io.IOException
- Throws:
java.io.IOException
getWriterSchema
public org.apache.avro.Schema getWriterSchema()
- Gets the writer schema for the key/value pair generic record.
- Returns:
- The writer schema used for entries of the Avro container file.
write
public void write(K key,
V value)
throws java.io.IOException
-
- Specified by:
write
in class org.apache.hadoop.mapreduce.RecordWriter<K,V>
- Throws:
java.io.IOException
close
public void close(org.apache.hadoop.mapreduce.TaskAttemptContext context)
throws java.io.IOException
-
- Specified by:
close
in class org.apache.hadoop.mapreduce.RecordWriter<K,V>
- Throws:
java.io.IOException
Matthew Hayes