datafu.hourglass.avro
Class CombinedAvroKeyInputFormat<T>

java.lang.Object
  org.apache.hadoop.mapreduce.InputFormat<K,V>
      org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>
          org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>
              datafu.hourglass.avro.CombinedAvroKeyInputFormat<T>

Type Parameters:

T - Type of data to be read

public class CombinedAvroKeyInputFormat<T>
extends org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

A combined input format for reading Avro data.

Author:

"Matthew Hayes"

Nested Class Summary

static class

CombinedAvroKeyInputFormat.CombinedAvroKeyRecordReader<T>

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

org.apache.hadoop.mapreduce.lib.input.FileInputFormat.Counter

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat

SPLIT_MINSIZE_PERNODE, SPLIT_MINSIZE_PERRACK

Constructor Summary

CombinedAvroKeyInputFormat()

Method Summary

org.apache.hadoop.mapreduce.RecordReader<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat

createPool, createPool, getFileBlockLocations, getSplits, isSplitable, setMaxSplitSize, setMinSplitSizeNode, setMinSplitSizeRack

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, listStatus, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CombinedAvroKeyInputFormat

public CombinedAvroKeyInputFormat()

Method Detail

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable> createRecordReader(org.apache.hadoop.mapreduce.InputSplit inputSplit,
                                                                                                                                        org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                                                                                 throws java.io.IOException

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.lib.input.CombineFileInputFormat<org.apache.avro.mapred.AvroKey<T>,org.apache.hadoop.io.NullWritable>

Throws:

java.io.IOException