|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object datafu.hourglass.mapreduce.ObjectProcessor datafu.hourglass.mapreduce.ObjectReducer datafu.hourglass.mapreduce.PartitioningReducer
public class PartitioningReducer
The reducer used by AbstractPartitionPreservingIncrementalJob
and its derived classes.
An implementation of Accumulator
is used to perform aggregation and produce the
output value.
The input key is assumed to have time and value fields. The value here is the true key, and the time represents the input partition the data was derived from. The true key is used as the key in the reducer output and the time is dropped. This reducer uses multiple outputs; the time is used to determine which output to write to, where the named outputs have the form yyyyMMdd derived from the time.
Constructor Summary | |
---|---|
PartitioningReducer()
|
Method Summary | |
---|---|
void |
close()
|
Accumulator<org.apache.avro.generic.GenericRecord,org.apache.avro.generic.GenericRecord> |
getAccumulator()
Gets the accumulator used to perform aggregation. |
PartitionPreservingSchemas |
getSchemas()
Gets the Avro schemas |
void |
reduce(java.lang.Object keyObj,
java.lang.Iterable<java.lang.Object> values,
org.apache.hadoop.mapreduce.ReduceContext<java.lang.Object,java.lang.Object,java.lang.Object,java.lang.Object> context)
|
void |
setAccumulator(Accumulator<org.apache.avro.generic.GenericRecord,org.apache.avro.generic.GenericRecord> acc)
Sets the accumulator used to perform aggregation. |
void |
setContext(org.apache.hadoop.mapreduce.TaskInputOutputContext<java.lang.Object,java.lang.Object,java.lang.Object,java.lang.Object> context)
|
void |
setSchemas(PartitionPreservingSchemas schemas)
Sets the Avro schemas. |
Methods inherited from class datafu.hourglass.mapreduce.ObjectProcessor |
---|
getContext |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public PartitioningReducer()
Method Detail |
---|
public void reduce(java.lang.Object keyObj, java.lang.Iterable<java.lang.Object> values, org.apache.hadoop.mapreduce.ReduceContext<java.lang.Object,java.lang.Object,java.lang.Object,java.lang.Object> context) throws java.io.IOException, java.lang.InterruptedException
reduce
in class ObjectReducer
java.io.IOException
java.lang.InterruptedException
public void setContext(org.apache.hadoop.mapreduce.TaskInputOutputContext<java.lang.Object,java.lang.Object,java.lang.Object,java.lang.Object> context)
setContext
in class ObjectProcessor
public void setAccumulator(Accumulator<org.apache.avro.generic.GenericRecord,org.apache.avro.generic.GenericRecord> acc)
acc
- The accumulatorpublic Accumulator<org.apache.avro.generic.GenericRecord,org.apache.avro.generic.GenericRecord> getAccumulator()
public void setSchemas(PartitionPreservingSchemas schemas)
schemas
- public PartitionPreservingSchemas getSchemas()
public void close() throws java.io.IOException, java.lang.InterruptedException
close
in class ObjectProcessor
java.io.IOException
java.lang.InterruptedException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |