public class AvroDateRangeMetadata
extends java.lang.Object
AbstractPartitionCollapsingIncrementalJob
so that when reusing previous
output it can determine the date range the data corresponds to.Modifier and Type | Field and Description |
---|---|
static java.lang.String |
METADATA_DATE_END |
static java.lang.String |
METADATA_DATE_START |
Constructor and Description |
---|
AvroDateRangeMetadata() |
Modifier and Type | Method and Description |
---|---|
static void |
configureOutputDateRange(org.apache.hadoop.conf.Configuration conf,
DateRange dateRange)
Updates the Hadoop configuration so that the Avro files which are written have date range
information stored in the metadata.
|
static DateRange |
getOutputFileDateRange(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Reads the date range from the metadata stored in an Avro file.
|
public static java.lang.String METADATA_DATE_START
public static java.lang.String METADATA_DATE_END
public static DateRange getOutputFileDateRange(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) throws java.io.IOException
fs
- file system to access pathpath
- path to get date range forjava.io.IOException
- IOExceptionpublic static void configureOutputDateRange(org.apache.hadoop.conf.Configuration conf, DateRange dateRange)
AvroKeyValueWithMetadataRecordWriter
.conf
- configuration to store date range indateRange
- date range