datafu.pig.hash.lsh.metric

Class MetricUDF

java.lang.Object

org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

datafu.pig.hash.lsh.metric.MetricUDF

Direct Known Subclasses:

Cosine, L1, L2

public abstract class MetricUDF
extends org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

A base UDF used to find a vector v in a bag such that for query point q, metric m and threshold t m(v,q) < t. In other words, find the first vector in the bag within a threshold distance away.

It returns one of the tuples of the bag of vectors. For an example of its use, please see datafu.pig.hash.lsh.CosineDistanceHash.

See Also:

CosineDistanceHash

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields

Modifier and Type	Field and Description
protected int	dim

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
MetricUDF(java.lang.String sDim) Create a new Metric UDF with a given dimension.

Method Summary

Modifier and Type	Method and Description
protected abstract double	dist(org.apache.commons.math.linear.RealVector v1, org.apache.commons.math.linear.RealVector v2) The distance metric used.
org.apache.pig.data.Tuple	exec(org.apache.pig.data.Tuple input) This UDF expects a query vector as the first element, a threshold (double) as the second, and a bag of vectors.
org.apache.pig.impl.logicalLayer.schema.Schema	outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input) Create the output schema, based on the input schema.

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, progress, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dim

protected int dim

Constructor Detail

MetricUDF

public MetricUDF(java.lang.String sDim)

Create a new Metric UDF with a given dimension.

Parameters:

sDim - dimension

Method Detail

dist

protected abstract double dist(org.apache.commons.math.linear.RealVector v1,
                               org.apache.commons.math.linear.RealVector v2)

The distance metric used. Given v1 and v2, compute the distance between those vectors.

Parameters:

v1 - first vector

v2 - second vector

Returns:

the distance between v1 and v2

exec

public org.apache.pig.data.Tuple exec(org.apache.pig.data.Tuple input)
                               throws java.io.IOException

This UDF expects a query vector as the first element, a threshold (double) as the second, and a bag of vectors. Vectors are represented by tuples with doubles as elements or bags of tuples representing position and value in the case of sparse vectors.

It returns one of the tuples of the bag of vectors. For an example of its use, please see datafu.pig.hash.lsh.CosineDistanceHash.

Specified by:

exec in class org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

Throws:

java.io.IOException

See Also:

CosineDistanceHash

outputSchema

public org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)

Create the output schema, based on the input schema.

Overrides:

outputSchema in class org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>

Returns:

the output schema, which is a tuple matching the schema of the third input field.