datafu.pig.hash

Class HasherRand

java.lang.Object

org.apache.pig.EvalFunc<T>

datafu.pig.util.ContextualEvalFunc<T>

datafu.pig.util.SimpleEvalFunc<java.lang.String>

datafu.pig.hash.Hasher

datafu.pig.hash.HasherRand

public class HasherRand
extends Hasher

Computes a hash value of a string using a randomly generated seed and outputs it in hex. This class should only be used for hashing algorithms that accept a seed (murmur3-32, murmur3-128 and sip24). It allows you to generate a well-mixed sequence of values, unpredictable for every run, without relying on the random number generator for each record. The seed is generated by the front end (i.e. when you launch your script) and so is identical for every task.

See Also:

Hasher

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

org.apache.pig.EvalFunc.SchemaType

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.String	algorithm
protected com.google.common.hash.HashFunction	hash_func

Fields inherited from class datafu.pig.hash.Hasher

SEEDED_HASH_NAMES

Fields inherited from class org.apache.pig.EvalFunc

log, pigLogger, reporter, returnType

Constructor Summary

Constructors

Constructor and Description
HasherRand() Generates hash values according to murmur3-32, a non-cryptographic-strength hash function with good mixing.
HasherRand(java.lang.String alg) Generates hash values according to the given hash algorithm.

Method Summary

Modifier and Type	Method and Description
java.lang.String	call(java.lang.String val) Generates the hash for a string value.
protected java.util.Random	getRandomGenerator()
protected void	onReady(org.apache.pig.impl.logicalLayer.schema.Schema in_schema, org.apache.pig.impl.logicalLayer.schema.Schema out_schema) Generate a seed exactly once on the front end, so all workers get same value

Methods inherited from class datafu.pig.hash.Hasher

intFromHex, longFromHex, makeHashFunc

Methods inherited from class datafu.pig.util.SimpleEvalFunc

exec, getReturnType, outputSchema

Methods inherited from class datafu.pig.util.ContextualEvalFunc

getContextProperties, getInstanceName, getInstanceProperties, setUDFContextSignature

Methods inherited from class org.apache.pig.EvalFunc

allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLogger, getPigLogger, getReporter, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, progress, setInputSchema, setPigLogger, setReporter, warn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

hash_func

protected com.google.common.hash.HashFunction hash_func

algorithm

protected final java.lang.String algorithm

Constructor Detail

HasherRand

public HasherRand()

Generates hash values according to murmur3-32, a non-cryptographic-strength hash function with good mixing.

See Also:

HasherRand(String alg)

HasherRand

public HasherRand(java.lang.String alg)

Generates hash values according to the given hash algorithm.

Parameters:

alg - the hash algorithm to use

See Also:

HasherRand()

Method Detail

call

public java.lang.String call(java.lang.String val)

Generates the hash for a string value.

Overrides:

call in class Hasher

Parameters:

val - the single string to hash

Returns:

val, hashed according to the algorithm specified at instantiation

onReady

protected void onReady(org.apache.pig.impl.logicalLayer.schema.Schema in_schema,
                       org.apache.pig.impl.logicalLayer.schema.Schema out_schema)

Generate a seed exactly once on the front end, so all workers get same value

Overrides:

onReady in class ContextualEvalFunc<java.lang.String>

Parameters:

in_schema - Input schema

out_schema - Output schema

getRandomGenerator

protected java.util.Random getRandomGenerator()