MCB112: Biological Data Analysis (Fall 2018)


Section 02: Hashing and Randomness

Notes by Daniel Eaton [9/21/2018], adapted from William Mallard [9/15/2017]

Hashing

Hash functions

A hash function maps keys (data of arbitrary size) to hashes (values of a fixed size). For example, consider a hash function that takes any possible string of characters as its input, and generates a 32-bit integer as its output. This function would take the string's bytes, smash them together via some special combination of bit shifts and logical operators, and spit out a 32-bit integer. It does so in such a way that keys will be uniformly distributed across the range of outputs -- so if you hash a string, and then change it by a single letter and hash it again, the two hash values will be totally different.

def hash_function(string):
    total = 0
    for char in string:
        total = total + ord(char)
    output = total%(2**32)
    string = '{:032b}'.format(output)
    return string

print(hash_function("What is love."))
print(hash_function("Baby don't hurt me."))
print(hash_function("Don't hurt me."))
print(hash_function("No more."))
print(hash_function("a"*1000000))

# Result:
# '00000000000000000000010010010100'
# '00000000000000000000011001111101'
# '00000000000000000000010010111111'
# '00000000000000000000001010111110'
# '00000101110010000001101001000000'

Hash tables

A hash table is a data structure built on top of a normal list, with its length equal to the number of possible hashes. To add a value to the hash table, you hash the key, and add some value to the list at the index corresponding to the hash. To illustrate why we use hash functions and hash tables, consider the problem of accessing keyed information without the aid of a hash. You would have to access each piece of data seperately and check if it is related to the key (presumably through some logical operation). In the worst case, you would comb through the entire dataset before finding your key's assocated value as the last entry.

Image taken from wikipedia (https://upload.wikimedia.org/wikipedia/commons/thumb/7/7d/Hash_table_3_1_1_0_1_0_0_SP.svg/315px-Hash_table_3_1_1_0_1_0_0_SP.svg.png), originally uploaded by Jorge Stolfi

Randomness

Random vs Pseudorandom

Randomness refers to the absence of any pattern. Truly random numbers only arise from physical processes (eg, radioactive decay, thermal noise, etc). Sequences of random numbers exhibit certain statistical properties that are useful for various computational applications.

Computers are deterministic, so they cannot generate truly random numbers on their own. However, there are ways to make them generate sequences of numbers with many of the statistial properties of a truly random sequence. We call these random-looking (though ultimately deterministic) sequences pseudorandom.

Pseudorandom Number Generators

Pseudorandom number generators (PRNGs) produce sequences of pseudorandom numbers. At their core is a recursive function combining bit shifts and bitwise logical operations. You feed the previous number into the generator to get the next number in the sequence.

RANDU (simple but bad)

RANDU is a pseudorandom number generator developed in the 1960s that produces awful, correlated random numbers. For this reason, it is no longer in widespread use. But, it is remarkably simple and illustrates some of the principles of pseudorandom number generators.

The jth random number generated by RANDU is defined by the following recurrance relation:

Vj+1 = 65539 * Vj mod 231

This will generate a pseudorandom integer on the interval [1 , 231 - 1]. Usually, you want to generate a pseudorandom number drawn from a continuous uniform distribution on (0,1) so you transform Vj as follows:

Xj = Vj / 231

Defining RANDU as a function in python, we can see it generates numbers that look pretty random:

def randu(V_j):
    big_number = 65539 * V_j
    V_j_1 = big_number%(2**31)
    return V_j_1

V_0 = 189243
V_1 = randu(V_0)
V_2 = randu(randu(V_0))
V_3 = randu(randu(randu(V_0)))

print(V_0,V_1,V_2,V_3)

# Result: 189243 1665378737 1400634643 2005333817
def convert_to_uniform(V_j):
    X_j = V_j/(2**31)
    return X_j

X_1 = convert_to_uniform(V_1)
X_2 = convert_to_uniform(V_2)
X_3 = convert_to_uniform(V_3)

print(X_1,X_2,X_3)

# Result: 0.7755024065263569 0.6522213309071958 0.9338063267059624
def nrandu(V_0,n):
    V_j = V_0
    for i in range(n):
        V_j = randu(V_j)
        X_j = convert_to_uniform(V_j)
        print(X_j)

V_0 = 189243
nrandu(V_0,20)
# Result:
# 0.7755024065263569
# 0.6522213309071958
# 0.9338063267059624
# 0.7328459820710123
# 0.9928189520724118
# 0.3612998737953603
# 0.23242867412045598
# 0.14287318056449294
# 0.7653810163028538
# 0.30642747273668647
# 0.9501356896944344
# 0.9429668835364282
# 0.10658009396865964
# 0.15277861198410392
# 0.9574508261866868
# 0.36969744926318526
# 0.6011272598989308
# 0.27948651602491736
# 0.2667737570591271
# 0.08526389813050628

Note that it is important to initialize RANDU with a seed that is odd. If an even number is chosen, RANDU will eventually output the number 231 / 65539 This is a very bad number for RANDU, because it produces an output of 0. 0 will then produce 0 and so on, crashing RANDU. You can see this here:

nrandu(((2**31)/65539),10)
# Result:
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0
# 0.0

Unfortunately RANDU does not produce random numbers, they are pseudo-random, with an emphasis on the pseudo. RANDU actually generates highly correlated numbers. The easiest way to see this is to make a 3D plot whose points' coordinates correspond to three consecutive numbers generated by RANDU. A truely random number generator should produce a uniform "cloud" in this space. As you can see, these points are very regularly distributed into 15 two-dimensional planes.

Plot taken from wikipedia (https://upload.wikimedia.org/wikipedia/commons/3/38/Randu.png), originally uploaded by Luis Sanchez and plotted in Matlab

Luckily python uses a much less pathological random number generator than RANDU that is (usually) suffecient for generic random sampling tasks.

Seeds

Where does the PRNG get its very first number? In the case of RANDU, we clearly specified a V0. By default, Python seeds its PRNG with whatever time it is when you ask for your first random number. But there's nothing stopping you from overriding that and giving it your favorite number!

A seed is a number you give a PRNG to initialize its internal state. So in a sense, the seed serves as a unique identifier for a sequence of pseudorandom numbers.

What's nice about this, is you can initialize Python's PRNG to some state at the beginning of your program, and then every subsequent run will use the same sequence of pseudorandom numbers.

Why would you want that?

1.) Debugging. This is useful for comparing your results as you tweak your code. If your edits didn't alter the number or order of calls to the PRNG, then the random data you're working with should be consistent across runs.

2.) Reproducibility. When you give your code to someone else, or publish it in a journal, other people can re-run your code and verify that they get the exact same output. Biology is currently plagued by irreproducible results. Biological systems are intrinsically noisy, so there's probably a limit to how reproducible we can make results from the bench. But computational analyses have no excuse for being irreproducible, as long as you provide your analysis code, and seed your random number generators!

To seed Python's pseudorandom number generator:

import numpy as np

np.random.seed(42)

You can then proceed to use functions from np.random as usual.

import numpy as np

# Seed the PRNG with 1, and randomly
# generate 20 integers from 0 through 9.
np.random.seed(1)
np.random.randint(10, size=20)
# Result: array([5, 8, 9, 5, 0, 0, 1, 7, 6, 9, 2, 4, 5, 2, 4, 2, 4, 7, 7, 9])
# Seed the PRNG with 2, and randomly
# generate 20 integers from 0 through 9.
np.random.seed(2)
np.random.randint(10, size=20)
# Result: array([8, 8, 6, 2, 8, 7, 2, 1, 5, 4, 4, 5, 7, 3, 6, 4, 3, 7, 6, 1])
# These 20 numbers differ from the first 20.
# Seed the PRNG with 1 again, and randomly
# generate 20 integers from 0 through 9.
np.random.seed(1)
np.random.randint(10, size=20)
# Result: array([5, 8, 9, 5, 0, 0, 1, 7, 6, 9, 2, 4, 5, 2, 4, 2, 4, 7, 7, 9])
# These 20 numbers match the first 20 exactly!
# Seed the PRNG with 1 again, and randomly
# generate 20 integers from 0 through 9.
np.random.seed(1)
print(np.random.randint(10, size=5))
print(np.random.randint(10, size=5))
print(np.random.randint(10, size=5))
print(np.random.randint(10, size=5))
# Result:
# [5 8 9 5 0]
# [0 1 7 6 9]
# [2 4 5 2 4]
# [2 4 7 7 9]
# These 20 numbers still match the first 20,
# even though we pulled them out 5 at a time.

Python tips

Random functions

NumPy includes a random module which includes a number of handy functions for generating and working with random numbers.

To access numpy.random's functions, import numpy:

import numpy as np

To generate a random number from the half-open interval [0,1):

x = np.random.random()

To select an item from a list according to a list of weights:

L = ['abc', 'def', 'ghi']
w = [.2, .5, .3]

x = np.random.choice(L, p=w)

Note that your list of weights is supposed to be a probability distribution, so it must sum to 1. If it doesn't, choice() will complain.

To select a number from a normal distribution with mean mu and standard deviation sigma:

x = np.random.normal(mu, sigma)

You can read up on the various random functions on the SciPy website. We'll discuss seed() and seeding your random number generator below.

Translation tables

If you want to transform a string according to some specific set of character substitutions, you can efficiently do so with a translation table.

T = str.maketrans('abc', 'xyz')

print('abacab'.translate(T))
print('ccccba'.translate(T))
print('bcbcba'.translate(T))
# Result:
# xyxzxy
# zzzzyx
# yzyzyx

Note that we only need to build the translation table once. As long as we store it somewhere, we can reuse the same translation table over and over again.

String reversal

To reverse a string, we can use a common list and string slicing idiom.

S = 'abcdef'
print(S[::-1])
# Result:
# fedcba

Writing gzip files

Text-based bioinformatics data is usually stored and shared in compressed form. Common compression tools are gzip (.gz files), WinZip (.zip files), and bzip2 (.bz2 files). If you have an uncompressed file called foo.txt, simply run gzip foo.txt to generate a compressed version called foo.txt.gz. If foo.txt was larger than a few kilobytes, this compressed version will take up a fraction of the space.

To generate a gzip'd text file directly from Python, you can use the gzip library. The gzip library provides an open() function that works just like the normal open() function, except it compresses the data before writing it to disk. There is one small difference in its usage: Instead of opening the file in write mode with 'w', we need to specify that we want to open the gzip file in text-writing mode with 'wt'.

import gzip

with gzip.open('foo.txt.gz', 'wt') as fd:
    for line in lines:
        print(line, file=fd)

Basic Numpy

Creating numpy arrays

Making a 1D array

import numpy as np

list_a = [0,1,2]
array_a = np.array(list_a)

print(array_a)
# Result: [0 1 2]

Making a 2D array

list_b = [[0,1,2],[3,4,5],[6,7,8]]
array_b = np.array(list_b)

print(array_b)
# Result:
# [[0 1 2]
#  [3 4 5]
#  [6 7 8]]

Checking the size of each dimension

print(array_a.shape)
print(array_b.shape)
# Result:
# (3,)
# (3,3)

Making a zero array of arbitrary shape

zero_array = np.zeros((4,3))
print(zero_array)
# Result:
# [[0. 0. 0.]
#  [0. 0. 0.]
#  [0. 0. 0.]
#  [0. 0. 0.]]

Simple operations

A = np.array([[1,0],[1,1]])
B = np.array([[0,1],[1,1]])
print(A)
print(B)
# Result:
# [[1 0]
#  [1 1]]
# [[0 1]
#  [1 1]]

Element-wise addition

C = A + B
print(C)
# Result:
# [[1 1]
#  [2 2]]

Element-wise multiplication

C = A * B
print(C)
# Result:
# [[0 0]
#  [1 1]]

Multiplicative scaling

C = A * 0.5
print(C)
# Result:
# [[0.5 0. ]
#  [0.5 0.5]]

Adding to each element

C = A + 0.5
print(C)
# Result:
# [[1.5 0.5]
#  [1.5 1.5]]

Matrix multiplication

C = A @ B
print(C)
# Result:
# [[0 1]
#  [1 2]]

Adding together all elements of an array

C = np.sum(A)
print(C)
# Result: 3

Summing along the rows of an array

C = np.sum(A, axis=1)
print(C)
# Result:
# [1 2]

Summing along the columns of an array

C = np.sum(A, axis=0)
print(C)
# Result:
# [2 1]