John Reif

Probability Theory:

(a) Random Variables:

Binomial and Geometric

(b) Useful Probabilistic Bounds and Inequalities

Auxiliary Reading Selections:

BB Chapter 8

Appendix of Queueing Systems, Vol. I by Kleinrock

a probability measure:

Prob is a mapping from a set of events to the reals such that

(1) for any event A

0 £ Prob(A) £ 1

(2) Prob (all possible events) = 1

(3) if A,B are mutually exclusive events, then

Prob (A or B) = Prob (A) + Prob (B)

Conditional Probability

Define:

Prob(A|B) = Prob (A Ù B)Prob (B)

for Prob(B) > 0

Bayes' Theorem If A1 ,..., An are mutually exclusive and contain all events

where Pj = Prob(B|Aj) × Prob(Aj)

Random Variable A

(over real numbers)

Density Function fA(x)=Prob(A=x)

probability Density Function

probability Distribution Function

If for Random Variables A,B

"x FA(x) £ FB(x) then

"A upper bounds B" and

"B lower bounds A"

FA(x) = Prob (A £ x)

FB(x) = Prob (B £ x)

Expectation of Random Variable A

note FA( mean of A) = 1/2

Variance of Random Variable A

n'th Moments of Random Variable A

moment generating function

note:

s is a formal parameter.

Discrete Random Variable A over nonnegative integers

Probability generating function

Definition of independence:

A,B independent if Prob(AÙB)=Prob(A) × Prob(B)

equivalent definition of independence

fAÙB (x) = fA (x) × fB(x)

MAÙB (s) = MA (s) × MB(s)

GAÙB (z) = GA (z) × GB(z)

If A1,...,An independent with same distribution

Combinatorics

n! = n×(n-1) ... 2×1

= number of permutations of n objects

Stirling's formula

n! = f(n) (1+o(1))

Bounds (due to Erdos & Spencer, p. 18)

Bernoulli Variable Ai is

1 with prob P

0 with prob 1-P

Binomial Variable B is sum of n independent Bernoulli variables Ai each with some probability p

procedure BINOMIAL with parameters n,p

begin

B = 0

for i=1 to n do

with probability P do B = B+1

output B

end

B is Binomial Variable with parameters n,p

generating function

Poisson trial Ai is 1 with prob Pi

and 0 with prob 1-Pi

Suppose B' is the sum of

n independent Poisson trials Ai with

probability Pi for i>1,...,n

Hoeffding's Theorem B' is upper bound

by a Binomial Variable B with

Geometric Variable V parameter p

procedure GEOMETRIC

parameter p

begin

V= 0

loop:

with probability p goto exit

GEOMETRIC generating function

Probabilistic Inequalities

for Random Variable A

Markov Inequality (uses only mean)

Chebychev Inequality (uses mean and variance)

example If B is Binomial with parameters n,p

Gaussian Density function

Normal Distribution

Let Sn be the sum of n independently

distributed variables A1,...,An

Strong Law of Large Numbers

Chernoff Bound (uses all moments)

of Random Variable A

need moment generating function

Chernoff Bound of

Discrete Random Variable A

choose z=zo to minimize above bound

need Probability Generating function

Chernoff Bounds for

Binomial B with parameters n,p

Below Mean x £ m

Anguin-Valiant’s Bounds

for Binomial B with parameters n,p

Just above mean m = np for 0< e <1

Just below mean m for 0< e <1

Þ tails are bounded by Normal distributions

Binomial Variable X with parameters p, N

and expectation µ=pN

By Chernoff, Bound for p≤1/2

Prob(X≥ N/2)<2N° pN/2

Raghavan - Spencer bound

For any ∂>0,

in FOCS‘86.