Probability and Statistics Probability vs Statistics • In probability, we build up from the mathematics of permutations and combinations and set theory a mathematical theory of how outcomes of an experiment will be distributed • In statistics we go in the opposite direction: We start from actual data, and measure it to determine what mathematical model it fits • Statistical measures are estimates of underlying random variables Basic Statistics • Let X be a finite sequence of numbers (data values) x1, …, xn • E.g X = 1, 3, 2, 4, 1, 4, (n=7) • Order doesn’t matter but we need a way of allowing duplicates (“multiset”) • Some measures: – Maximum: – Minimum: – Median (as many ≥ as ≤): • 1, 1, 1, 2, 3, 4, so median = – Mode (maximum frequency): Sample Mean • Let X be a finite sequence of numbers x1, …, xn • The sample mean of Xn is what we usually call xi the average:  X = ∑ n i=1 • For example, X = 1, 3, 2, 4, 1, 4, , then μX=16/7 • Note that the mean need not be one of the data values • We might as well write this as E[X] following the notation used for random variables Sample Variance • The sample variance of a sequence of data points is the mean of the square of the difference from the sample mean:    Var(X)           Standard Deviation • Standard deviation is the square root of the variance: n  = ∑ (xi − μ X ) n i=1 X • σ is a measure of spread in the same units as the data σ Measures “Spread” • • • • • • • X = 1, 2, μ=2 σ2 = (1/3) ∙ ((1-2)2+(2-2)2+(3-2)2) = 2/3 σ ≈ 82 Y = 1, 2, 3, 4, μ=3 σ2 = (1/5) ∙ ((1-3)2+(2-3)2+(3-3)2+(4-3)2+(53)2) = 10/5 σ ≈ 1.4 Small σ Indicates “Centeredness” • • • • • • • • • • • X = 1, 2, σ ≈ 82 Z = 1, 2, 2, 2, μ=2 σ2 = (1/5) ∙ ((1-2)2+3∙(2-2)2+(3-2)2) = 2/5 σ ≈ 63 W = 1, 1, 2, 3, μ=2 σ2 = (1/5) ∙ (2∙(1-2)2+(2-2)2+2∙(3-2)2) = 4/5 σ ≈ 89 (“Bimodal”) Covariance • Sometimes two quantities tend to vary in the same way, even though neither is exactly a function of the other • For example,Weight height and weight of people Height Covariance for Random Variables • Roll two dice Let X = larger of the two values, Y = sum of the two values • Mean of X = (1/36) × (1×1 [only possibility is (1,1)] +3×2 [(1,2), (2,1), (2,2)] +5×3 [(1,3), (2,3), (3,3), (3,2), (3,1)] +7×4 + 9×5 + 11×6) = 4.47 Mean of Y = How we say that X tends to be large when Y is large and vice versa? Joint Probability • f(x,y) = Pr(X=x and Y=y) is a probability • Sums to over all possible x and y • Pr(X=1 and Y=12) = • Pr(X=5 and Y=9) = 2/36 • Pr(X≤5 and Y≥8) = 4/36 [(4,4), (4,5), (5,4), (5,5)] Covariance of Random Variables • Cov(X, Y) = E[ (X − μX) ∙ (Y −μY) ] • INSIDE the brackets, each of X and Y is compared to its own mean • The OUTER expectation is with respect to the joint probability that X=x AND Y=y • Positive if X tends to be greater than its mean when Y is greater than its mean • Negative if X tends to be greater than its mean when Y is less than its mean • But what are the units? Sample Covariance • Suppose we just have the data x1, …, xN and y1, …, yN and we want to know the extent to which these two sets of values covary (eg height and weight) The sample covariance is  (xi          n  • An estimate of the covariance A Better Measure: Correlation • Correlation is Covariance scaled to [-1,1]  XY Cov(X,Y ) E[(X − μ X )⋅(Y − μ Y )] = = σ X ⋅σ Y Var(X)Var(Y ) • This is a unitless number! • If X and Y vary in the same direction then correlation is close to +1 • If they vary inversely then correlation is close to -1 • If neither depends on the mean of the other than the correlation is close to Positively Correlated Data Correlation Examples http://upload.wikimedia.org/wikipedia/commons/d/d4/Correlation_examples2.svg FINIS ... Statistics • In probability, we build up from the mathematics of permutations and combinations and set theory a mathematical theory of how outcomes of an experiment will be distributed • In statistics. .. ∑ n i=1 • For example, X = 1, 3, 2, 4, 1, 4, , then μX=16/7 • Note that the mean need not be one of the data values • We might as well write this as E[X] following the notation used for random... same way, even though neither is exactly a function of the other • For example,Weight height and weight of people Height Covariance for Random Variables • Roll two dice Let X = larger of the two