Statistics for Business and Economics 7th Edition Chapter 11 Simple Regression Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-1 Chapter Goals After completing this chapter, you should be able to:  Explain the simple linear regression model  Obtain and interpret the simple linear regression equation for a set of data  Describe R2 as a measure of explanatory power of the regression model  Understand the assumptions behind regression analysis  Explain measures of variation and determine whether the independent variable is significant Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-2 Chapter Goals (continued) After completing this chapter, you should be able to:  Calculate and interpret confidence intervals for the regression coefficients  Use a regression equation for prediction  Form forecast intervals around an estimated Y value for a given X  Use graphical analysis to recognize potential problems in regression analysis  Explain the correlation coefficient and perform a hypothesis test for zero population correlation Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-3 11.1  Overview of Linear Models An equation can be fit to show the best linear relationship between two variables: Y = β0 + β1X Where Y is the dependent variable and X is the independent variable β0 is the Y-intercept β1 is the slope Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-4 Least Squares Regression  Estimates for coefficients β0 and β1 are found using a Least Squares Regression technique  The least-squares regression line, based on sample data, is yˆ b0  b1x  Where b1 is the slope of the line and b0 is the yintercept: Cov(x, y) b1  s2x Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall b y  b1x Ch 11-5 Introduction to Regression Analysis  Regression analysis is used to:  Predict the value of a dependent variable based on the value of at least one independent variable  Explain the impact of changes in an independent variable on the dependent variable Dependent variable: the variable we wish to explain (also called the endogenous variable) Independent variable: the variable used to explain the dependent variable (also called the exogenous variable) Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-6 11.2 Linear Regression Model  The relationship between X and Y is described by a linear function  Changes in Y are assumed to be caused by changes in X  Linear regression population equation model Yi β0  β1x i  ε i  Where 0 and 1 are the population model coefficients and  is a random error term Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-7 Simple Linear Regression Model The population regression model: Population Y intercept Dependent Variable Population Slope Coefficient Independent Variable Random Error term Yi β0  β1Xi  ε i Linear component Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Random Error component Ch 11-8 Simple Linear Regression Model (continued) Y Yi β0  β1Xi  ε i Observed Value of Y for Xi εi Predicted Value of Y for Xi Slope = β1 Random Error for this Xi value Intercept = β0 Xi Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall X Ch 11-9 Simple Linear Regression Equation The simple linear regression equation provides an estimate of the population regression line Estimated (or predicted) y value for observation i Estimate of the regression Estimate of the regression slope intercept yˆ i b0  b1x i Value of x for observation i The individual random error terms ei have a mean of zero ei ( y i - yˆ i ) y i - (b0  b1x i ) Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-10 F-Test for Significance (continued) Test Statistic: H0 : β = MSR F 11.08 MSE H1 : β ≠  = 05 df1= df2 = Decision: Reject H0 at  = 0.05 Critical Value: F = 5.32 Conclusion:  = 05 Do not reject H0 Reject H0 F There is sufficient evidence that house size affects selling price F.05 = 5.32 Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-50 11.6 Prediction  The regression equation can be used to predict a value for y, given a particular x  For a specified value, xn+1 , the predicted value is yˆ n1 b0  b1x n1 Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-51 Predictions Using Regression Analysis Predict the price for a house with 2000 square feet: house price  98.25  0.1098 (sq.ft.)  98.25  0.1098(200 0)  317.85 The predicted price for a house with 2000 square feet is 317.85($1,000s) = $317,850 Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-52 Relevant Data Range  When using a regression model for prediction, only predict within the relevant range of data Relevant data range Risky to try to extrapolate far beyond the range of observed X’s Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-53 Estimating Mean Values and Predicting Individual Values Goal: Form intervals around y to express uncertainty about the value of y for a given xi Confidence Interval for the expected value of y, given xi Y  y  y = b0+b1xi Prediction Interval for an single observed y, given xi Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall xi X Ch 11-54 Confidence Interval for the Average Y, Given X Confidence interval estimate for the expected value of y given a particular xi Confidence interval for E(Yn1 | Xn1 ) : yˆ n1 t n 2,α/2se  (x n1  x)2    2  n  (x i  x)  Notice that the formula involves the term (x n1  x) so the size of interval varies according to the distance xn+1 is from the mean, x Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-55 Prediction Interval for an Individual Y, Given X Confidence interval estimate for an actual observed value of y given a particular xi Confidence interval for yˆ n1 : yˆ n1 t n 2,α/2 se  (x n1  x)2  1  2  n  (x i  x)  This extra term adds to the interval width to reflect the added uncertainty for an individual case Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-56 Estimation of Mean Values: Example Confidence Interval Estimate for E(Yn+1|Xn+1) Find the 95% confidence interval for the mean price of 2,000 square-foot houses  Predicted Price yi = 317.85 ($1,000s) yˆ n1 t n-2,α/2 se (x i  x)2  317.85 37.12 n  (xi  x) The confidence interval endpoints are 280.66 and 354.90, or from $280,660 to $354,900 Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-57 Estimation of Individual Values: Example  Confidence Interval Estimate for yn+1 Find the 95% confidence interval for an individual house with 2,000 square feet  Predicted Price yi = 317.85 ($1,000s) yˆ n1 t n-1,α/2se (Xi  X)2 1  317.85 102.28 n  (Xi  X) The confidence interval endpoints are 215.50 and 420.07, or from $215,500 to $420,070 Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-58 11.7  Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables  Correlation is only concerned with strength of the relationship  No causal effect is implied with correlation  Correlation was first presented in Chapter Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-59 Correlation Analysis  The population correlation coefficient is denoted ρ (the Greek letter rho)  The sample correlation coefficient is r s xy sxsy where s xy (x  x)(y  y)   Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall i i n Ch 11-60 Hypothesis Test for Correlation  To test the null hypothesis of no linear association, H0 : ρ 0 the test statistic follows the Student’s t distribution with (n – ) degrees of freedom: t r (n  2) (1 r ) Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-61 Decision Rules Hypothesis Test for Correlation Lower-tail test: Upper-tail test: Two-tail test: H0: ρ  H1: ρ < H0: ρ ≤ H1: ρ > H0: ρ = H1: ρ ≠   -t t Reject H0 if t < -tn-2,  Where t  Reject H0 if t > tn-2,  r (n  2) (1 r ) Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall  /2 -t/2  /2 t/2 Reject H0 if t < -tn-2,    or t > tn-2,   has n - d.f Ch 11-62 11.9 Graphical Analysis  The linear regression model is based on minimizing the sum of squared errors  If outliers exist, their potentially large squared errors may have a strong influence on the fitted regression line  Be sure to examine your data graphically for outliers and extreme points  Decide, based on your model and logic, whether the extreme points should remain or be removed Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-63 Chapter Summary  Introduced the linear regression model  Reviewed correlation and the assumptions of linear regression  Discussed estimating the simple linear regression coefficients  Described measures of variation  Described inference about the slope  Addressed estimation of mean values and prediction of individual values Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11-64 ... Publishing as Prentice Hall Ch 11- 2 Chapter Goals (continued) After completing this chapter, you should be able to:  Calculate and interpret confidence intervals for the regression coefficients... intervals for the regression coefficients  Use a regression equation for prediction  Form forecast intervals around an estimated Y value for a given X  Use graphical analysis to recognize potential... correlation coefficient and perform a hypothesis test for zero population correlation Copyright © 2010 Pearson Education, Inc Publishing as Prentice Hall Ch 11- 3 11. 1  Overview of Linear Models