Đang tải... (xem toàn văn)
In this chapter, students will be able to understand: The least squares estimators as random variables, the sampling properties of the least squares estimators, the Gauss-Markov theorem, the probability distribution of the least squares estimators, estimating the variance of the error term.
Chapter Properties of the Least Squares Estimators Assumptions of the Simple Linear Regression Model SR1 yt = β + β xt + et SR2 E(et) = ⇔ E[yt] = β1 + β2xt SR3 var(et) = σ2 = var(yt) SR4 cov(ei, ej) = cov(yi, yj) = SR5 xt is not random and takes at least two values SR6 et ~ N(0, σ2) ⇔ yt ~ N[(β1 + β2xt), σ2] (optional) Slide Undergraduate Econometrics, 2nd Edition –Chapter 4.1 The Least Squares Estimators as Random Variables To repeat an important passage from Chapter 3, when the formulas for b1 and b2, given in Equation (3.3.8), are taken to be rules that are used whatever the sample data turn out to be, then b1 and b2 are random variables since their values depend on the random variable y whose values are not known until the sample is collected In this context we call b1 and b2 the least squares estimators When actual sample values, numbers, are substituted into the formulas, we obtain numbers that are values of random variables In this context we call b1 and b2 the least squares estimates Slide Undergraduate Econometrics, 2nd Edition –Chapter 4.2 The Sampling Properties of the Least Squares Estimators The means (expected values) and variances of random variables provide information about the location and spread of their probability distributions (see Chapter 2.3) As such, the means and variances of b1 and b2 provide information about the range of values that b1 and b2 are likely to take Knowing this range is important, because our objective is to obtain estimates that are close to the true parameter values Since b1 and b2 are random variables, they may have covariance, and this we will determine as well These “predata” characteristics of b1 and b2 are called sampling properties, because the randomness of the estimators is brought on by sampling from a population Slide Undergraduate Econometrics, 2nd Edition –Chapter 4.2.1 The Expected Values of b1 and b2 • The least squares estimator b2 of the slope parameter β2, based on a sample of T observations, is b2 = T ∑ xt yt − ∑ xt ∑ yt T ∑ x − ( ∑ xt ) t (3.3.8a) • The least squares estimator b1 of the intercept parameter β1 is b1 = y − b2 x (3.3.8b) where y = ∑ yt / T and x = ∑ xt / T are the sample means of the observations on y and x, respectively Slide Undergraduate Econometrics, 2nd Edition –Chapter • We begin by rewriting the formula in Equation (3.3.8a) into the following one that is more convenient for theoretical purposes: b2 = β2 + ∑ wt et (4.2.1) where wt is a constant (non-random) given by wt = xt − x ∑ ( xt − x )2 (4.2.2) Since wt is a constant, depending only on the values of xt, we can find the expected value of b2 using the fact that the expected value of a sum is the sum of the expected values (see Chapter 2.5.1): Slide Undergraduate Econometrics, 2nd Edition –Chapter E (b2 ) = E ( β2 + ∑ wt et ) = E (β2 ) + ∑ E ( wt et ) (4.2.3) = β2 + ∑ wt E (et ) = β2 [since E (et ) = 0] When the expected value of any estimator of a parameter equals the true parameter value, then that estimator is unbiased Since E(b2) = β2, the least squares estimator b2 is an unbiased estimator of β2 If many samples of size T are collected, and the formula (3.3.8a) for b2 is used to estimate β2, then the average value of the estimates b2 obtained from all those samples will be β2, if the statistical model assumptions are correct • However, if the assumptions we have made are not correct, then the least squares estimator may not be unbiased In Equation (4.2.3) note in particular the role of the assumptions SR1 and SR2 The assumption that E(et) = 0, for each and every t, makes Slide Undergraduate Econometrics, 2nd Edition –Chapter ∑ wt E(et ) = and E(b2) = β2 If E(et) ≠ 0, then E(b2) ≠ β2 Recall that et contains, among other things, factors affecting yt that are omitted from the economic model If we have omitted anything that is important, then we would expect that E(et) ≠ and E(b2) ≠ β2 Thus, having an econometric model that is correctly specified, in the sense that it includes all relevant explanatory variables, is a must in order for the least squares estimators to be unbiased • The unbiasedness of the estimator b2 is an important sampling property When sampling repeatedly from a population, the least squares estimator is “correct,” on average, and this is one desirable property of an estimator This statistical property by itself does not mean that b2 is a good estimator of β2, but it is part of the story The unbiasedness property depends on having many samples of data from the same population The fact that b2 is unbiased does not imply anything about what might happen in just one sample An individual estimate (number) b2 may be near to, or far from β2 Since β2 is never known, we will never know, given one sample, whether our Slide Undergraduate Econometrics, 2nd Edition –Chapter estimate is “close” to β2 or not The least squares estimator b1 of β1 is also an unbiased estimator, and E(b1) = β1 4.2.1a The Repeated Sampling Context • To illustrate unbiased estimation in a slightly different way, we present in Table 4.1 least squares estimates of the food expenditure model from 10 random samples of size T = 40 from the same population Note the variability of the least squares parameter estimates from sample to sample This sampling variation is due to the simple fact that we obtained 40 different households in each sample, and their weekly food expenditure varies randomly Slide Undergraduate Econometrics, 2nd Edition –Chapter Table 4.1 Least Squares Estimates from 10 Random Samples of size T=40 n b1 b2 51.1314 0.1442 61.2045 0.1286 40.7882 0.1417 80.1396 0.0886 31.0110 0.1669 54.3099 0.1086 69.6749 0.1003 71.1541 0.1009 18.8290 0.1758 10 36.1433 0.1626 • The property of unbiasedness is about the average values of b1 and b2 if many samples of the same size are drawn from the same population The average value of b1 in these 10 samples is b1 = 51.43859 The average value of b2 is b2 = 0.13182 If we took the averages of estimates from many samples, these averages would approach the true Slide Undergraduate Econometrics, 2nd Edition –Chapter parameter values β1 and β2 Unbiasedness does not say that an estimate from any one sample is close to the true parameter value, and thus we can not say that an estimate is unbiased We can say that the least squares estimation procedure (or the least squares estimator) is unbiased 4.2.1b Derivation of Equation 4.2.1 • In this section we show that Equation (4.2.1) is correct The first step in the conversion of the formula for b2 into Equation (4.2.1) is to use some tricks involving summation signs The first useful fact is that − 2x ∑ xt + T x = ∑ xt2 − x T ∑ xt + T x T = ∑ xt2 − 2T x + T x = ∑ xt2 − T x ∑ (x − x ) = ∑ x t t (4.2.4a) 10 Slide Undergraduate Econometrics, 2nd Edition –Chapter The formula in Equation (4.5.2) is unfortunately of no use, since the random error et are unobservable! • While the random errors themselves are unknown, we have an analogue to them, namely, the least squares residuals Recall that the random errors are e t = y t – β1 – β x t and the least squares residuals are obtained by replacing the unknown parameters by their least squares estimators, eˆt = yt − b1 − b2 xt • It seems reasonable to replace the random errors et in Equation (4.5.2) by their analogues, the least squares residuals, to obtain 40 Slide Undergraduate Econometrics, 2nd Edition –Chapter σˆ ∑ eˆ = t T (4.5.3) • Unfortunately, the estimator in Equation (4.5.3) is a biased estimator of σ2 Happily, there is a simple modification that produces an unbiased estimator, and that is σˆ ∑ eˆ = t T −2 (4.5.4) The “2” that is subtracted in the denominator is the number of regression parameters (β1, β2) in the model The reason that the sum of the squared residuals is divided by T − is that while there are T data points or observations, the estimation of the intercept and slope puts two constraints on the data This leaves T − unconstrained 41 Slide Undergraduate Econometrics, 2nd Edition –Chapter observations with which to estimate the residual variance This subtraction makes the estimator σˆ unbiased, so that E (σˆ ) = σ (4.5.5) Consequently, before the data are obtained, we have an unbiased estimation procedure for the variance of the error term, σ2, at our disposal 4.5.1 Estimating the Variances and Covariances of the Least Squares Estimators • Replace the unknown error variance σ2 in Equation (4.2.10) by its estimator to obtain: 42 Slide Undergraduate Econometrics, 2nd Edition –Chapter xt2 ∑ ? b1 ) = σˆ var( , 2 T ∑ ( xt − x ) σˆ ? b2 ) = var( , ∑ ( xt − x ) se(b1 ) = var(b1 ) se(b2 ) = var(b2 ) (4.5.6) −x ˆ b1 , b2 ) = σˆ cov( 2 ∑ ( xt − x ) Therefore, having an unbiased estimator of the error variance, we can estimate the variances of the least squares estimators b1 and b2, and the covariance between them The square roots of the estimated variances, se(b1) and se(b2), are the standard errors of b1 and b2 43 Slide Undergraduate Econometrics, 2nd Edition –Chapter 4.5.2 The Estimated Variances and Covariances for the Food Expenditure Example • The least squares estimates of the parameters in the food expenditure model are given in Chapter 3.3.2 In order to estimate the variance and covariance of the least squares estimators, we must compute the least squares residuals and calculate the estimate of the error variance in Equation (4.6.4) In Table 4.2 are the least squares residuals for the first five households in Table 3.1 Table 4.2 Least Squares Residuals for Food Expenditure Data yˆ = b1 + b2 x e? = y − y y 52.25 58.32 81.79 119.90 125.80 73.9045 84.7834 95.2902 100.7424 102.7181 −21.6545 −26.4634 −13.5002 19.1576 23.0819 44 Slide Undergraduate Econometrics, 2nd Edition –Chapter • Using the residuals for all T = 40 observations, we estimate the error variance to be σˆ ∑ eˆ = t T −2 = 54311.3315 = 1429.2456 38 The estimated variances, covariances and corresponding standard errors are x 21020623 ∑ t ˆ ˆ = var(b1 ) = σ 1429.2456 40(1532463) = 490.1200 − T x x ( ) ∑ t ˆ b1 ) = 490.1200 = 22.1387 se(b1 ) = var( σˆ 1429.2456 ˆ b2 ) = = = 0.0009326 var( − x x ( ) 1532463 ∑ t 45 Slide Undergraduate Econometrics, 2nd Edition –Chapter ˆ b2 ) = 0.0009326 = 0.0305 se(b2 ) = var( −x −698 ˆ = cov(b1 , b2 ) = σˆ 1429.2456 1532463 = −0.6510 x x − ( ) ∑ t 46 Slide Undergraduate Econometrics, 2nd Edition –Chapter 4.5.3 Sample Computer Output Dependent Variable: FOODEXP Method: Least Squares Sample: 40 Included observations: 40 Variable Coefficient Std Error t-Statistic Prob C 40.76756 22.13865 1.841465 0.0734 INCOME 0.128289 0.030539 4.200777 0.0002 R-squared 0.317118 Mean dependent var 130.3130 Adjusted R-squared 0.299148 S.D dependent var 45.15857 S.E of regression 37.80536 Akaike info criterion 10.15149 Sum squared resid 54311.33 Schwarz criterion 10.23593 F-statistic 17.64653 Prob(F-statistic) 0.000155 Log likelihood Durbin-Watson stat -201.0297 2.370373 Table 4.3 EViews Regression Output 47 Slide Undergraduate Econometrics, 2nd Edition –Chapter Dependent Variable: FOODEXP Analysis of Variance Source DF Sum of Squares Mean Square Model Error C Total 38 39 25221.22299 54311.33145 79532.55444 25221.22299 1429.24556 Root MSE Dep Mean C.V 37.80536 130.31300 29.01120 R-square Adj R-sq F Value Prob>F 17.647 0.0002 0.3171 0.2991 Parameter Estimates Variable DF Parameter Estimate Standard Error T for H0: Parameter=0 Prob > |T| INTERCEP INCOME 1 40.767556 0.128289 22.13865442 0.03053925 1.841 4.201 0.0734 0.0002 Table 4.4 SAS Regression Output 48 Slide Undergraduate Econometrics, 2nd Edition –Chapter VARIANCE OF THE ESTIMATE-SIGMA**2 = 1429.2 VARIABLE NAME X CONSTANT ESTIMATED STANDARD COEFFICIENT ERROR 0.12829 0.3054E-01 40.768 22.14 Table 4.5 SHAZAM Regression Output Covariance of Estimates COVB INTERCEP X INTERCEP X 490.12001955 -0.650986935 -0.650986935 0.000932646 Table 4.6 SAS Estimated Covariance Array Appendix 49 Slide Undergraduate Econometrics, 2nd Edition –Chapter Deriving Equation (4.5.4) eˆt = yt − b1 − b2 xt = (β1 + β2 xt + et ) − b1 − b2 xt = −(b1 − β1) − (b2 − β2 ) xt + et = (b2 − β2 ) x − e − (b2 − β2 ) xt + et (see derivation of var(b1)) = −(b2 − β2 )( xt − x) + (et − e) eˆt2 = (b2 − β2 )2 ( xt − x)2 + (et − e)2 − 2(b2 − β2 )( xt − x)(et − e) = (b2 − β2 )2 ( xt − x)2 + (et − e)2 − 2(b2 − β2 )( xt − x)et + 2(b2 − β2 )( xt − x)e ∑ eˆt2 = (b2 − β2 )2 ∑ ( xt − x)2 + ∑ (et − e)2 − 2(b2 − β2 )∑ ( xt − x)et + 2(b2 − β2 )∑ ( xt − x)e = (b2 − β2 )2 ∑ ( xt − x)2 + ∑ (et − e)2 − 2(b2 − β2 )∑ ( xt − x)et (Q∑ ( xt − x)et = 0) 50 Slide Undergraduate Econometrics, 2nd Edition –Chapter Note: b2 − β2 = ∑ wt et = ∑ ( xt − x)et ∑ ( xt − x)2 (from Equations (4.2.1) and (4.2.2) ⇒ ∑ ( xt − x)et = (b2 − β2 )∑ ( xt − x)2 Note: ∑ (et − e)2 = ∑ (et2 − 2et e + e ) = ∑ et2 − 2e∑ et + Te2 = ∑ et2 − 2( ∑ et )(∑ et ) + T ( ∑ et )2 T T = ∑ et2 − (∑ et )2 + (∑ et )2 = ∑ et2 − (∑ et )2 T T T = ∑ et2 − (∑ et2 + 2∑ eie j ) = T −1∑ et2 − ∑ eie j T T T i≠ j i≠ j 51 Slide Undergraduate Econometrics, 2nd Edition –Chapter Now, ∑ eˆt2 = −(b2 − β2 )2 ∑ ( xt − x)2 + ∑ (et − e)2 = −(b2 − β2 )2 ∑ ( xt − x)2 + T −1 ∑ et2 − ∑ eie j T T i≠i E[∑ eˆt2 ] = − E[(b2 − β2 )2 ]∑ ( xt − x)2 + T −1∑ E[et2 ] − ∑ E (eie j ) T T i≠i = − E[(b2 − β2 )2 ]∑ ( xt − x)2 + T −1 ∑ E[et2 ] (Q E (eie j ) = 0) T = − var(b2 )∑ ( xt − x)2 + T −1(Tσ2 ) T = −( σ2 )∑ ( xt − x)2 + (T −1)σ2 (from Equation (4.2.11)) ∑ ( xt − x) = −σ2 + (T −1)σ2 = (T − 2)σ2 52 Slide Undergraduate Econometrics, 2nd Edition –Chapter Finally, E[σ?2 ] = E[∑ et2 ] T −2 = (T − 2)σ2 = σ2 T −2 53 Slide Undergraduate Econometrics, 2nd Edition –Chapter Exercise 4.1 4.6 4.17 4.2 4.13 4.3 4.14 4.4 4.15 4.5 4.16 54 Slide Undergraduate Econometrics, 2nd Edition –Chapter ... values of the independent or explanatory variable x The more they are spread out, the larger the sum of squares The less they are spread out the smaller the sum of squares The larger the sum of squares, ... b2 are the ones to use The Gauss-Markov theorem applies to the least squares estimators It does not apply to the least squares estimates from a single sample Proof of the Gauss-Markov Theorem:... determining where the least squares line must fall, because they are more spread out along the x-axis The larger the sample size T the smaller the variances and covariance of the least squares estimators;