Chapter 8 Multiple Linear Regression

Sometimes one independent variable just doesn’t cut it.

\[PRF:\;Y_i=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}+...+\beta_kX_{ki}+\varepsilon_i\]

\[SRF:\;Y_i=\hat{\beta}_0+\hat{\beta}_1X_{1i}+\hat{\beta}_2X_{2i}+...+\hat{\beta}_kX_{ki}+e_i\]

A Multiple Regression Model is a direct extension of the Simple Regression Model by adding additional independent variables. Adding additional independent variables allows the regression to use more information when trying to explain movements in the single dependent variable. In other words, multiple independent variables can explain changes in the dependent variable along different dimensions.

The multiple regression model has a lot in common with the simple regression model.

  1. It is still the case that we establish a population regression function (PRF) that we believe holds in the population, but we are forced to estimate a sample regression function (SRF) because we can only observe a sample (i.e., subset) of the population.

  2. The SRF is still solved via OLS. The first-order conditions are a bit more complicated than those stemming from a simple regression, but are conceptually the same.

  3. The PRF and SRF still contain a single intercept term and a single residual term each.

  4. The model we are examining is still a line equation - only it is a multi-dimensional line equation (i.e., a plane in the case of two dimensions, and something unimaginable in more than two dimensions).

The only slight change we need to make when moving from a simple to multiple regression model concerns the interpretation of the slope coefficients of our model. These slope coefficients still deliver the expected or average change in the dependent variable given a unit change in an independent variable. However, since we are looking at multiple independent variables simultaneously, we need to be explicit that we are examining these relationships one independent variable at a time. In other words, when we examine the relationship between the dependent variable and one particular independent variable, we need to explicitly state that we are holding all other independent variables constant.

\[\beta_k=\frac{\Delta Y}{\Delta X_{k}}\]

In the population: a PRF slope coefficient indicates the EXPECTED or AVERAGE change in the dependent variable associated with a one-unit increase in the kth explanatory variable holding all other explanatory variables constant.

\[\hat{\beta}_k=\frac{\Delta Y}{\Delta X_{k}}\]

In the sample: a SRF slope coefficient indicates the expected or average change in the dependent variable associated with a one-unit increase in the kth explanatory variable holding all other explanatory variables constant.