Week 6 Practice Problems

Author
Affiliation

Alex Kaizer

University of Colorado-Anschutz Medical Campus

This page includes optional practice problems, many of which are structured to assist you on the homework with Solutions provided on a separate page. Data sets, if needed, are provided on the BIOS 6618 Canvas page for students registered for the course.

This week’s extra practice exercise focuses on showing the total sums of squares is indeed equal to the model sums of squares plus the error sums of squares.

Exercise 1: Deriving the Sums of Squares

In our lecture slides we noted that SStotal = SSmodel + SSerror. For this problem, you will work through the math and perform some algebraic acrobatics to “prove” this is true.

Consider \(\hat{Y}_{i} = \hat{\beta}_{0} + \hat{\beta}_{1}X_{i}\). Using material from the lecture slides, we can rewrite this in terms of our residual (\(\hat{e}_{i} = Y_{i} - \hat{Y}_{i}\)):

\(\begin{aligned} \frac{\partial}{\partial \beta_{0}} SS_{Error} =& \frac{\partial}{\partial \beta_{0}} \left( \sum_{i=1}^{n} (Y_{i} - \beta_{0} - \beta_{1}X_{i})^2 \right) = 0 \to \sum_{i=1}^{n} -2 (Y_{i} - \hat{\beta}_{0} - \hat{\beta}_{1}X_{i}) = -2 \sum_{i=1}^{n} \hat{e}_{i} = 0 \\ \frac{\partial}{\partial \beta_{1}} SS_{Error} =& \frac{\partial}{\partial \beta_{1}} \left( \sum_{i=1}^{n} (Y_{i} - \beta_{0} - \beta_{1}X_{i})^2 \right) = 0 \to \sum_{i=1}^{n} -2 X_{i} (Y_{i} - \hat{\beta}_{0} - \hat{\beta}_{1}X_{i}) = -2 \sum_{i=1}^{n} X_{i} \hat{e}_{i} = 0 \end{aligned}\)

Additionally, we can use the following properties/definitions and hint:

  • Note 1: \(\sum_{i=1}^{n} \hat{e}_{i} = \sum_{i=1}^{n} (Y_{i} - \hat{Y}_{i}) = 0\)
  • Note 2: \(\sum_{i=1}^{n} X_{i} \hat{e}_{i} = 0\)
  • Note 3: \(SS_{Error} = \sum_{i=1}^{n} (Y_{i} - \hat{Y}_{i})^2\)
  • Note 4: \(SS_{Model} = \sum_{i=1}^{n} (\hat{Y}_{i} - \bar{Y})^2\)
  • Hint:

\[\begin{align} SS_{Total} =& \sum_{i=1}^{n} (Y_{i} - \bar{Y})^2 \\ =& \sum_{i=1}^{n} \left( (Y_{i} - \hat{Y}_{i}) + (\hat{Y}_{i} - \bar{Y}) \right)^2 \\ =& \sum_{i=1}^{n} (Y_{i} - \hat{Y}_{i})^2 + \sum_{i=1}^{n} (\hat{Y}_{i} - \bar{Y})^2 + 2 \sum_{i=1}^{n} (Y_{i} - \hat{Y}_{i})(\hat{Y}_{i} - \bar{Y}) \end{align}\]