Course Outline

list High School / Algebra + Data Science (G)

Book
  • High School / Advanced Statistics and Data Science I (ABC)
  • High School / Statistics and Data Science I (AB)
  • High School / Statistics and Data Science II (XCD)
  • High School / Algebra + Data Science (G)
  • College / Introductory Statistics with R (ABC)
  • College / Advanced Statistics with R (ABCD)
  • College / Accelerated Statistics with R (XCD)
  • CKHub: Jupyter made easy

3.10 Proportional Reduction in Error (PRE)

Great models not only have the least amount of error left, they also reduce the most error. Thus, we will introduce one more measure of a model: Proportional Reduction in Error, or PRE. As implied by the name, PRE is a measure of the proportion of total error in an outcome variable that is explained (or reduced) by a statistical model.

To get started with PRE, we first need to calculate the total error in an outcome variable and then see how much our model reduced it. To calculate total error, we will go back to the concept of the empty model. As a model with no predictor variables, just an outcome variable, its SSE represents the total error that could possibly be explained by any model with a predictor variable.

A scatter plot of body_mass_kg predicted by flipper_length_m. A horizontal blue line is plotted on the graph as the empty model and runs through the mean of body mass. A few data points are highlighted to show their squared distance between them and the mean.

When we use the empty model, represented with a horizontal line, we make the same prediction for every penguin’s body mass. Regardless of their flipper length, our prediction is going to be the mean of body mass (4.207 kg). This isn’t a very good model, but the mean (4.207) turns out to be the best single number to use for our empty model. Any other number (e.g., 4.201 or 4.307) would result in a worse sum of squares. So the empty model’s sum of squares gets a special name: Sum of Squares Total (SST).

We can calculate SST using our sse() function:

SST <- sse(Y, mean(Y))

SST provides a way of measuring total error and SSE represents how much of that error is left when we use our best-fitting model.

SSE <- sse(Y, best_function(X))

On the left, a teal circle titled as the Empty Model, using the mean to predict body mass, and inside the circle is labeled as SST Sum of Squares Total. To the right, is another teal circle title as the Best-Fitting Model, using flipper length to predict body mass. A large part of the circle is shaded with lines to represent Error Reduced (SST minus SSE), and the remaining part of the circle is shaded in solid teal and labeled as SSE, error left after using best-fitting model.

PRE is a proportion. It tells us the proportion of total error in the outcome variable that was reduced by adding a predictor variable (flipper_length_m) to the model. The amount of error reduced by adding a predictor variable can be calculated as \(SST-SSE\). To calculate PRE, we divide the error reduced by the SST.

\[PRE = \frac{SST - SSE}{SST}\]

In the code window below, we’ve put the code for calculating SSE and SST. Use them to calculate PRE.

require(coursekata) require(Metrics) Y <- penguins$body_mass_kg X <- penguins$flipper_length_m # assume Y and X have been defined for you # this creates the best-fitting model best_function <- function(X){-5.872 + 50.153*X} # this calculate SSE and SST (SS from the empty model) SSE <- sse(Y, best_function(X)) SST <- sse(Y, mean(Y)) # use SSE and SST to calculate PRE # (hint: you'll need to add parentheses) # assume Y and X have been defined for you # this creates the best-fitting model best_function <- function(X){-5.872 + 50.153*X} # this calculate SSE and SST (SS from the empty model) SSE <- sse(Y, best_function(X)) SST <- sse(Y, mean(Y)) # use SSE and SST to calculate PRE # (hint: you'll need to add parentheses) (SST - SSE) / SST msg <- "Check your use of sse() or the <- operator" ex() %>% { check_object(., "SSE") %>% check_equal(incorrect_msg = msg) check_object(., "SST") %>% check_equal(incorrect_msg = msg) check_operator(., "-") %>% check_result() %>% check_equal() check_operator(., "/") %>% check_result() %>% check_equal() }
0.762092154937311

The PRE of 0.76 means that 76% of the total error in body mass is reduced (or explained) by our best_function model.

While the best-fitting model minimizes SSE, MSE, and RMSE, it maximizes PRE. There are no other values of \(b_0\) and \(b_1\) that lead to a better PRE than our best-fitting model.

Note: PRE goes by other names in other traditions. It is sometimes called \(R^2\) (pronounced R squared) and also called \(\eta^2\) (pronounced eta squared). Both terms refer to the same thing: PRE.

Responses

Responses
close