Explore a model to look at predicted vs. actual target share

Estimated reading time: 7 minutes

Hello, everyone, and welcome to the first installment of my PFF Statistical Model series.

My name is Joseph Bryan, and I enjoy creating advanced statistical models for NFL analysis. Today, I am excited to present my latest project: “Coach, I was open!” — A model designed to predict target distribution in NFL games

an introduction

The idea for this model first came to me Eagles vs. Vikings matchup in Week 2 of the 2023 season. Devonta Smith Had a great game, finishing with a 75.4 PFF grade on four catches for 131 yards and five targets. his teammate, AJ Brown, Finished with a 72.6 PFF grade on four receptions for 29 yards and six targets — and he was Visibly disappointed on the sidelines.

That raises a key question: Why isn’t Brown, an elite talent, getting more opportunities? Interestingly, after just one week, he was targeted 13 times.

The goal is to have a way to measure these fluctuations, and that’s what led me to develop this model.

Model construction

Up to five players can run a route in any game, but only one will receive a goal. By examining key data points and characteristics of that game, we can predict the likelihood of each player being targeted.

Okay, so you’re looking to create a model. The first step is to identify the response variable, which is what we are primarily concerned with. In this case, it’s straightforward: the response variable is “target” — whether or not a player received a target in a given game. Simple as that.

Next, we need to make some predictions. In our scenario, we care about the following questions:

• What kind of route does the receiver run?
• How deep was the path?
• What was the distance up and down?
• How well do receivers run routes?
• What level of separation have they got?
• Were they open?

Each of these factors directly affects a player’s likelihood of being targeted. This will help our model determine what influences the decision to throw a particular player on a given play.

Next, we need to put all this data into a mathematical model. I chose to use one XGBOUST Models are everywhere because we have a lot of data to process and complex, non-linear relationships.

XGBOUST models are machine learning models that examine the relationship between our response variable (target) and our predictors (list above) and learn how our predictors lead to our response variable.

Is an open 10-yard slant on first down more likely to be a target than a 15-yard open post route on third down? These are the relationships we are trying to teach our model with our predictors

Computers then use us to perform many computational tasks Hyperparameters. I won’t get into the weeds, but the bottom line is that we want a strong R-squared value, meaning our model can accurately predict the outcome of new data.

For example, if we train the model using data from 2019 to 2022, we want it to predict well for 2023. Many models you see online don’t do this properly, so beware of them.

To access game time data and grades,
Sign up for a PFF+ membership today

Model results

Now, let’s dive into the results.

First, what factors predict goals on a play-by-play basis? Although the real value of the model is not here, it is important to present this basic insight.

The variable names are probably mostly meaningless to you, but we can elaborate on our top three predictions:

1. “final_rec_grade” refers to the PFF grade assigned to the route runner, which serves as a positive indicator for both our model and PFF’s evaluation system.

2. “route_name_group_Screen” indicates that the player runs a screen route. This is logical, as screens are usually designed for a specific player, so that they are more likely to get the target.

3. “route_name_group_Other” is a bit more unusual, as it’s a classification I made up. Essentially, it became a category for “wind sprints” or clearout routes that players ran. It is predictive as it highlights situations where players are less likely to score.

Now, let’s dive into the “Predicted Goals” results on a game-by-game basis using 2023 data. Remember, this data is new to the model, meaning it hasn’t been trained on this data, making predictions a true test of its accuracy.

This means our new metric, “Predicted Goals,” can explain 53% of the variance in actual goals.

So, what does this mean for you and me? Let’s go back to the beginning of the article and revisit the earlier idea: “Interestingly, after just one week, Brown was targeted 13 times.”

What if, instead of comparing “actual goals” with “predicted goals” we focused on the difference between the two and examined how many more goals a player scored in the next week compared to the current week?

Let’s introduce a new metric called “CIWO” (coach, I was open). CIWO represents the difference in the number of targets a player has hit next week compared to this week.

For example, if a player gets five targets one week and eight the next, their CIWO will be +3.

This graph may look complicated, but it’s actually straightforward.

What this shows is simple: when a player has more “predicted goals” than actual goals, they may see an increase in goals next week compared to this week.

That was our initial hypothesis and so great This relationship holds for each term.

This table shows that when a player’s “predicted goals” exceed their actual goals by three, they are expected to score an average of 2.57 more goals the following week.

FPWO is the same concept applied to fantasy points. A player with a -3 differential can expect a 3.69-point increase in fantasy points over the current week.

Astute observers note that “difference >= 3” has the highest average target for the following week. My theory on this is simple: teams apply force to their best players. When players meet this threshold, we still see an average of three fewer goals the following week.

Week 4 Predicted Goals Table

The moment you’ve all been waiting for.

How can we use the results of this model practically? Well, we can create some tables using our new metrics and some other variables that the nerds (me) care about.

Note: If the targeted player commits a penalty, the game is removed from the data. If the other player commits a penalty, the play is not moved.

Note: I would suspect players who are not elite and are on the red table. Elite players will get their targets whether they are unlocked or not.

One way to think about “predicted goals” is that they represent two things:

1. How good or open a player was during the week
2. How often a team shows up for a given player in the scheme

As we analyze this table, it critical Knowing that the more goals a player sees in Week N, the harder it is to stay more Aim for week N+1.

Players that caught my eye Chris Godwin, Justin Watson, Jordan Addison, Keenan Allen And Jameson Williams.

• Godwin is a huge target earner in a good offense, and Evans has a +4 differential. If he negatively reverts to actual goals, those goals may go Godwin’s way.

• Watson could see the route grow this week and there was a really big difference.

• Addison should run more routes in a good passing offense.

• Allen should run more routes this week.

• Williams is off this week but could look good next week.

Remember, when looking at the tables above, if the difference is less than -3, we can expect a 2.57 average increase in goals for those players. It’s not specific and specific to any individual player, but it’s something we can expect in a broad sense. Any given player can fail, but as a unit, they see more goals.

Ultimately, it’s just one big puzzle when it comes to predicting upcoming performance. We should not base all our decisions on a single table or model. We should use different sources to come to any good conclusion.

Follow Joseph at X at @Qualitative statistics.