Penalties

Talking way too much about refs and some Seattle homecookin'

Disclaimer

The data used for this post has complications in it that were not known before the analysis was done, graphics were made, and the post was first published. Thanks to the AUDL stat team for the clarification, it’s possible almost all of the analysis is inaccurate to some extent.

The confusion/inaccuracies have to do with the way event data is recorded in the data base and the way in which each team’s stat keepers record individual penalties. Because of the inconsistencies, I will go through and re-do the analysis at a later date with those inconsistencies taken into account, but that likely will not be until the offseason.

You are still welcome to read the post, but keep in mind these errors, and I hope to have them fixed at some point!

---

For the average avid and cynical fan of basketball, one of the most exciting parts of post game festivities after a hotly contested and even controversial NBA game is over is the issuance of the Last Two Minute Report. In it, the league reviews and states which calls were made correctly and which calls were missed by the officials. Until then (but not necessarily ending with it), heated debate ensues over whether or not the defender was in "legal guarding position", as translated into every individual fan's "expert" opinion.

Once there's clarity on the Last Two Minute Report discussion usually winds down, but that doesn't stop the intrigue into what happened during the rest of the game. What about how Scott Foster reffed Chris Paul onto the bench in important game after important game? That doesn't show up in the L2M Report.

And so the brave analyst dives into the reffing data and begins to find out if there is any signal in referee bias to support some of these claims, but not without knowing that any future statement with Whistleblower Podcast-like findings will be looked at with severe skepticism.

Luckily (or unluckily?) for us, refereeing is no less controversial in Ultimate, even if for different reasons. My personal opinion is that it dramatically improves the consumption and enjoyment of a sport that can sometimes so painfully and unnecessarily drag on in the amateur divisions. But it's not without its setbacks. Refs in the AUDL can be downright bad for a myriad of reasons, the first of which being there are so few opportunities to ref Ultimate. And since practice makes perfect, we can kindly say that most AUDL refs don't have enough practice yet.

It's for exactly these reasons that it's mostly for fun that today's post came about. There are so many moving pieces in making calls on the field and no data on what those calls are that there isn't much to conclude. So, just purely for fun, let's swim through a sea of concocted graphics and tables.

---

Which games to watch and which to not

The benefit of refs, as I stated before, is that the pace of the game can move on rather quickly compared to club or college games. However, are there still games in which we end up watching more stoppages than we'd like?

The graph below shows the total number of penalties called in each game so far in 2023 plotted against the score differential for that game. The gold bar down the middle represents games that ended with a score differential within 3 points, typically games that are closer and more compelling (but not always). Overlapped with the gold band horizontally is a red area, a blue area, and a green area. The red area represents the top 10% of games with the most penalties, the green area contains games with the bottom 25% of penalties called, and the blue region is everything in between.

Specifically, the overlapping green/gold section would be games that move along rather nicely without hardly any interruption from the refs and are also typically more exciting because of how close the game is. The overlapping red/gold section are games that are close, but you're going to end up watching at least 10 stoppages per quarter! If you're looking to rewind and watch some AUDL footage for fun, the red zone outside of the gold band is not the place to go unless you love stoppages and blowouts. Below is a supplementary table to the plot delineating some of these games. They're color coded the same so they're easy to identify where on the plot they are represented.

Looking at this made me wonder: does there seem to be any obvious differences between the reffing in the different parts of the country? Well, good news for fans, there doesn't seem to be. The rain cloud plot below shows the distribution of games and the amount of penalties called by each divisional home game (This is assuming refs don't work games in multiple divisions regularly).

Maybe the refs in the South division tend to be more involved (or the game more physical) than the refs in the rest of the league? Overall though, there isn't a strong indication anything systemic is going on there.

---

Homecookin' and lopsided games

Just like Chris Paul's intuition that certain refs have it out for him, I have a hunch that my favorite team has gotten the short end of the stick every game I've attended, too! This table shows which teams have been paying off their refs and which have it out for their home team (jokes). The game by game column shows for each home game whether or not the team was granted more favorable calls by the ref than their opponent. Blue means more favorable calls for the home team and red represents more favorable calls made for the visiting team. The season percent column is an aggregated percentage of how many calls are going that team's way at home so far this season.

I don't know what you see, but I see some strong homecooking up in Seattle. On the other hand, I'd be curious to know in what way the Minnesota players/coaching staff have wronged their reffing crew.

Those little indicators in the game by game column of the table don't tell us how severe each game's bias was, though, so I found the top 5% most lopsided games in the AUDL this season. If DraftKings was still setting lines, we might wonder about some of these refs.

For all the whining on Twitter, it sure seems like the Phoenix didn't have the short end of the stick against the Glory this last week.

---

Conclusion

For as tongue-in-cheek as I've been so far, it is interesting to see how aggressive certain teams are on certain weeks. The plot below shows a team's number of infractions for each game so far this year. The red, blue, and green dots represent the same thresholds as before. Each of the games with the highest 10% of penalties are also labeled.

We can never be certain what portion of the penalty total for each game is attributed to aggressive play and how much is the referees that day, but the Spiders have had a 4 games where they conceded over 200 yards of offense due to penalties (as a side note, it will be interesting in the future to be able to evaluate how valuable 200+ yards of offense is over the course of a game; is it more or less than a point?).

The final graph below summarizes all of these games onto one plot for an average for each team. On the x axis are the penalties against each team and on the y axis are fouls a team drew each game.

The logos of the teams represent their season average per game. Below the diagonal line we could think of these teams as either disadvantaged by the refs on average or more aggressive in play style and more likely to foul. Teams of above the dashed line represent teams that the refs tend to favor or play naturally cleaner Ultimate. Or some unknown mix of the two.

---

Another Championship Weekend update

Hopefully you're not growing tired of the posting of the probabilities of making Championship Weekend every week, but I must for consistency's sake.

Every week it seems the model is getting more uncertain about the Central division.

---

감사합니다

What do you think, do the refs hate your favorite team?

Grateful for the available data @AUDLStats. You may leave comments or questions here, or you can reach out to me on Twitter @Comet_Miller. Thanks for reading!

— 밀혜성