WPA Q & A

tangotiger · Post by **tangotiger** » June 7 07, 1:27 pm

Someone asked me to post my thoughts on WPA, but I see this thread has 12 pages. Perhaps someone can summarize the various views on WPA in a post here, and I can add whatever I can to that...

New Pagodi · Post by **New Pagodi** » June 7 07, 1:41 pm

I guess the primary question about WPA is for players over the long run does it correlate well with run scoring or something else that would lead to wins, or is it just random and without much meaning.

And on a related note, does WPA have any significant meaning for pitchers over the long run since it seems to overrate relievers.

tangotiger · Post by **tangotiger** » June 7 07, 1:55 pm

New Pagodi wrote:I guess the primary question about WPA is for players over the long run does it correlate well with run scoring or something else that would lead to wins, or is it just random and without much meaning.

And on a related note, does WPA have any significant meaning for pitchers over the long run since it seems to overrate relievers.

The first thing to remember is why does WPA exist. It exists to give you a snapshot of the game. That is, at this point in time,
- what are the chances of winning
- after this at bat, what are the chances of winning

You capture the difference, and associate it to the batter/runner/pitcher.

Now, the next temptation is to aggregate this over a number of games, or season. But, that's not really why it exists. That is a byproduct.

Even so, let's say we do want to aggregate it, does it tell us anything. If you look at this thread:
http://www.fangraphs.com/blogs/index.ph ... r-batters/

You will see that WPA is NOT as predictive as OBP, SLG, or OPS. But, it's still fairly strong.

However, if you look at this post:
http://www.fangraphs.com/blogs/index.ph ... ment-10954

You will see that WPA divided by LI is as predictive as all the major ones.

***

In the long run, say several years, WPA will converge to the major numbers (OPS, RC, LWTS). So, it definitely means something.

WPA applies as well to pitchers as hitters.

As for "overrating" relievers, it does no such thing, any more than an average reliever having a better ERA than an average starter overrates relievers.

It simply is what it is. What you need to do is interpret it on that basis, that relievers need a different baseline, not only for WPA, but for ERA, K per IP, etc.

Fat Strat · Post by **Fat Strat** » June 7 07, 2:26 pm

Hmmm.

I'm going to have to think about this and the appropriate way to use it.

After all, Pujols had a WPA of 1.36, an OPS of 895

Polanco has a WPA of 2.05, an OPS of 806.

What does that actually tell me? That you're more likely to win with Polanco, and that's somewhat predictive?

tangotiger · Post by **tangotiger** » June 7 07, 2:33 pm

No, it means that you're showing me something with 250 PA, and that this means very little. You used the name "Pujols", which means we know alot more than what your two data points are showing.

haltz · Post by **haltz** » June 7 07, 3:02 pm

I guess after a bunch of pages in the other thread my question would boil down to, this. If you were to try and compare two players' to-date production, are you better off using an offensive production metric like VORP or runs created or whatever, or are you better off just trusting the +/- WPA number you have at the moment?

I guess a common "flaw" cited with WPA is that it can only tell you the LI at that moment by using historical WE base-state averages, or whatever. That if you were to go back through the game that something that gained +.10 WPA was really the turning point. But, WPA can't account for that, it can only account for the clutch of that situation, it can't look any farther into the game than the hitter, let's say. So it's useful in that sense, but it may not weight a players win contribution correctly in hindsight. I think I might be saying that awkwardly. Or even more to that end, I guess that some would say that once the game is over, all five of the runs from the 5-4 winning team were equally important.

I may not have done any better on this post I made in the Cust thread, but I think I was trying to get at the same idea there as well.

jim · Post by **jim** » June 7 07, 3:02 pm

At what point can you begin to maybe think that a particularly high (or low) WPA might point to some other skill like clutchiness? How big does the sample size have to be?

tangotiger · Post by **tangotiger** » June 7 07, 3:27 pm

haltz wrote:I guess after a bunch of pages in the other thread my question would boil down to, this. If you were to try and compare two players' to-date production, are you better off using an offensive production metric like VORP or runs created or whatever, or are you better off just trusting the +/- WPA number you have at the moment?

Let me ask you the question:
1. If Albert Pujols gets a few walk offs, where the Cards were down to their last out, and he pulled one out of his back pocket, turning a sure loss into a real win, is that impressive?

2. If Mariano Rivera comes in with the bases loaded, 0 outs, up by 1 in the 9th inning, and strikes out the side, is that impressive?

What WPA does is measure that. It takes the pulse of the game before the batter comes to bat, and then after, and says "here you go, this is what happened! This is why Scutaro gets a huge WPA gain and Mariano must get the exact opposite WPA loss". If you look at it from both perspectives (hitter, pitcher) at the same time, you are forced to conclude that not all runs are created equal, without the benefit of hindsight. That the evaluation of the game, of how you feel, exists in real-time, and not in a "if I had known". If you knew Mariano was going to give up that HR to Scutaro, why would you even bother going to the game?

In fact, if Vlad hits a grand slam against the Redsox in the playoffs to send the game into extra innings, only for the Angels to have subsequently lost the game, he may have as well struck out, since it's the exact same results.

tangotiger · Post by **tangotiger** » June 7 07, 3:42 pm

jim wrote:At what point can you begin to maybe think that a particularly high (or low) WPA might point to some other skill like clutchiness? How big does the sample size have to be?

The base comparison is the sum of each WPA/LI (that is WPA divided by LI on a PA-by-PA basis). So, the question is: what does the sum(WPA) give you over the sum(WPA/LI)?

Let's look at the best hitter in the league:
http://www.fangraphs.com/statss.aspx?pl ... osition=1B

In 2006, his WPA/LI is +6.1 wins. In 2005, his WPA/LI is also +6.1 wins. This measure is purely a reflection of his hitting skill with respect to the game situation, but without the extra leverage. So, a HR in the 9th inning or 1st will still be worth around +.12 wins. (It changes a little, but not enough to concern us at the moment.)

Similarly, you can use OPSwins as:
wins = PA * .025 * (1.7*OBP+SLG-1)
to get to the same results.
In 2006, Pujols would be:
634 * .025 * (1.7*.431+.671-1) = +6.4 wins
And in 2005:
700 * .025 * (1.7*.430+.609-1) = +6.0 wins

So, that's what he is, a +6.0 to +6.4 wins hitter.

In 2006 however, he put on one of the best clutch hitting performances you will ever see. His WPA in 2006 was +9.6 wins, giving him a clutch impact (though not clutch skill) of +3.5 wins. It is a fantastically high number.

If you sort by LI:
http://www.fangraphs.com/statsp.aspx?pl ... eason=2006

You will see that in high-leverage situations, he was unbelievable. In the 10 most important at bats last year (other than the IBB), he made an out only TWO times! That is clutch.

However, in 2005, in the 10 most important at bats, he made an out EIGHT times. That's not so clutch.

So, that's what WPA captures. Pujols was lucky in 2006... not that he was lucky to hit HR, but lucky that he TIMED the HR when he did. What you have here is a great hitter, who was great in 2005 and great in 2006, and yet, somehow, was better able to time that greatness in 2006 when the Cards needed him the most.

While clutch hitting does exist, the amount of PA you need to even determine that someone is a clutch hitter is huge, on the order of several thousand (a career's worth). At that point, who cares, other than for retrospective thought.

You certainly wouldn't base your decision on whether to bring in a certain player based on his "clutch stats". You *could* bring someone in if you believe in something beyond the numbers. But, the numbers aren't going to help you out.

haltz · Post by **haltz** » June 7 07, 4:09 pm

tangotiger wrote:What WPA does is measure that. It takes the pulse of the game before the batter comes to bat, and then after, and says "here you go, this is what happened! This is why Scutaro gets a huge WPA gain and Mariano must get the exact opposite WPA loss". If you look at it from both perspectives (hitter, pitcher) at the same time, you are forced to conclude that not all runs are created equal, without the benefit of hindsight. That the evaluation of the game, of how you feel, exists in real-time, and not in a "if I had known". If you knew Mariano was going to give up that HR to Scutaro, why would you even bother going to the game?

This is why I really do like the stat. I follow the games at fangraphs more often than not, and for me it's a fascinating way of judging reliever performance and usage as well as clutch performance.

There was really no debate until I questioned that once one does have the benefit of hindsight, does that change the numbers such that, we might be better off just comparing a couple of second baseman by their runs created numbers in a small sample.

I gave the example of the guy who hurts himself in WPA by hitting two homers in one game instead of one, and even wondered aloud whether those guys aren't getting shortchanged that offensively, through their own earlier run contributions, put Pujols in that position to hit those walk-offs.

And yeah, I couldn't be more impressed than when Mo or Albert does either one of those things. Those are obviously the most important moments in the game, but once the book closes on an offense all five of those runs I previously used as an example are in a certain way (to my mind), just as important.

I brought up in the other thread that I wasn't really comfortable using a small-sample WPA to compare two hitters, unless they were having a clutch result above or beyond their own relative OPSs.

In fact, if Vlad hits a grand slam against the Redsox in the playoffs to send the game into extra innings, only for the Angels to have subsequently lost the game, he may have as well struck out, since it's the exact same results.

An extra innings-sending grand slam would make him a WPA star that game, no? Sorry if I'm being dense.

I'm just kind of repeating myself a lot here, I probably should've gotten more out of your points/questions but I'm still not quite seeing it. I want to though.