Starling Marte, Defense and the Limitations of WAR

As we get ready for the opening of the 2016 baseball season, I thought I'd share some thoughts on advance metrics and what the sabermetric community knows and what it is still trying to figure out through the lens of Starling Marte’s statistics. This isn’t exactly breaking new ground, but I still think Marte’s numbers specifically are worth looking at.

For years the Holy Grail of sabermetrics has been to find a singular number that could neatly summarize a player’s season and provide context for that season versus other seasons from the same player, different players and different eras. The number is designed to take into account every aspect of a player's performance--offense, defense/fielding and base running. The term commonly used for this is WAR or Wins Above Replacement.

As this excellent primer on WAR states, 

Talking effectively about WAR requires considerable time and effort, given how much work goes into computing this number, how many factors / metrics it takes into account, and the fact that there are several permutations of it available to the public. 

Please keep that in mind as we gloss over many of the details. Currently there are three main forms of WAR, all of which calculate the metric differently. The thing is, when we talk about the value of position players, we probably know 95% of what we will ever be able to ascertain from a batter’s contribution in the batter’s box. There are various offensive numbers such as Weighted Runs Created (wRC), On-base Plus Slugging Plus (OPS+) and True Average (TAv) which all do a good job of telling us a player’s offensive contribution in a single statistic by analyzing the counting and rate stats and using a formula to distill them down to a single number.

The problem is when it comes to defense (and a lesser degree base running), it’s hard to tell exactly what we know and how accurate the data is. We can see this clearly when taking a look at Marte’s numbers during his three full season in the majors. The table below compares fWAR, the version supplied by Fangraphs, bWAR, the version found on Baseball-Reference and WARP, the version provided by Baseball Prospectus.

Starling Marte	fWAR	bWAR	WARP
2013	4.8	5.4	2.7
2014	4.4	5.1	3.3
2015	3.6	5.4	2.5

Almost the entire difference in these three numbers each year comes from the different defensive components used by the three methods. bWAR, which clearly views Marte most-favorably in all three seasons, shows Marte leading all left fielders by a large margin in defensive runs saved in 2015, leading a to strong positive defensive contribution to his overall bWAR figure. fWAR looks at Marte’s 2015 defensive season favorably, but has a handful of players ranked ahead of him. WARP calculates Marte as having a negative Fielding Runs Above Average (FRAA) in 2015 (and in his other two seasons as well.). The reason for these discrepancies is they each use different methods to come up with their defensive valuations. They are all evaluating the same player, looking at the same plays, but coming up with surprisingly disparate results.

There are complex calculations that go into these numbers and this is probably an oversimplification even as clear as the differences are, but it illustrates how the different methods of evaluating defense can come up with dramatically different views on a player. And although it doesn't impact the analysis of Marte, there is a valid argument that the massive increase in the use of defensive shifting has actually made defensive metrics less reliable over the last five years. Hopefully the new data being recorded by Statcast and Trackman, which tracks exact player positioning and movement, will allow us to better analyze a player’s defensive capabilities going forward.

And there there is the question of the “eye test” and whether that is a reliable way to evaluate a player. The anecdote of watching a player play an entire season and knowing that if a .270 hitter gets one more hit every to weeks he becomes a .300 hitter is instructive. Could you tell the difference? Of course the answer for virtually all of us, unless you are recording the information, is no.

Having said that I believe evaluating a player’s defensive skill set is a bit different. I think that if you watch a player play 150 games in a season you will have a very good understanding of his defensive strengths and weakness. But translating that understanding into useful data and comparing it to players playing different positions is a whole different problem. Having watched Starling Marte throughout his career I can give you the scouting report: He has great range in left field. He tracks balls well but doesn’t catch everything he gets to. He occasionally misses the routine play (St. Louis Sept 2014 anyone?), but he has good instincts and a very strong accurate arm. #DontRunOnMarte. But can I give you a solid comparison between Marte and Alex Gordon or Christian Yelich? No. I do see Marte 150 times a year, but I only see the others maybe 10-15 times. That might be enough to provide a basic scouting report, but it isn’t enough of a sample size to have a deep understanding of the others’ strengths and weaknesses and thus I can’t come up with a valid quantitative comparison between the players. 

Back to WAR. In today’s world 1 WAR is valued at between $7-9 million. How do we put any kind of accurate value on Starling Marte’s worth when one method calculates his 2015 season at 2.5 WAR, one has him at 3.6 and the third has him at 5.4? Using $8M/WAR the range of value is $20-$43 million. Not very precise.

So remember as we go through the season analyzing players and situations, the data and on-going study of that data has revealed so much that has changed how we think about the game. Just looking at the numbers may tell us virtually everything about a player’s offensive performance. But we’ve got a ways to go to figure out the rest. And be careful when someone brings up WAR. It's a useful tool, but it's still pretty blunt.

Clint Hurdle, Sabermetrician

Back in December, on the day the Pirates announced the signing of John Jaso, I tweeted out a Pirates batting order that I thought would be optimal vs. right handed pitchers.

My #Pirates (needs more thought) lineup vs RHP
1) Jaso
2) Cutch
3) Marte
4) Kang
5) Polanco
6) Cervelli
7) Harrison
8) Mercer
I like it.

— David Todd (@DTonPirates) December 23, 2015

Now I figured this was something even the analytically-driven Pirates would never adopt. Jaso is not the prototypical speedy leadoff hitter, but his .361 career OBP and his excellent baserunning skills make him an ideal candidate to hit at the top of the order. It is also the lineup spot he has been penciled into most frequently when playing for analytically-astute teams like the Rays and the A's. But the Pirates had Josh Harrison and Gregory Polanco both hit at the top of the order last year and both were back this year. Habit is a strong force to overcome. Plugging McCutchen in the 2-hole was something that seemed even less likely than batting Jaso leadoff. McCutchen hadn't hit anywhere but third or fourth since 2011, but all of a sudden Cutch batting second has been the feature story out of Bradenton with Opening Day less than two weeks away.

It's been argued for years now that teams should bat their best hitter in the 2-hole, but very few teams have actually done it. The Reds did it last year with Joey Votto. We've seen the Angels do it with Mike Trout. Clint summed up why he had never done it when he presented the idea to McCutchen, "I told Andrew the challenge for me is for 47 years, the baddest dude hits third. If got to rearrange my thinking on it and what's best for our team. How do we maximize our run production?" Now it looks like the Pirates will follow the Reds lead. We can debate the impact of lineup optimization and whether it's worth all the time we spend discussing it, but for a team like the Pirates that has appeared in three straight wild card games, The Clint Hurdle Invitational as Joe Sheehan likes to call it, every extra run can make a difference.

All of which reminded me of a story, one of the very first times I interacted with Hurdle.

Coming off a 105-loss season the Pirates introduced Clint as their new manager in November 2010. Clint was not a favorite of the sabermetric community and his hiring certainly raised some eyebrows and "same old Pirates" reactions. Of course none of this bothered Hurdle. Before the season he was very visible promoting the organization and talking frequently about "rebranding a city with its baseball team," laying out his plan to once again make the Pirates a respected, winning franchise. "I'm proud to be a Pirate," he had boomed at his introductory presser, a phrase not often uttered during the team's 18-consecutive losing seasons.

That season, my second back in Pittsburgh, I was hired to be the Pirates pre- and post-game host, so I had some interaction with Clint early on. Listening to him talk about baseball in those first weeks, he would often answer a question about why he pursued this or that particular in-game strategy ending it with something like, "there is no Book. If there was a Book, we would all just go to the appropriate page and have all the answers." Clint said this frequently enough that I realized he didn't know there actually was "a Book," and ironically, it was called The Book: Playing the Percentages in Baseball, written by three well-respected baseball statisticians in 2007. It covered topics such as batter-pitcher matchups, platooning, sacrifice bunts, intentional walks, lineup optimization and a plethora of other situations and strategies that are regularly encountered in the game. The authors approached all these topics by analyzing reams of historical data and then presenting the optimal decisions for each situation based on that analysis. The underlying premise being that by optimizing process and decision making a manager/team will give itself a better chance of "winning" each decision and therefore winning more games.

One Sunday morning in May of that first season I brought my copy of The Book to the media's pregame meeting with Clint in his office. After the briefing, when everyone had cleared out, I approached Clint who was sitting behind his desk. For those who don't know him, here is what Pirates GM Neal Huntington said when he introduced Clint as manager, "He does have a great personality, he's got a big presence and he's very charismatic, which allows him to be a tremendous leader." If anything Neal undersold that aspect of him.

That Sunday morning Clint probably knew my name, but not much more. I walked over and re-introduced myself. He stood up and extended his hand. Clint is a big man. If his goal was ever to intimidate, just standing up would be a good start. So there I was about to hand a book about baseball to a man who had spent his entire life in the sport. More than that, I had printed out a couple of pages from The Book which discussed run expectancy and why sacrifice bunts were generally a bad idea and, by inference, why Clint using the sac bunt so much was a bad idea. I mean I'd been involved in the game as a broadcaster for a couple weeks now. Why shouldn't I tell the manager what I thought?

Clint had every right to tell me exactly where to stick my book and kick me out of his office. He did the opposite. He couldn't have been more gracious. He admitted he was unaware The Book existed. (How many would do even that?) And after I briefly described its contents he engaged me for five minutes, even discussing the run expectancy printouts and the theory behind it. Having overstayed my time, with him having a game to prepare for, I turned to walk out. Clint thanked me and said he looked forward to reading and discussing The Book.

Fast-forward to spring training 2016. Under the stewardship of Neal and Clint the Pirates are one of baseball's model-franchises. They are viewed as being on the cutting-edge of data analysis and are at the forefront of implementing change in the game, which is wonderfully-profiled by Travis Sawchick in his fantastic book Big Data Baseball. Each season the Pirates have tried to find advantages from pitching inside, to defensive shifting to player health management where they can find an edge on the competition.

This spring the Pirates have experimented with a more aggressive running game. Is that something that can be exploited now that the running game has been de-emphasized in today's lower run environment? The data have shown they need to have their outfielders playing shallower because fly balls from their primarily ground ball pitching staff have a shallower launch angle and too many balls were falling in for hits. And now they are looking to optimize their batting order.

The guy in charge of implementing these and all the other changes is a very different manager than the one the Pirates hired back in 2010. One that has committed himself to learning and adapting to changes in the game and then communicating that information to his coaches and players to get an organizational buy-in. The results have been staggering to many who think small-market teams can no longer compete. The Pirates have won 94, 88 and 98 games the past three seasons, one of only three teams to make the playoffs all three years. The guy in charge is still Clint Hurdle. But now it's Clint Hurdle, sabermetrician.

Gerrit Cole and a Brutal, Misfocused 72-Hour News Cycle for the Pirates

I think now everybody has taken their shot at the Pirates.

I'm sure you've seen the story. Late Saturday night it was reported the team had offered Gerrit Cole a paycut after a 2015 season where he finished fourth in the National League Cy Young Award voting. Less than 16 hours later the beat writer who broke the original story wrote another about how Pirates General Manager Neal Huntington admitted the Pirates made a mistake:

“We made a mistake in the process,” Huntington said. “We've owned that. We'll evolve. Our hope is that Gerrit is ready to move forward and put this behind him.”

Under baseball's rules, Gerrit Cole is an indentured servant, like every other player, until he accrues three years of service time. During those three years, as long as a MLB team offers a player at least the league minimum salary of $507,000, the player has no recourse. The Pirates made Cole and his agent Scott Boras a contract offer of $538,000. It turns out that number happened to be $3,000 less than the $541,000 Cole earned last year. Depending how you want to slice it Cole actually got a raise. His 2015 salary was $531,000 and he also received a $10,000 bonus for making the All-Star team. Assuming a similar bonus was in effect with the Pirates offer this season, there isn't much of story here. In any business, bonuses are different than base salaries. Raises come off the base salary. Cole got a raise, small as it is, from $531K to $538K. But he wasn't guaranteed to make more money than he did last year. And that set off the alarm bells for Scott Boras and got all of baseball focused on the Pirates in the lull before spring training games commenced.

Boras saw blood in the water and wasn't going to pass up the opportunity to turn the story into a national feeding-frenzy with the Pirates as the chum. Not surprisingly, he spoke up,

“When you have a system that doesn't account for special performances by a player, it doesn't reward those achievements,” Boras said.

A bit more surprising was the fact that Gerrit Cole commented as well,

“I understand the business of this game, but it is hard to accept that a year of performance success does not warrant an increase in pay,” Cole told the Tribune-Review.

I have no problem with Cole voicing his opinion. He is the new Pirates player rep having taken over for Neil Walker. Maybe both he and Boras thought this was a good way to make an impression on his teammates and the Players' Union about being a vocal contributor in his new role. Fine.

The issue I do have a problem with is Cole's revelation the Pirates actually "threatened" to cut his pay back to the minimum of $507K if he didn't sign the deal that was offered. Players actually have no obligation to sign the actual contract and more than one player has played out the season without signing, viewing that as their only available form of protest. But I don't even know where to go with the thought the Pirates actually played the real paycut-card. It's hard to believe anyone with any authority in the organization would suggest that to either Boras or Cole. If there is a story, this is it and it should be addressed.

But it's worth pointing out again, that the Pirates immediately admitted to making a mistake. The clerical process of submitting Cole's contract to his agent is about five pay-grades below Neal Huntington's duties. The team obviously wasn't factoring in bonuses to formulaic, rubber-stamped pay increases. Someone messed up and the Pirates are getting bombarded with bad PR. Buster Olney took his shot at the Bucs and their "penny-foolish" ways. Joe Posnanski dropped Cole's tale of woe into a larger story of today's player economics and my friend Dave Schoenfield, who I think missed the mark, compared the Cole situation to the Royals extension of catcher Salvador Perez.

I am the last to defend the Pirates and Bob Nutting's frugal ways. I have spent the last four months on my show on ESPN Pittsburgh (and the three years prior) ripping the team for their limited, self-imposed budget and a starting rotation that is to include Jeff Locke and more-shockingly, free agent Ryan Vogelsong to open the season. Tyler Glasnow, Jameson Taillon, Josh Bell, Elias Diaz and Alen Hanson and their corresponding minimum salaries of $507K are all expected to matriculate to the majors over the next 12-18 months. In addition, more than $25+ million in salaries will be coming off the books next year with Mark Melancon ($9.65M), Michael Morse (~$5M), Neftali Feliz ($3.9M),  Francisco Cervelli ($3.5M), Juan Nicasio ($3M) and Sean Rodriguez ($2.5M) all becoming free agents and likely to leave. This season seemed the perfect year to spend a bit more to supplement one of the best rosters in baseball with more starting pitching depth as payroll is likely to fall next year. The Pirates didn't do that. And if they miss the playoffs while waiting for Glasnow & Taillon to develop, and more-pointedly get past the Super-2 window, there should be hell to pay. But after winning 94, 88 and 98 games the last three years under Nutting's Draconian financial policies, Neal Huntington deserves the benefit of the doubt. We'll see.

But that's the real story. That the Pirates didn't supplement a roster of 22-23 players that is as good as any in the game, not that the Pirates made a mistake over $3K with Cole. Yesterday the Cardinals renewed pitcher Michael Wacha, a close comp to Cole in both service-time and performance, at $539K. Not a word was heard. The Mets renewed Noah Syndergaard at $531,875. Not a mention. The Pirates made a small mistake and Scott Boras pounced, undoubtedly with larger motives at work. With rosters mostly finalized and spring training games not yet underway, everyone was looking for any story to fill the news cycle. They got one. One that focused on a frugal, cheap if you prefer, and foolish decision by the Pirates. Unfortunately the media focused on the wrong Pirates financial story.

Not that they did it on purpose, but I wonder if the Pirates actually prefer it that way?