Should managers fear ‘reliever familiarity effect’ in World Series?

In an October version of baseball that is increasingly ruled by how often you can roll out your filthiest relievers – where “starters” very much means only that – there’s an unexpected yet fascinating question to be asked:

Why does it seem like all of the most dominating relievers in the game have gotten torched this October?

The problem, then, is clear: The “make the whole plane out of relievers” trend must have finally gone too far. Teams are asking far too much of their relievers, all the time, particularly in asking them to face the same batters repeatedly in a short series. It’s not always a planned strategy so much as it is an absolute necessity – Cleveland and Detroit, in particular, simply had extremely thin rotations that would not likely have fared better had they been used in a traditional manner – but entering the World Series, relievers have thrown more than 52% of all postseason pitches, well up from 34% back in 2013.

You’ve surely heard a lot this fall about the dangers of a reliever facing a batter multiple times. But what’s really happening in 2024? It’s a lot more complicated than that.

First things first: Despite all the extra work, relievers haven’t been worse in 2024 than in recent postseasons. In fact, they’ve been almost exactly as effective this year as they’ve been over the last eight postseasons, dating back to the fall of 2016, when the usage of Andrew Miller (and the non-usage of Zack Britton) marked a turning point in how October relievers are deployed.

Relievers in the playoffs

Which makes sense, or at least it should; for all the big hits we’ve seen relievers give up, it’s easy to forget that we still saw ongoing dominance from top arms like Smith, Weaver, Blake Treinen and lesser-known-yet-effective names like Will Vest and Erik Sabrowski. We’re remembering the bad moments – how could you not think about Clase giving up as many postseason homers in about five minutes as Mariano Rivera did in an entire career – and that perhaps overshadows all the other good work.

But, maybe, there’s also the question of whether the big hits are about hitters getting too many looks at the same reliever, which has become something of the topic du jour this month. Ben Lindbergh dug into this idea last October for The Ringer, sharing a research paper and explaining that it’s a similar idea as the “third time through” effect we talk about with starting pitchers so often, that simply seeing a pitcher’s repertoire repeatedly will help train the batter on what’s coming.

It’s come up a lot this month, and it’s true, clearly, if you look at most any time period in recent baseball history. For example, you’ve probably seen a version of this chart recently, showing that reliever effectiveness the third time facing a hitter in the same series plummets, to the tune of an increase of more than 180 points of OPS over the 2016-23 period.

There’s no denying the effect – it’s both intuitive and well-studied, and most likely due to familiarity. To cherry-pick a timeframe from decades ago, postseason relievers from 1996-2002 were 76 points worse the third time they saw a batter than the first.

Except … it’s not happening this year, at least not like that.

Run the same chart back again but with 2024 included, and you can see that the first time a reliever faces a batter, the outcomes are about the same as they’ve always been, with an OPS just south of .700. But in 2024, those relievers have gotten better the second time, and then the third-time OPS bump is relatively small. This year, at least, the relievers are actually having more success in the third matchup than they are in the first.

(We’re not showing the fourth time, because there are only 13 such plate appearances, but if we did, it would tell you that hitters have a .167 average and a .481 OPS this year. That’s Scott vs. Ohtani, sure, but it’s also things like Tim Hill allowing Steven Kwan a first-time single before retiring him the next three times.)

All of which goes against the popular narrative. So what’s that about – and how much might it impact the World Series?

Initially, it would seem likely that it’s a one-year fluke, that we’ll look back next year and wonder what was happening in 2024. Except: It didn’t happen in 2022, either, when relievers facing a batter the third time in a series allowed an OPS of a mere .396, which was 200 points lower than the first time. You might remember David Robertson retiring Yordan Alvarez four times in the World Series (with three strikeouts!), as well as José Alvarado also getting Kyle Tucker to post an 0-for-4 in the Series.

Most other recent years do show that there’s a mild downturn in performance the third time and further. It’s just wildly noisy and inconsistent, as opposed to the stable first and second matchups.

This might explain most of what’s happening in 2024. Over time, relievers are worse the third time (or more) they see a batter in a series, but it’s not at all a consistent trend from year to year.

That’s because it just doesn’t happen that much, really; 71% of all reliever postseason plate appearances are when they are facing a batter the first time that series, approximately 22% come on the second matchup, and the remainder, barely 7%, are on the third time or more. We’re at only 100 such plate appearances this year over the entire month.

If it feels, though, like we’re seeing the best arms continuing to give up the biggest hits, we might have some explanations.

For one thing, it’s almost certainly a case of unreasonable expectations for relievers, where the demand of perfection every single time is simply not possible to attain. Take Smith, for example; even with his ALCS Game 4 tater allowed to Stanton, he ended his October with a line of .152/.194/.273 against and a 16/2 K/BB. He was absolutely dominant, yet the expectation, it seems, is a line of .000/.000/.000 against. It’s simply not possible; sometimes, the batter is going to win. (They do 31% of the time, if you look at reliever OBP in October going back to 2016.) Even the great Rivera famously blew a World Series.

If there is an issue this year, it seems to be more about when those hits are coming. Relievers in the 2024 postseason have the third-worst Win Probability Added of the 21st century, and unlike batting average, OPS or most anything else, that’s a metric which is all about game context – i.e., a solo home run allowed in a tie game in the ninth is far more damaging than a fifth-inning solo homer in a blowout. For the first time in eight seasons, postseason relievers do not have a positive cumulative Win Probability Added, which would certainly lend to the “feel” of outcomes being worse, even if it’s possibly a fluky outcome.

But enough with the hypotheticals. Let’s look at some of those highest-profile blowups. What happened – and how can managers Dave Roberts and Aaron Boone learn from it?

We know, for example, that when Carpenter took Clase deep in the ALDS, it was because of a terrible pitch, a cement-mixer slider that had less horizontal break than all but one previous Clase slider, and it was the first time they’d seen each other all season long anyway. We also know that when Judge took Clase deep, it was on a perfectly nasty pitch that only left the yard because Judge, the all-but-certain AL MVP, did something no right-handed hitter had ever done before against Clase – and that was the second head-to-head matchup, with Clase winning on the first and third ones. When Hader let Houston’s season slip away, there was ample evidence that he’d been made less effective by his usage the day before. There are always reasons.

We know, too, that the Yankee trio of Judge, Stanton and Juan Soto have combined for 15 extra-base hits in October, not exactly the types of outcomes that demand FBI investigations. But it doesn’t mean nothing had changed: Stanton had started off 0-for-3 against Smith, and when he finally touched him the fourth time, Smith was throwing fastballs 2 mph slower than he usually is, which clearly points to overall fatigue.

For the Dodgers and Yankees specifically, their experiences were very different in their respective postseason rounds.

Their relievers allowed a .741 OPS the first time facing a batter in a series – that’s worse than average – but were elite the following times, with a .492 OPS the second time and a .486 OPS the third time. What’s most interesting here is who was pitching most of those times, because it mostly wasn’t their high-leverage relievers.

That’s overwhelmingly Honeywell, their long man, who soaked up 7 2/3 relief innings in a pair of NLCS outings, in situations where length was more important than matchups. Aside from the dominant Treinen, who retired both Luis Arraez and Fernando Tatis Jr. the third time he saw them in the NLDS, Roberts was able to mostly avoid this situation despite inconsistent starting pitching. Whether that’s because of Roberts purposefully managing with this in mind, or because much of the NLCS was a lopsided affair in one way or the other, is hard to say. The Dodgers had 9% of relief appearances coming the third time or later.

New York’s experience, however, was considerably different, likely because of how tight the ALCS ended up being. While 11% of their relief plate appearances coming in the third matchup or later doesn’t seem like a lot, look at who took those.

Those are Boone’s leverage relievers, and the performance here followed more of a traditional shape, as the team allowed a .606 OPS the first time, a .661 the second time, and a .690 the third time – all of which are still good numbers. In Hill’s case, the strategy of being a submariner with the lowest arm arm angle of any lefty in baseball probably makes him a matchup pick no matter how many times he’s seen a hitter. He retired three of the hitters he faced the third time or more in the ALCS, and walked Brayan Rocchio.

In Holmes’ case, all of these excess matchup plate appearances came against the Royals in the ALDS. Against Cleveland, Weaver got three outs, watched Anthony Rizzo make an error and allowed a double to Lane Thomas.

In the World Series, none of this means that Roberts and Boone shouldn’t be concerned about how many looks hitters get against the same reliever. The data is clear that they should be, even if it’s a little noisy year-to-year, and we’re a long way from the 1973 World Series, when Oakland’s Rollie Fingers faced New York’s Cleon Jones nine times in relief, winning six of those matchups.

But so far, these matchups haven’t hurt their clubs much. They only account for about one in every 11 relief plate appearances, anyway. At this point in the season, everyone’s tired. Everyone has been seen. The choice between a starter being seen for a third time that day or adding on another look against a reliever that may bite you later in the series won’t be an easy one.