CASE STUDY
We built an MLB model that prices hits like a sportsbook. Then we rejected eight upgrades to it.
A Lagrange Labs case study on what honest machine learning actually looks like: leakage hunts, pre-registered gates, negative results, and a live system putting its picks on the record every day.
TL;DR. We built a model for MLB’s Beat the Streak contest that picks the batter most likely to get a hit each day. Across three held-out seasons its daily top pick hit 80.1% of the time. On 34,000 matched player-props it beats the de-vigged multi-book consensus and is a statistical tie with DraftKings’ closing lines, built entirely from free public data. After that, we tried eight successive model upgrades. Every one failed a pre-registered adoption gate and was rejected. The model now publishes its picks live, timestamped, every morning. Watch it calibrate.
The problem
MLB’s Beat the Streak contest asks one thing: pick a batter every day, and if your picks record a hit in 57 straight games, you win $5.6M. Nobody has ever won it.
As a machine-learning problem it’s deceptively nasty. About 58% of everyday hitters record a hit on any given day, so the signal-to-noise ratio is brutal: you’re not finding needles in haystacks, you’re ranking hay by needle-likelihood. And the contest only ever consumes the very top of your ranking: whether your model is smart about the 200th-best batter today is worth nothing. The product is the top two rows of the board.
It’s also (and this matters later) a free-entry contest. There is no vig to clear. Hold that thought.
The bug that taught us everything
Our first model looked incredible. Its top daily picks were predicted at 0.94 and the simulated probability of a 57-game streak came out to 27%. Champagne, right?
The picks hit at 0.70. Every prediction above 0.80 landed flat at ~0.65 actual.
That gap is the diagnosis. A true top pick in this sport lives around 0.70–0.75. That’s what the best hitter against the weakest pitcher in the best park is actually worth. A model that says 0.94 isn’t finding an edge; it’s leaking the answer key. Ours was: batter-vs-pitcher career aggregates had been merged onto training rows including each row’s own season. Every training example contained a whiff of its own outcome.
Two fixes, both structural:
- Batter-vs-pitcher stats became expanding-by-season: a row in season S sees only seasons before S, with empirical-Bayes shrinkage so a 4-for-6 career line doesn’t masquerade as a .667 hitter.
- The contest simulator got its own out-of-time recalibration layer: an expanding isotonic fit on strictly-past days, refit weekly, so the Monte Carlo simulation samples from probabilities that have been honest so far, never from the model’s own optimism.
Post-fix, everything got worse and better at once: predictions dropped to sane levels, AUC rose from 0.614 to 0.627, top-1 accuracy rose from 0.698 to 0.742, and the fantasy P(57)=27% collapsed to approximately zero, which is the correct answer, and why the prize is $5.6M.
The rule we took away: in a domain with a known performance ceiling, results that beat the ceiling are bugs. It became our cheapest and most reliable test.
The system
The final architecture is almost boring, which is the point:
- Data: pitch-level Statcast (free), aggregated to per-game batter and pitcher logs; five seasons.
- Features: rolling-window form (5/10/30/60 games,
shift(1)so a row never sees its own game), season-to-date rates, shrunk prior-season baselines for April cold-starts, opposing-starter quality-of-contact allowed, bullpen hit-rate, platoon splits, park factors, shrunk BvP. - Model: LightGBM classifier, early-stopped on a time-ordered tail (settles at ~65–125 trees; the original fixed 500 was 4–8× overgrown).
- Evaluation: strict walk-forward. Retrain weekly on everything before each date, predict that date, never look ahead. Judged primarily on top-of-distribution metrics (daily top-1/top-3 hit rate) because global AUC over 48,000 rows is dominated by mid-ranking batters the contest never touches.
Three held-out seasons, one config, no per-season tuning:
| Season | AUC | Daily top-1 hit rate |
|---|---|---|
| 2023 | 0.628 | 76.9% |
| 2024 | 0.632 | 81.6% |
| 2025 | 0.629 | 81.9% |
| Pooled | 80.1% (440/549) |
The gate, and eight funerals
Once the baseline stabilized, every proposed improvement faced the same pre-registered gate: pooled three-season AUC must improve AND pooled top-1 must hold. Single-season top-1 has a standard error of ±3 points, and accepting anything on one good season is how you ship noise.
Eight upgrades entered. Zero left:
- Plate-discipline features (whiff/chase rolling rates): metrics identical to four decimals; K-rate already carries the signal.
- Monotone constraints (“more hitting can’t lower P(hit)”): AUC flat, top-1 shuffled within noise.
- Model-level calibration (isotonic/sigmoid inside training): adds nothing once the simulation layer recalibrates out-of-time; costs 5× the training time.
- Hyperparameter search (31-trial holdout): the surface is a plateau; the best trial beat production by 0.00005 logloss.
- Seed ensembling + recency decay + pitch-mix matchups: pooled AUC +0.0007, pooled top-1 down 0.801 → 0.77.
- Pitch-mix features alone: same signature. AUC up a hair, top of the board worse.
- LambdaRank (rank the slate directly, optimizing exactly what the contest pays for): top-1 collapsed 0.801 → 0.729. Mechanism: lambdarank assumes sparse relevance, but 58% of a slate are positives; binary-relevance ranking throws away the probability information that carries the actual signal.
- Per-PA two-stage model (predict each plate appearance, compose over the PA distribution; architecturally the “right” factorization): AUC worse on all three seasons, pooled top-1 0.769. Mechanism: independence composition over-values a fifth plate appearance that’s really a correlated retry against the same pitching, so the model over-ranks high-volume contact hitters. It picked Luis Arraez 29 times in 2025. On days it disagreed with the baseline, it hit 0.694 to the baseline’s 0.826. Even blending the two models’ rankings made things worse: the second model’s errors are biased, not noisy, and averaging with bias buys nothing.
A later ninth probe reached for genuinely new information rather than another transformation: a fixed 24-hour-ahead temperature forecast plus stadium roof type. It produced the now-familiar signature (mean AUC +0.00043, pooled top-1 down 0.8015 → 0.7596) and was rejected. Weather stayed on the live board as a postponement/safety check, not in the ranking model.
Notice the recurring signature in 5–8: global metrics up, decision metrics down. If we had gated on AUC (the default instinct), we’d have shipped four consecutive downgrades to the only thing the contest pays for. The metric you gate on has to be the decision you ship.
Eight rejections isn’t failure. Eight rejections is what a real edge looks like once you’ve found it: everything else you try makes it worse, and you can prove it.
The external check: pricing against Vegas
Internal backtests can fool you in ways you haven’t imagined yet. So we bought the ground truth: 1,966 games of 2025 closing “batter to record a hit” prop lines, and compared prices on ~34,000 matched player-days.
- The model beats the de-vigged multi-book closing consensus: logloss 0.6551 vs 0.6607, Brier 0.2314 vs 0.2341, top-1 hit rate 0.810 vs 0.762.
- Against the two sharpest books individually, it’s a statistical tie (DraftKings logloss 0.66592 vs our 0.66544), though it wins the assumption-free ranking test against FanDuel (top-1 0.810 vs 0.776).
- And the punchline: a naive betting backtest showed +7–30% ROI, which a sharp-book audit dissolved entirely. The “profit” lived at soft books with stale prices and betting limits; at books that matter, after vig, it’s DraftKings −1.5% and FanDuel −3.2%.
Read that carefully, because it’s the honest shape of most real edges: we reached parity with professional prop desks using free public data, and parity minus a 6–7% vig is a losing bet. The model’s edge is real and it is small, so the only place it pays is a venue with no rake. Beat the Streak is a free-entry contest that rewards exactly the thing the model is best at: its top-1 pick. That’s not a consolation prize; that’s the strategy. The generalizable version: find the structurally subsidized game before you build the model, not after.
Luck or process? Autopsy every failure
Across three seasons the top pick failed 109 times in 549 days. Are those modeling errors or baseball? We joined every failed pick to its actual plate appearances and composed a “deserved” hit probability from per-batted-ball expected batting average (xBA), what the day’s contact quality should have produced:
- 50 of 109 failures deserved to win (deserved-p ≥ 0.5): correct picks that lost coin flips. Winners’ mean deserved-p was 0.81; failures’ 0.48.
- Only ~12 picks per season were “truly wrong” (weak contact or overpowered), and converting even a third of those would move top-1 by less than a point.
- Stated probabilities implied ~136 expected failures. We observed 109. The top of the board outperforms its own stated confidence.
We also sliced failures by everything we could think of (predicted probability, gap to the #2 pick, slate size, number of eligible alternatives), looking for a “don’t play today” signal. Flat on every axis. Three independent analyses (threshold sweep, day-selection diagnostic, double-down sweep) agree: there is no exploitable structure in when to play. When the residual is baseball, more modeling is procrastination.
The strategy layer had bugs too
The contest layer got the same treatment as the model. The simulator’s multi-entry mode had a subtle correlation bug: ten entries holding the same player were rolling ten independent dice, phantom diversification that inflated every tail metric. Fixed (outcomes drawn once per player-day, shared across entries), the true dominant lever emerged: spreading independently controlled entries across distinct players, which raised the simulated probability of a 30-game streak by 68% on 2025. The 2026 rules allow only one registration per person, so this is a group-of-distinct-entrants research scenario, not a single-person playbook. A follow-up test found no reliable benefit from forcing a double-down pair into different games (pooled joint hit rate fell 0.585→0.574). And a sweep of “should we ever pick only one player on marginal days?” came back unanimous: always take both picks. Advancing the streak two games at a time at ~55% joint success beats safety at every bar we tested.
Honest expectations, from recalibrated probabilities: a single disciplined entry averages a ~16-game best streak per season; hypothetical groups of ten independently controlled, diversified entries reach a 30-game streak in the mid-to-high single digits with the 2026 Streak Saver. Exact dynamic programming puts a single entry’s P(56) around one in 100,000 seasons at the model’s operating point; 57 remains a lottery. A target-aware endgame (raise the guardrail at 50 and take only one pick at 56) improves that microscopic P(57) by about 36% across held-out seasons. Anyone selling you better odds is selling.
Show your work: one season, three ways
Assertions like “averages a ~16-game streak” and “57 is a lottery” are easy to write and hard to feel. So here is the 2025 season rendered three ways: the realized path, the distribution behind it, and the frontier that governs the whole game. Every figure below is generated from the saved 2025 predictions at build time; none of these numbers are typed by hand.
The 2025 season, day by day
Take the model’s single highest-probability hitter every day and never sit out. This is what the streak actually did: a sawtooth that climbs one game per hit and collapses to zero when an unprotected pick goes hitless. The top pick hit 81.9% of the time (149 of 182). That was a favorable draw, and the model was, if anything, slightly under-confident at the top of the board. Its longest raw run was 17; applying the 2026 one-time Streak Saver lifts the rules-adjusted maximum to 20, still nowhere near 57. That gap between a great hit rate and a short best streak is why 57 is hard.
Daily hit rate
81.9%
149 of 182 picks hit
Longest streak
20
May 5 – May 25
Streaks run
26
separate streaks all year
Times broken
25
hitless picks that reset it
Median streak
4.5 g
half die by 4.5 games
The season, game by game
Streak length on each of 182 game-days. A red dot marks a broken streak, at the length it reached before the pick went hitless. Hover any day for the pick.
View as table
| Date | Pick | P(hit) | Result | Streak |
|---|---|---|---|---|
| Mar 27 | Vladimir Guerrero Jr. | 72% | no hit | 0 |
| Mar 28 | Lourdes Gurriel Jr. | 71% | hit | 1 |
| Mar 29 | Jose Altuve | 72% | hit | 2 |
| Mar 30 | Adley Rutschman | 72% | hit | 3 |
| Mar 31 | Jose Altuve | 71% | hit | 4 |
| Apr 1 | Jose Altuve | 71% | hit | 5 |
| Apr 2 | Jarren Duran | 72% | no hit | 0 |
| Apr 3 | Alec Bohm | 73% | hit | 1 |
| Apr 4 | JJ Bleday | 75% | hit | 2 |
| Apr 5 | Brenton Doyle | 76% | hit | 3 |
| Apr 6 | Jarren Duran | 76% | hit | 4 |
| Apr 7 | Bobby Witt Jr. | 80% | no hit | 0 |
| Apr 8 | Bobby Witt Jr. | 79% | hit | 1 |
| Apr 9 | Bobby Witt Jr. | 80% | hit | 2 |
| Apr 10 | Bobby Witt Jr. | 79% | hit | 3 |
| Apr 11 | Yandy Díaz | 74% | hit | 4 |
| Apr 12 | Bobby Witt Jr. | 73% | hit | 5 |
| Apr 13 | Bobby Witt Jr. | 73% | hit | 6 |
| Apr 14 | Luis Arraez | 72% | hit | 7 |
| Apr 15 | Steven Kwan | 72% | hit | 8 |
| Apr 16 | Jeremy Peña | 73% | hit | 9 |
| Apr 17 | Bobby Witt Jr. | 73% | hit | 10 |
| Apr 18 | Jarren Duran | 75% | hit | 11 |
| Apr 19 | Bobby Witt Jr. | 74% | hit | 12 |
| Apr 20 | Bobby Witt Jr. | 74% | hit | 13 |
| Apr 21 | Jarren Duran | 73% | hit | 14 |
| Apr 22 | Bobby Witt Jr. | 73% | hit | 15 |
| Apr 23 | Yandy Díaz | 72% | hit | 16 |
| Apr 24 | Bobby Witt Jr. | 72% | hit | 17 |
| Apr 25 | TJ Friedl | 73% | no hit | 0 |
| Apr 26 | Elly De La Cruz | 72% | hit | 1 |
| Apr 27 | Gleyber Torres | 72% | hit | 2 |
| Apr 28 | Aaron Judge | 72% | hit | 3 |
| Apr 29 | Ozzie Albies | 72% | hit | 4 |
| Apr 30 | Shohei Ohtani | 72% | hit | 5 |
| May 1 | Vladimir Guerrero Jr. | 72% | hit | 6 |
| May 2 | Bobby Witt Jr. | 72% | no hit | 0 |
| May 3 | Bobby Witt Jr. | 73% | no hit | 0 |
| May 4 | Alex Bregman | 73% | no hit | 0 |
| May 5 | Bryan Reynolds | 73% | hit | 1 |
| May 6 | Steven Kwan | 73% | hit | 2 |
| May 7 | Corey Seager | 73% | hit | 3 |
| May 8 | Gleyber Torres | 73% | hit | 4 |
| May 9 | Manny Machado | 73% | hit | 5 |
| May 10 | Luis Arraez | 74% | hit | 6 |
| May 11 | Manny Machado | 74% | hit | 7 |
| May 12 | Xavier Edwards | 71% | hit | 8 |
| May 13 | Jackson Chourio | 71% | hit | 9 |
| May 14 | Gunnar Henderson | 73% | hit | 10 |
| May 15 | Vladimir Guerrero Jr. | 74% | hit | 11 |
| May 16 | Bobby Witt Jr. | 74% | hit | 12 |
| May 17 | Bobby Witt Jr. | 75% | no hit | 12 |
| May 18 | Yandy Díaz | 75% | hit | 13 |
| May 19 | Trea Turner | 75% | hit | 14 |
| May 20 | Trea Turner | 75% | hit | 15 |
| May 21 | Ezequiel Tovar | 75% | hit | 16 |
| May 22 | Bryce Harper | 75% | hit | 17 |
| May 23 | Jarren Duran | 75% | hit | 18 |
| May 24 | Jarren Duran | 75% | hit | 19 |
| May 25 | Paul Goldschmidt | 75% | hit | 20 |
| May 26 | Bobby Witt Jr. | 75% | no hit | 0 |
| May 27 | Bobby Witt Jr. | 75% | no hit | 0 |
| May 28 | Vladimir Guerrero Jr. | 72% | hit | 1 |
| May 29 | Vladimir Guerrero Jr. | 72% | hit | 2 |
| May 30 | Vladimir Guerrero Jr. | 75% | hit | 3 |
| May 31 | Vladimir Guerrero Jr. | 74% | hit | 4 |
| Jun 1 | Bobby Witt Jr. | 74% | hit | 5 |
| Jun 2 | Nolan Schanuel | 73% | no hit | 0 |
| Jun 3 | Bobby Witt Jr. | 74% | hit | 1 |
| Jun 4 | Jarren Duran | 74% | hit | 2 |
| Jun 5 | Bobby Witt Jr. | 74% | hit | 3 |
| Jun 6 | Pete Alonso | 74% | hit | 4 |
| Jun 7 | Francisco Lindor | 74% | hit | 5 |
| Jun 8 | Francisco Lindor | 74% | no hit | 0 |
| Jun 9 | Yandy Díaz | 74% | hit | 1 |
| Jun 10 | Yandy Díaz | 74% | hit | 2 |
| Jun 11 | Jung Hoo Lee | 74% | hit | 3 |
| Jun 12 | Jung Hoo Lee | 74% | hit | 4 |
| Jun 13 | Bobby Witt Jr. | 74% | no hit | 0 |
| Jun 14 | Brent Rooker | 76% | no hit | 0 |
| Jun 15 | Aaron Judge | 76% | no hit | 0 |
| Jun 16 | Trea Turner | 76% | hit | 1 |
| Jun 17 | Alec Bohm | 76% | hit | 2 |
| Jun 18 | Luis García Jr. | 76% | hit | 3 |
| Jun 19 | Vladimir Guerrero Jr. | 76% | hit | 4 |
| Jun 20 | Ketel Marte | 76% | hit | 5 |
| Jun 21 | Ketel Marte | 77% | hit | 6 |
| Jun 22 | Josh Naylor | 77% | hit | 7 |
| Jun 23 | Seiya Suzuki | 75% | hit | 8 |
| Jun 24 | Mookie Betts | 77% | hit | 9 |
| Jun 25 | Yandy Díaz | 77% | hit | 10 |
| Jun 26 | Yandy Díaz | 77% | no hit | 0 |
| Jun 27 | Vladimir Guerrero Jr. | 77% | hit | 1 |
| Jun 28 | Vladimir Guerrero Jr. | 76% | no hit | 0 |
| Jun 29 | Yandy Díaz | 76% | hit | 1 |
| Jun 30 | Bobby Witt Jr. | 75% | hit | 2 |
| Jul 1 | Yainer Diaz | 76% | no hit | 0 |
| Jul 2 | Jose Altuve | 76% | hit | 1 |
| Jul 3 | Jose Altuve | 76% | hit | 2 |
| Jul 4 | Chase Meidroth | 76% | no hit | 0 |
| Jul 5 | Hunter Goodman | 77% | hit | 1 |
| Jul 6 | Chase Meidroth | 77% | hit | 2 |
| Jul 7 | Yandy Díaz | 75% | hit | 3 |
| Jul 8 | Bobby Witt Jr. | 77% | hit | 4 |
| Jul 9 | Jarren Duran | 77% | hit | 5 |
| Jul 10 | Yandy Díaz | 77% | hit | 6 |
| Jul 11 | Yandy Díaz | 76% | no hit | 0 |
| Jul 12 | Elly De La Cruz | 73% | hit | 1 |
| Jul 13 | Elly De La Cruz | 76% | hit | 2 |
| Jul 18 | Carlos Correa | 76% | no hit | 0 |
| Jul 19 | Bobby Witt Jr. | 76% | no hit | 0 |
| Jul 20 | Carlos Correa | 76% | hit | 1 |
| Jul 21 | Iván Herrera | 76% | hit | 2 |
| Jul 22 | Alec Burleson | 76% | hit | 3 |
| Jul 23 | Alec Burleson | 74% | no hit | 0 |
| Jul 24 | Nolan Schanuel | 71% | hit | 1 |
| Jul 25 | Manny Machado | 74% | hit | 2 |
| Jul 26 | Jordan Westburg | 74% | hit | 3 |
| Jul 27 | Teoscar Hernández | 74% | no hit | 0 |
| Jul 28 | Vladimir Guerrero Jr. | 76% | hit | 1 |
| Jul 29 | Bobby Witt Jr. | 76% | hit | 2 |
| Jul 30 | Bobby Witt Jr. | 74% | no hit | 0 |
| Jul 31 | Matt McLain | 72% | hit | 1 |
| Aug 1 | Yainer Diaz | 74% | hit | 2 |
| Aug 2 | Bryan Reynolds | 74% | hit | 3 |
| Aug 3 | Bryan Reynolds | 74% | no hit | 0 |
| Aug 4 | Bo Bichette | 74% | hit | 1 |
| Aug 5 | Bo Bichette | 74% | hit | 2 |
| Aug 6 | Bo Bichette | 74% | hit | 3 |
| Aug 7 | Brent Rooker | 74% | hit | 4 |
| Aug 8 | Xavier Edwards | 74% | hit | 5 |
| Aug 9 | Xavier Edwards | 74% | hit | 6 |
| Aug 10 | Corbin Carroll | 74% | hit | 7 |
| Aug 11 | Luis Arraez | 74% | no hit | 0 |
| Aug 12 | Bobby Witt Jr. | 74% | hit | 1 |
| Aug 13 | Bobby Witt Jr. | 75% | hit | 2 |
| Aug 14 | Lourdes Gurriel Jr. | 75% | hit | 3 |
| Aug 15 | Ketel Marte | 75% | hit | 4 |
| Aug 16 | Xavier Edwards | 75% | no hit | 0 |
| Aug 17 | Bobby Witt Jr. | 75% | hit | 1 |
| Aug 18 | Mookie Betts | 75% | hit | 2 |
| Aug 19 | Freddie Freeman | 75% | hit | 3 |
| Aug 20 | Freddie Freeman | 75% | hit | 4 |
| Aug 21 | Bobby Witt Jr. | 75% | hit | 5 |
| Aug 22 | Nick Gonzales | 75% | hit | 6 |
| Aug 23 | Trea Turner | 75% | hit | 7 |
| Aug 24 | Francisco Lindor | 75% | hit | 8 |
| Aug 25 | Nick Gonzales | 73% | hit | 9 |
| Aug 26 | Carlos Correa | 75% | hit | 10 |
| Aug 27 | Bobby Witt Jr. | 76% | hit | 11 |
| Aug 28 | Bryce Harper | 75% | hit | 12 |
| Aug 29 | Bobby Witt Jr. | 76% | hit | 13 |
| Aug 30 | Yandy Díaz | 75% | no hit | 0 |
| Aug 31 | Seiya Suzuki | 76% | hit | 1 |
| Sep 1 | Heliot Ramos | 76% | hit | 2 |
| Sep 2 | Heliot Ramos | 76% | hit | 3 |
| Sep 3 | Heliot Ramos | 75% | hit | 4 |
| Sep 4 | Bobby Witt Jr. | 75% | hit | 5 |
| Sep 5 | Manny Machado | 75% | no hit | 0 |
| Sep 6 | Luis Arraez | 75% | hit | 1 |
| Sep 7 | Luis Arraez | 75% | no hit | 0 |
| Sep 8 | Julio Rodríguez | 72% | hit | 1 |
| Sep 9 | Agustín Ramírez | 72% | hit | 2 |
| Sep 10 | Heliot Ramos | 73% | hit | 3 |
| Sep 11 | Xavier Edwards | 68% | hit | 4 |
| Sep 12 | Bobby Witt Jr. | 74% | hit | 5 |
| Sep 13 | Bobby Witt Jr. | 74% | no hit | 0 |
| Sep 14 | Bobby Witt Jr. | 73% | hit | 1 |
| Sep 15 | Jurickson Profar | 73% | hit | 2 |
| Sep 16 | Agustín Ramírez | 75% | hit | 3 |
| Sep 17 | Agustín Ramírez | 74% | hit | 4 |
| Sep 18 | Agustín Ramírez | 75% | hit | 5 |
| Sep 19 | Nolan Schanuel | 74% | hit | 6 |
| Sep 20 | Nolan Schanuel | 75% | hit | 7 |
| Sep 21 | Jo Adell | 75% | hit | 8 |
| Sep 22 | Heliot Ramos | 74% | hit | 9 |
| Sep 23 | Chandler Simpson | 74% | hit | 10 |
| Sep 24 | Luis Arraez | 68% | hit | 11 |
| Sep 25 | Xavier Edwards | 74% | no hit | 0 |
| Sep 26 | Vladimir Guerrero Jr. | 74% | hit | 1 |
| Sep 27 | Heliot Ramos | 74% | hit | 2 |
| Sep 28 | Jarren Duran | 74% | hit | 3 |
How far streaks got before breaking
Count of the 26 separate streaks by the length they reached. Most die young; the long tail is rare.
What randomness and strategy change
One season is a single roll of the dice. Replay the same daily picks under the model’s recalibrated probabilities 10,000 times and the spread you get is the randomness, and it reads off how far each contest strategy pushes you toward 57. A lone double-down entry peaks at a median of 16; a hypothetical group of ten independently controlled, diversified entries moves the median into the low 20s and the odds of a 30-game streak into the high single digits. The magic number stays offscreen.
Four ways to play, on the road to 57
Where each strategy's max-streak distribution lands across 10,000 simulated seasons. Thin bar = middle 90% of seasons, thick = middle 50%, dot = median; the red dot is this season's one realized result. Adding accounts shifts you right, but 57 stays out of reach.
| Strategy | Avg peak | Median | P(≥20) | P(≥30) | P(≥40) | P(≥57) |
|---|---|---|---|---|---|---|
| This season (actual) | 20 | — | — | — | — | — |
| Single pick | 16.1 | 16 | 18.5% | 0.96% | 0.03% | ~0 |
| Double down | 16.8 | 16 | 24.8% | 1.4% | 0.06% | ~0 |
| Ten accounts | 22.8 | 22 | 84.0% | 7.7% | 0.33% | 0.01% |
The accuracy × accounts frontier
Two levers move you toward 57: per-pick accuracy, and the number of independent accounts (made independent by diversifying picks). Here is P(reach 57 in one season) for every combination, from an exact longest-run model over 186 game-days, cross-checked against the simulator. The only honest reading sits in the bottom-left corner, because everything that reaches 57 lives above a wall baseball won’t let you cross.
P(reach 57) · accuracy × accounts
Double-down, 186 game-days. The model's top pick today sits at ≈0.743 (near the 0.75 row). Everything above the red line needs a per-pick accuracy no real MLB matchup delivers.
↑ above the dashed line = beyond the real single-game hit ceiling (~0.80–0.82)
| accuracy ↓ | 1 | 5 | 10 | 25 | 50 | 100 | 250 |
|---|---|---|---|---|---|---|---|
| 0.92 | 21% | 70% | 91% | ~100% | ~100% | ~100% | ~100% |
| 0.90 | 8.0% | 34% | 56% | 87% | 98% | ~100% | ~100% |
| 0.88 | 2.7% | 13% | 24% | 50% | 75% | 94% | ~100% |
| 0.85 | 0.48% | 2.4% | 4.7% | 11% | 21% | 38% | 70% |
| 0.82 | <0.1% | 0.38% | 0.75% | 1.9% | 3.7% | 7.3% | 17% |
| 0.80 | <0.1% | 0.10% | 0.21% | 0.52% | 1.0% | 2.1% | 5.1% |
| 0.78 | <0.1% | <0.1% | <0.1% | 0.14% | 0.28% | 0.56% | 1.4% |
| 0.75 ◄ model ≈0.74 | <0.1% | <0.1% | <0.1% | <0.1% | <0.1% | <0.1% | 0.18% |
| 0.72 | <0.1% | <0.1% | <0.1% | <0.1% | <0.1% | <0.1% | <0.1% |
Bottom line
- At the model's accuracy today (a calibrated ~0.743 per pick), 57 is off the table. Even 250 diversified double-down accounts reach it in about 0.18% of seasons (the 0.75 row, 250-account column).
- A realistic season-long shot at 57 only appears at accuracies of 0.85 and up: 0.85 × 100 accounts → 38%, × 250 → 70%, and no MLB matchup delivers a 0.85 single-game hit probability.
- Even at the real single-game ceiling (~0.80–0.82 P(hit)), you'd need 250 accounts for roughly 17% and 100 for about 7.3%, and you can't field that many distinct high-quality picks.
- So chase 30–40, not 57. The best realistic setup (about ten diversified double-down accounts) peaks at a median of 22 and reaches a 30-game streak in 7.7% of seasons; 57 essentially never.
Why the bottom-left of the table is the only real world
- A single game tops out near 0.80–0.82 P(hit) even for the best hitter in the best matchup; the calibrated model's top pick sits around 0.743, and no model can out-predict the underlying baseball.
- Only a handful of hitters clear the operating point on any given slate, so the first few accounts decorrelate almost for free, but each extra account has to reach for a weaker pick, and accuracy falls exactly as you add shots.
- The two levers fight each other: the accuracy band that reaches 57 (≥0.85) is physically unreachable, and the band you can actually field (≤0.82) would need hundreds of accounts of distinct, high-quality picks you don't have.
Method: an exact longest-run Markov model over 186 game-days (double-down needs a run of 29 winning days at p²), cross-checked against the project's Monte Carlo simulator; the two agree on P(reach 20) to within 0.0005 at the operating point.
The live system
A backtest is a claim; a live system is evidence. The model now runs every morning on entirely free APIs (MLB Stats API for schedules, probables, lineups, roster status, venue and weather; Statcast for yesterday’s results), retrains through yesterday, and publishes a ranked board with its play policy before first pitch. Same feature code path as the backtest, by construction (shadow runs against the 2025 backtest agree with correlation 0.96–0.98). The displayed ranking can appear before lineups, but an official recommendation is held until lineup, starter, active roster, data freshness, calibration, weather/roof, and game-lock checks all pass. Picks are logged before games start and never edited; outcomes settle against official box scores; the calibration chart updates as live days accumulate. Contest entry remains manual.
The whole daily run takes about three minutes on free data. The board is public. Watch it hit, miss, and calibrate in real time.
What to steal from this project
- Pre-register an adoption gate on pooled held-out data, and let it kill your favorite ideas. Ours killed eight straight. That’s the gate working.
- Gate on the decision you ship, not the global metric. Four upgrades improved AUC while degrading the top pick.
- Know the ceiling of your domain. Results above it are bugs. The best debugging tool we had was the number 0.75.
- Recalibrate out-of-time before simulating decisions. A simulator fed raw model output turns miscalibration into fiction at compound interest.
- Buy external validation when a market exists. Sportsbook closing lines are the best forecasters on earth for this problem; parity with them recontextualized every internal number.
- Autopsy failures against counterfactual ground truth (we used contact-quality xBA) to separate variance from error before “fixing” anything.
- Audit the decision layer as hard as the model. Our biggest single win (+68% on the tail metric) was a correlation bug in the simulator, not the model.
- Publish live. Nothing disciplines a model like a public, timestamped, unedited pick log.
Lagrange Labs is a product studio. We build data platforms, full-stack apps, and AI products, and we stay on to run them in production. This project is our idea of a résumé: the model’s picks are live right now.
Not affiliated with, sponsored, or endorsed by Major League Baseball. “Beat the Streak” is a trademark of MLB Advanced Media, L.P. Data from the MLB Stats API and Baseball Savant. For entertainment and research, not betting advice.
See today's board