---
id: "lib-2026-11-02-look-before-you-launch-learned-throughput"
type: "library"
title: "Look Before You Launch: Learned Throughput Prediction for Hardware Selection in a Production Training Fleet"
lang: "EN"
org: "Meta"
source_type: "Report"
date: "2026-11-02"
week: "2026-W45"
url: "https://nosov.ai/library/lib-2026-11-02-look-before-you-launch-learned-throughput/"
original_url: "https://arxiv.org/abs/2611.04057"
tier: "B"
tier_name: "Useful, limited"
tier_reason: "Meta engineers report a production throughput predictor routing training jobs across 20+ hardware types; solid practice with ablations, but the savings figure is the operator's own estimate on internal data."
summary_basis: "full-text"
original_minutes: "35"
tags: ["ai technology", "infrastructure cost"]
---

# Look Before You Launch: Learned Throughput Prediction for Hardware Selection in a Production Training Fleet

Tier B (Useful, limited): Meta engineers report a production throughput predictor routing training jobs across 20+ hardware types; solid practice with ablations, but the savings figure is the operator's own estimate on internal data.

Summary based on the full text.

## In short

An arXiv paper (2 November 2026) by Hossein Askari and colleagues at Meta. In a mixed-vendor training fleet, many jobs can run on several accelerator types, and a poor choice wastes compute without any error. LEAP predicts each job's throughput on every candidate type at submission and ranks options by throughput per unit cost. It replaced a lookup table and now covers almost all eligible GPU-hours.

## Key numbers

- 77%: prediction accuracy over 30 nightly retrains
- 58% to 95%: share of eligible training GPU-hours ranked by LEAP over six months
- about 3%: estimated saving in training compute cost
- more than 20: hardware types in the mixed-vendor fleet; tens of thousands of job attempts routed a day

## What it reports

- Routes tens of thousands of training job attempts a day across hundreds of model types and more than 20 hardware types.
- Accuracy 77% over 30 nightly retrains, higher than the lookup table it replaced and at least as high as every alternative tested, including fine-tuned language models.
- Coverage of eligible training GPU-hours grew from 58% to 95% over six months.
- Estimated saving: about 3% of training compute cost.
- Hardware routing governs roughly half of the fleet's half-year GPU-hours; the long tail matters, with the top 100 model types covering 78% of one day's attempts.
- Each job's own prior throughput is a key input; the paper reports which drivers move accuracy and which do not.

## How they got it

A learned prediction service trained on submission-time configuration and execution history, evaluated against the legacy table and alternative models, with coverage and savings tracked in production.

## Limits

- Self-report on internal data; the 3% saving is an estimate, not a controlled comparison.
- The accuracy metric's definition matters and is specific to the paper.
- Applies to fleets large and heterogeneous enough to have real routing choices.

## Why it matters for companies

Silent inefficiency is common wherever workloads can run on several hardware or cloud options: jobs finish, so nobody notices the waste. A simple learned predictor using each job's own history captured a few percent of compute at Meta, and the same idea applies to cloud instance selection. Language models were not better than a conventional model for this task.

Original: https://arxiv.org/abs/2611.04057