Predictive World Models for AI Infrastructure

The optimal scheduling decision depends on constraints that have not emerged yet. Aurelius builds a predictive world model of the future cluster state, forecasts the operational and economic constraints a decision will face, simulates candidate workload decisions against that state, and selects the economic optimum subject to SLA, capacity, power, policy, and safety gates.

This report describes a finding that sharpened the architecture. A correct world model and a correct objective are not enough on their own: the economic optimum is only realized if the planner actually evaluates the right coupled candidate decisions. When candidate generation is made physics-guided, proposing the high-value, coupled bundles the operating regime implies, and searched under a regret audit, Aurelius recovers a Pareto-dominant result during expensive electricity windows, at zero search regret against the exhaustive ground truth.

Earlier versions of Aurelius were evaluated under a capped replay harness to isolate algorithmic behavior. This report instead evaluates uncapped, production-scale replay, which better reflects the operating regime of real serving systems and is therefore the primary benchmark going forward. Replaying the full selected public-trace benchmark windows with no per-period request limit (roughly 1.5M requests across PJM, ERCOT, and CAISO), Aurelius averages +724% higher SLA-safe goodput per dollar than a production-class scheduler baseline (per-market +698% / +718% / +755%), while holding SLA strictly better and using ~84% fewer GPU-hours. The number is a simple arithmetic mean of the three per-market deltas; §10 derives it from the underlying values, and §11 describes the baseline. This is a simulated replay on public trace windows: evidence, not a guarantee, and not a production deployment.

Most schedulers optimize a current-state or local objective: availability, fairness, utilization, latency, queue state, immediate capacity. Each is reasonable in isolation. But the economic outcome of a placement is decided by constraints that arrive after the placement is made: power and capacity headroom, congestion, memory and topology pressure, demand, and the price of electricity, which on public wholesale markets can swing several-fold within a day.

Once a job lands, those costs are largely locked in. An observe → decide loop cannot price a constraint it has not yet seen. Aurelius closes that gap with an observe → forecast → simulate → decide loop, a different control architecture, not another scheduler.

Each control period, Aurelius forecasts the workload and conditions, generates candidate action bundles, simulates each against the forecasted world state, scores it by risk-adjusted SLA-safe goodput per dollar, rejects anything that fails a hard gate, and recommends the best safe bundle, falling back deterministically when forecast confidence is low.

Crucially, it evaluates coupled action bundles, not one knob. Workload timing, placement, routing, capacity, batching, and the precision/clock policy are scored together, because those decisions interact, and, as this report shows, the interactions are exactly where the economic value lives.

Aurelius maintains a persistent world model of the cluster: a forward model of the fleet, not a snapshot of free GPUs. It carries the replica warm/cold state, server and rack capacity, placement and locality, queue dynamics, in-flight migrations, and the operator-cost basis, and it advances that state one control period at a time so a candidate decision can be scored against the state it will actually meet.

Candidate decisions are evaluated on read-only simulated future states, so scoring never mutates the live cluster. Inputs are tagged by provenance (measured, trace-derived, benchmark-calibrated, or modeled), so no result silently treats a modeled value as a measured one.

Aurelius forecasts arrival rate, request characteristics (such as prompt and output length), queue dynamics, and infrastructure and pricing conditions, including wholesale electricity price, using specialized forecasting models. Forecasts use only history up to the current period; there is no future-truth leakage into the decision.

Honest scope: several constraints are represented in the world model but are not yet consumed as planner-input forecasts. Among them are cache-reuse, emergent queue and SLA pressure, and carbon and network-congestion conditions. They are treated as out-of-scope inputs today rather than implied.

Candidate decisions are evaluated as coupled bundles across the active optimization surfaces Aurelius can actually score today. The planner only recommends decisions on wired levers, so the report does not imply support for surfaces that are merely modeled, experimental, or future work.

Because the surfaces interact, Aurelius evaluates coupled decision bundles rather than tuning one knob at a time, and which candidates it evaluates is itself a first-class step, not an afterthought. A physics-guided generator proposes a focused set of high-value coupled bundles implied by the operating regime, always including the known-strong candidates so a high-value option is never silently dropped. The set is then searched in a way that captures cross-surface coupling, expanding to more surfaces only when a decision is close and stopping early when the choice is clear.

Where exact enumeration is tractable, the bounded search is scored against it so any regret is measured rather than assumed, the discipline that surfaced the finding in §8.

The objective is SLA-safe goodput per operator dollar: the numerator counts requests that met their SLA deadline; the denominator is infrastructure cost, including energy at the prevailing market price and the cost of holding replicas warm and of migrations. A forecast-uncertainty risk penalty enters the same economic objective.

Gating is two-tier. Hard constraints (SLA, capacity, power, residency, policy) reject a candidate outright; the objective only ranks the feasible ones; and a Pareto gate accepts a recommendation only when it beats the baseline without regressing SLA. Every headline in this report passed that gate. When forecast confidence is low, Aurelius falls back deterministically to a safe default.

The finding behind this report is blunt: with the world model, the simulator, the objective, and the safety gate all unchanged, enabling more decision levers initially made results worse. The cause was not the controller; it was containment. The planner’s candidate set had collapsed to a single surface (the GPU clock), so the bundle that actually wins during an expensive window (lower precision + aggressive batching + a higher clock, which packs far more goodput per GPU-hour) was never in the set being evaluated. The optimizer cannot pick what it never sees.

Making candidate generation physics-guided, and searching the coupled set under a regret audit, removes that ceiling. On the same window, simply restoring the right candidate set recovers most of the gain; letting the search reach the coupled combinations the fixed grid misses recovers the rest, and a regret audit confirms it forfeits nothing the exhaustive search would have found.

The headline result replays the full selected benchmark windows with the per-period request limit removed: the same public serving trace and the same three independent electricity markets’ expensive price windows (PJM, ERCOT, CAISO), each running at its natural, production-scale volume of roughly 442k–577k requests across three control periods, about 1.5M requests in total. It is the full selected windows at full load, not a sample and not the entire lifetime of the public traces.

At that volume the naive SLA-aware baseline does not complete: its no-batching, no-admission replay cost grows super-linearly and times out, so it is reported as a reference only. Both the production-class scheduler baseline (§11) and Aurelius complete at full load, which makes Aurelius vs the production-class scheduler the primary, production-comparable comparison: two policies run the identical load, and the one built from real serving-stack components is the bar.

Across the three markets Aurelius delivers +724% higher SLA-safe goodput per dollar on average (per-market +698% / +718% / +755%; full values in §12), with SLA violations cut from ~3.8–4.6% to ~0.1–0.2% and ~84% fewer GPU-hours at the same served load. That is a Pareto win, not goodput bought by spending more.

Evaluation is deterministic simulated replay: a public production serving trace is replayed against a fixed harness with public wholesale-electricity price traces, and every policy is scored on the same windows under a single metric, SLA-safe goodput per operator dollar (gp/$): SLA-meeting requests in the numerator, and infrastructure cost (energy at the prevailing market price plus warm-hold and migration cost) in the denominator. No policy sees future arrivals or future prices; per-request execution time is used rather than predicted, so there is no oracle. The simulator, reward, cost model, and Pareto gate are held identical across every arm.

The headline is built bottom-up, never asserted:

1. Uncapped high-load replay. Each market's selected window is replayed with the per-period request cap removed, so it runs at its full production-scale volume (PJM ~577k, ERCOT ~443k, CAISO ~530k requests across three control periods).
2. Per-market gp/$. For each market we read Aurelius' gp/$ and the production-class scheduler's gp/$ directly from the run output, with no intermediate rounding.
3. Per-market percent delta. Computed straight from those two values as (Aurelius − production) ÷ production: PJM +698.1%, ERCOT +717.8%, CAISO +755.3%.
4. Arithmetic mean across markets. The +724% headline is the simple unweighted average of the three per-market deltas: (698.1 + 717.8 + 755.3) ÷ 3 = 723.7%, each market weighted equally, not pooled or request-weighted.

The naive SLA-aware baseline times out uncapped on all three markets and is therefore excluded from the headline; the production-class scheduler and Aurelius both complete, so the +724% is strictly an Aurelius-vs-production-class-scheduler comparison on identical load. Because the metric is a ratio of equally-weighted per-market deltas, the figure moves with the request volume, and is reported only with its uncapped, full-window scope attached.

The headline bar is a production-class scheduler baseline: a single, deterministic, causal heuristic that reacts to recent observable load and sets the serving-stack levers a real modern GPU-fleet deployment runs. It is not a strawman and it is not an Aurelius component; it sees no future prices, no future workload, and none of Aurelius’ economic, precision, clock, or migration arbitrage. It is assembled, in general terms, from the controls that production serving stacks and cluster schedulers actually expose:

• Continuous (iteration-level) batching, always on, the defining throughput feature of modern LLM serving.[1][2]
• SLA-aware request ordering with deadline protection, the discipline behind predictable, SLO-driven inference serving.[3]
• KV / prefix-cache-aware routing, so requests sharing a prefix reuse cached computation instead of recomputing it.[4]
• Topology-aware placement (rack/locality-aware), the standard way cluster schedulers keep communicating work on nearby hardware.[5]
• Backlog-driven autoscaling with a modest warm-pool headroom (a standard ~80%-utilization target), never under- or blanket-over-provisioning.[6][7]
• Class-based admission that defers best-effort work only under genuine pressure: overload control, not free shedding.[3]

Each lever maps to an established production technique, so the composite is as close to a deployed scheduler as a public replay allows: it is deliberately stronger than the naive SLA-aware policy, and it runs the deployed model as-is. The economic arbitrage Aurelius adds on top is the only difference being measured; the baseline is what a competent operator already runs, not a handicapped reference.

One integrity rule governs every comparison: a baseline that violates SLA on a meaningful fraction of requests is not a valid SLA-safe baseline, and a delta earned by quietly dropping completions is excluded from any headline. The production-class scheduler holds SLA (uncapped violation rate ~3.8–4.6%), so the +724% is a real goodput-per-dollar gap, not completions shed to flatter the ratio.

gp/$ and SLA-violation rates are read directly from the run output; the SLA-aware baseline timed out uncapped and is not shown. Δ % is per-market (Aurelius − production) ÷ production; the +724% headline is the average of the three (723.7%, shown unrounded).

GPU-hours are reported by each run for the same served requests, so the reduction is computed per market as (production − Aurelius) ÷ production and totals to −84.5%, the ~84% fewer GPU-hours quoted in the headline. This GPU-hours figure belongs to the uncapped high-load replay and must not be mixed with the Benchmark V1 continuity number in Appendix A.

Because an approximate search could quietly leave value on the table, its loss is measured wherever exhaustive enumeration is feasible. There the bounded coupled-bundle search shows zero regret against the exhaustive ground truth in all three markets, at roughly 40 evaluations per decision; it finds the exact safe optimum the exhaustive search finds, far more cheaply. The remaining gap to a forecast oracle (~4.5%) is forecast quality, not search: with search solved, request-characteristic forecasting is the next highest-value lever.

Aurelius reads only the metadata a scheduler already exposes (job timing, resource requests, constraints), never payloads or model outputs. Candidate scoring runs read-only; unsafe candidates are rejected at the constraint gate before execution; and when confidence is low the controller falls back deterministically. The path from telemetry to any production change is staged and reversible, and constraints are checked before a recommendation exists:

• The headline is a simulated, uncapped replay of selected public-trace windows: evidence of achievable savings, not a guarantee for any specific fleet, and not a production deployment.
• The +724% figure is load-dependent: part of the gap is a fixed-policy production scheduler degrading under heavy uncapped load while Aurelius adapts, so it is reported only with its uncapped, full-window scope, never as a fleet constant.
• Magnitudes are simulator-inferred and depend on the serving model's precision/batching bands; the robust findings are the direction and the zero search regret.
• It replays the selected benchmark windows across three markets, not the entire lifetime of the public traces and not a long-horizon production deployment.
• Simulator fidelity must still be tested against real operator telemetry, the only way to isolate model error.
• An aggressive low-precision mode can score higher but is excluded from every headline because its quality risk is not yet modeled.
• The largest gains appear during expensive electricity windows; quiet, cheap-power periods leave less to recover.
• Some surfaces are modeled but not yet active production levers, and several constraints are represented but not yet forecast inputs.

With search no longer the bottleneck, the residual gap to a forecast oracle points the roadmap at forecast quality first: better request-characteristic and arrival forecasting is the largest measured lever. Beyond that: validating world-model fidelity against real operator telemetry, extending the result across more windows, markets, and workloads, promoting modeled-but-not-yet-optimized surfaces into active levers, and modeling precision quality so the aggressive low-precision ceiling can be claimed safely rather than only as a diagnostic.

Replay your historical scheduler telemetry, simulate the counterfactual outcomes, and produce an audited savings estimate, without production changes or payload access. Aurelius starts from read-only scheduler metadata and never touches payloads or model outputs.

Benchmark V1 is the prior, frozen, cap-controlled harness: a small fixed per-period request cap, scored against the strongest SLA-aware baseline. It is a different baseline and a different load from the uncapped result in §9, so the two are reported separately and never averaged together. The numbers below are unchanged from the prior published report, retained for continuity and to support the candidate-generation finding in Appendix B.

The mechanism behind the headline is clearest on Benchmark V1, where exhaustive enumeration is tractable and the gain can be attributed exactly. The same mechanism drives the uncapped result in §9; it simply has more load to work against there.

The decomposition in Table A3 tells the whole story. Restoring the right candidate set, without touching the search, already turns a regression into a large Pareto-safe gain, because the moment the high-value bundle is in the set, the unchanged simulator picks it. Letting the search reach the coupled combinations a fixed grid does not contain adds the rest, and it does so while matching the exhaustive optimum exactly. The prior single-surface set, by contrast, forfeited most of the achievable goodput and never held SLA.

Because the simulator, reward, cost model, and Pareto gate were unchanged, the gain is attributable to search and candidate generation, not to a model that was quietly made more optimistic. The zero-regret audit against the exhaustive ground truth is the proof.

This documents how the public technical report evolved, not every internal experiment. Each revision records a methodological or structural change visible in the report, reconstructed from the repository’s commit history; dates are actual revision dates.

The production-class scheduler baseline (§11) is assembled from established production serving and cluster-scheduling techniques. These are the primary sources for each component; they are cited for the realism of the baseline’s mechanisms, not for any Aurelius result.

1. [1]Yu et al. “Orca: A Distributed Serving System for Transformer-Based Generative Models.” OSDI 2022. Iteration-level (continuous) batching. usenix.org
2. [2]Kwon et al. “Efficient Memory Management for Large Language Model Serving with PagedAttention” (vLLM). SOSP 2023. Continuous batching and KV-cache management. dl.acm.org
3. [3]Gujarati et al. “Serving DNNs like Clockwork: Performance Predictability from the Bottom Up.” OSDI 2020. SLO-aware scheduling and admission for predictable serving. usenix.org
4. [4]Zheng et al. “SGLang: Efficient Execution of Structured Language Model Programs” (RadixAttention). 2024. Prefix/KV-cache-aware scheduling and reuse. arxiv.org
5. [5]SchedMD. “Slurm Workload Manager, Topology Guide.” Topology-aware (switch/locality) placement in production cluster schedulers. slurm.schedmd.com
6. [6]Kubernetes. “Horizontal Pod Autoscaling.” Utilization-target autoscaling, the standard reactive capacity control. kubernetes.io
7. [7]Rzadca et al. “Autopilot: Workload Autoscaling at Google.” EuroSys 2020. Autoscaling with utilization headroom at production scale. dl.acm.org