SSD Streaming

Not swap, not accidental paging

SSD Streaming is explicit model I/O. The runtime knows which region a stage will need and requests that region from the container. The operating system is not asked to discover the model's access pattern after memory pressure has already occurred.

Because the schedule is model-aware, Minirun can distinguish weights that are used in a fixed sequence from regions selected dynamically, such as routed experts. Each path has its own bounded prefetcher, suited to what is knowable before compute reaches it.

Deterministic read-ahead

For weights consumed in a fixed order. It stages one region ahead — depth 1 — into a double buffer wherever a budget has been stated, because the staging space is charged to that budget and a library cannot invent one.

Expert prefetch

For routed-expert reads, which are only knowable once routing has chosen them. It is on by default and schedules the chosen reads rather than issuing each one urgently and waiting.

The moving window

At any point, memory contains the model state required to make progress, regions intentionally kept resident, and a limited amount of staged I/O. As compute advances, completed streamed regions leave that window and later regions enter it.

When overlap helps

A read is hidden only when it actually overlaps useful compute and finishes before the consumer needs it. Starting more I/O is not automatically faster: queue depth, storage latency, compute duration, memory reserve, and the model's access pattern all affect the result.

Instruments reports overlap from the current run when the runtime has sufficient read and compute boundaries. The result belongs to that device, model, storage path, and run configuration. It should not be copied into a universal product claim.

What determines local performance

Model architecture

Determines region order, active experts, compute intensity, and reuse.

Container layout

Controls whether required regions can be read precisely and efficiently.

Memory budget

Changes residency, staging capacity, and model-specific cache decisions.

Storage path

Changes latency, sustained reads, and available overlap.

Prompt and generation

Change prefill work, token count, and model routing behavior.

Device conditions

Competing load and thermal state can change both compute and storage behavior.