ModelsGitHub

How the Engine Works

From an addressable model container to a bounded, storage-native execution plan.

Storage is an input to planning

Most local inference systems begin by loading or mapping a checkpoint and then react when memory becomes scarce. Minirun begins earlier. A supported model adapter knows the logical regions used by each stage, while the container identifies where those regions live on storage. The planner combines that information with an explicit memory budget before execution starts.

The resulting plan names the minimum working set, resident regions, staged reads, model-specific cache allowance, and unused headroom. If those terms cannot be represented inside the budget, the run is refused.

How weights reach the GPU in MinirunTwo paths leave an SSD. On the upper path, container tiles are read a few at a time, cross the memory line into a bounded window sized by the memory budget, and continue to MLX. On the lower path, a request to load the whole model at once widens as it travels and stops at the memory line without crossing it.SSDcontainer tilesmemoryone tile, one readbudget you setheld, then releasedMLXload everythingstops here
Tiles are read from the drive a few at a time and pass through a window whose size you chose. The second track is the conventional arrangement — load the model, then run it — and it stops at the memory line. The drawing carries no numbers: it is the shape of the idea, not a timing trace.

The data path

Conceptual engine path
01Verified container

Addresses model regions and binds the local bytes.

02Memory plan

Admits the run and assigns resident, staged, and streamed work.

03Storage scheduler

Issues exact reads and stages them inside the budget.

04Model adapter + MLX

Consumes regions in architecture-specific order.

05Release or retain

Frees completed regions or keeps planned state resident.

This is a conceptual diagram, not a timing trace. Each model family supplies the architecture-specific adapter that makes the generic path concrete.

Addressable and verifiable weights

Container metadata maps logical model regions to checked file ranges. A matching full verification result authorizes the local container for the current publication and filesystem identity. The detailed inventory and evidence rules are defined in the container guide.

Overlap without breaking the ceiling

When the next region is knowable, the scheduler can read it while the current region computes. The staging buffer is charged to the same memory plan. After consumption, the region is released or retained according to that plan.

Generic core, model-specific execution

Storage safety, bounded reads, planning primitives, and lifecycle rules are generic. Tensor order, routed experts, recurrent state, tokenizer behavior, and generation semantics belong to the model adapter and runner. This boundary keeps one model's assumptions from leaking into every container or into the storage layer.

New model families therefore need an explicit product binding rather than a name added to a menu. The binding must understand the container, validate its runtime metadata, assemble the supported input format, execute with bounded memory, and report honest lifecycle and measurement events.

What Minirun does not do

  • It does not treat operating-system swap as the execution strategy.
  • It does not make an unknown model runnable through an unbounded generic fallback.
  • It does not infer verification from a repository name or a path that used to contain valid bytes.
  • It does not present an unavailable measurement as a measured zero.