NewYour coding agent can read the release notes before it upgrades.Set up the MCP server →

v0.26.129 Aug 2026

refactor: Stop two parse panics, refuse a typeless projector, and cor…

refactor: Stop two parse panics, refuse a typeless projector, and cor…

…rect three graph shapes (#49)

* Return zero bytes for a tensor with an empty dimension

GGUFTensorInfo.Bytes subtracts 1 from each dimension to compute the
strides. A zero-length dimension is unsigned, so the subtraction wraps to
2^64-1 and the overflow guard refuses the multiplication, which panics the
parse. ggml_nbytes returns 0 for such a tensor before it reaches the same
arithmetic.

Return 0 when any dimension is zero.

mudler/face-detect-gguf/antelopev2.gguf holds one such tensor out of 748,
det.392 with dimensions [0]. It now parses.

* Refuse the files llama.cpp refuses to load

Two kinds of file panicked the parser or returned a confident estimate for
a model that cannot start. Both are now refused while the file is parsed,
so the caller gets an error.

general.file_type written as a string panicked in ValueNumeric, which a
caller cannot catch. llama.cpp reads the key as a uint32 and throws "key
general.file_type has wrong type". Four Serveurperso repositories publish
files on this schema: ACE-Step, MiniMax-Music3, OmniVoice and Qwen3-TTS.

A projector that names no type was reported as an mlp projector. llama.cpp
resolves the type from clip.projector_type, then from the key of the
modality it is loading, and refuses the load with "unknown projector type"
when neither names one. mys/ggml_llava-v1.5-7b carries no such key and was
estimated at 6866 MiB with no warning; the same is true of every mmproj on
the format that predates the key.

The fallback chain now also reads clip.gen.audio.projector_type, so a
projector that names its type only there is neither refused nor reported
as mlp. The invented mlp default is gone: a parsed clip file always names
its own type.

ValueNumeric's type list moves to GGUFMetadataValueType.IsNumeric so the
parse check and the accessor agree on what a number is.

* Charge RWKV7's decay projection as the two-dimensional node it is

RWKV6 and RWKV7 both name a tensor "time_mix_w2". RWKV6 reshapes it to
[ne0, ne1, 1, 5] and multiplies the five-way lerp stack, so the node is
four-dimensional and its last dimension is that fan-out. RWKV7 gives the
name to a plain decay projection whose node is [n_embd, n_tokens]. The
estimate applied RWKV6's shape to both, and RWKV7's last dimension is the
embedding length rather than a small fan-out, so the node was charged
n_embd times over: 8192 MiB of a 8216 MiB compute graph on a 1.5B model.

Size the node by architecture.

Measured on an A40 with llama-server, context 2304, one sequence, flash
attention, q8_0 key and value cache. Device total, estimate against the
figure llama.cpp reports for itself, in MiB:

rwkv7-g1c-1.5b 9710 -> 1523, really uses 1562
ARWKV-7B-Preview 33085 -> 8004, really uses 8089
rwkv-6-world-1.6b 1604 unchanged, really uses 1600
Nemotron3-Nano-4B 2636 unchanged, really uses 2636

Both RWKV6 and the other recurrent architectures are byte-identical: the
new branch is reached only by rwkv7 and arwkv7, which share the same graph
builder in llama.cpp.

* Charge gemma3's vision encoder over the grid it attends to

gemma3 pools its 64x64 patch grid 4x4 into 256 projector tokens. The
pooling runs after the vision transformer, so the encoder still attends
over all 4096 patches, and its attention term is quadratic in that count.
The estimate used the projector's output token count for both, which
understates the attention scores by the square of the pooling factor.

The estimate already carries the un-pooled position count, but only on the
path that handles a projector type it does not recognise. Set it in the
gemma3 case as well.

Measured on an A40 with llama-server, context 4096, one sequence. The
projector's own device total, estimate against llama.cpp's own figure, in
MiB:

flash attention off 831 -> 1918, really uses 1944
flash attention on 827 -> 926, really uses 933

This is also what made flash attention look free for a projector: the
switch is worth 1011 MiB on gemma3 and was modelled as 4, because the term
it halves was computed over 256 positions instead of 4096. The estimate now
moves 992 MiB across the same switch.

idefics3, llama4 and internvl reduce their patch count the same way and are
left alone, because they are not measured here. Every other projector type
is byte-identical.

* Reserve a dynamic-resolution projector for llama.cpp's warm-up image

The estimate guessed the largest image a caller might send, defaulting to
1024 pixels a side. llama.cpp reserves the projector's graph for one square
warm-up image instead, and derives its side from a token count carried in
its own source rather than from anything the file declares:

warmup_image_size = sqrt(n_tokens) * patch_size * n_merge

The Qwen vision family warms up at 46*46 tokens and pixtral at 256, so the
encoder attends over 8464 positions on the first and 256 on the second,
against the 5476 and 4096 the guess produced. The attention term is
quadratic in that count, which is why three families read low and pixtral
read about seventy times high: one wrong assumption, not four errors.

Carry llama.cpp's token counts and size the warm-up image the same way.

llama.cpp prints the size it uses, so the rule is checked against its own
figure rather than inferred from the buffer. Measured on an A40:

gemma3 896 x 896, the declared image size, unchanged
qwen2vl 1288 x 1288 = 46 * 14 * 2
qwen2.5vl 1288 x 1288 = 46 * 14 * 2
pixtral 256 x 256 = 16 * 16 * 1

The projector's own compute buffer, estimate against llama.cpp's own
reserve_compute_meta figure, context 4096, one sequence, flash attention
off then on, in MiB:

qwen2vl 1937 / 164 -> 4567 / 331, really uses 4598 / 267
qwen2.5vl 1937 / 164 -> 4840 / 605, really uses 4872 / 705
qwen3vl 360 / 46 -> 4538 / 302, really uses 4728 / 322
pixtral 1121 / 98 -> 11 / 6, really uses 16 / 16

The flash-attention-off column governs whether a model fits, and it lands
within 4 percent on the three Qwen families. Flash attention on stays
looser, and pixtral is now low by 5 MiB rather than high by 1105.

Only these four projector types are affected. Every other type reserves for
the image size it declares, which is already what the estimate uses, and
gemma3, SmolVLM, InternVL3, MiniCPM-V and ultravox were measured byte for
byte unchanged.

Open source →

Additional notes

v0.26.1

Compare

Choose a tag to compare

Open source →