GPU Parity Engine
One model graph. Two vendors. Native performance on both AMD and NVIDIA. No platform-specific rewrites. No inference-time compilation divergence. Same graph, same results, same speed across the stack.
Benchmarked against llama.cpp (b10791) on real hardware — NVIDIA Blackwell, AMD Radeon, and AMD Ryzen CPU — each engine measured on the same box, back to back. One graph, ported once, running natively on every one.
SPIR-V intermediate representation means the same computation graph compiles natively to both AMD RDNA and NVIDIA CUDA. No per-vendor rewrites. No specialized kernels. One truth.
Model graphs in SPIR-V, not vendor-specific bytecode. Compile once, deploy anywhere.
Each vendor compiles to native ISA at deployment time, reaching hardware limits.
Same code path on AMD and NVIDIA—no hidden differences, no vendor-specific bugs.
Side-by-side benchmark comparison: Demerzel's performance against the llama.cpp baseline on AMD and NVIDIA hardware.
RC4 lands the win we were chasing. Demerzel already matched llama.cpp on a single request — now it pulls decisively ahead under real serving load, most of all on the large Mixture‑of‑Experts models real deployments run. Every number below is measured on real hardware, both engines back‑to‑back, same model files.
| Processor | Demerzel pp | llama pp | Demerzel tg | llama tg | ||
|---|---|---|---|---|---|---|
| Intel Xeon E-2186MCoffee Lake · 6C/12T | 130.8 | 113.2 | 116% | 32.5 | 35.2 | 92% |
| AMD Ryzen 9 6900HXZen 3+ · 8C/16T | 359.9 | 270.0 | 133% | 65.9 | 67.2 | 98% |
| AMD Ryzen 7 5700UZen 3 · 8C/16T | 233.4 | 201.2 | 116% | 41.9 | 35.5 | 118% |
| Device | Model | Demerzel pp | llama pp | Demerzel tg | llama tg | ||
|---|---|---|---|---|---|---|---|
| AMD Radeon Vega 8Renoir · integrated · Vulkan | TinyLlama | 647.1 | 485.5 | 133% | 56.4 | 54.0 | 104% |
| AMD Radeon 680MRembrandt · integrated · Vulkan | TinyLlama | 1374 | 1470 | 93% | 89.5 | 88.6 | 101% |
| NVIDIA Quadro P4200Pascal · 8 GB · Vulkan | TinyLlama | 1526.9 | 1367.0 | 112% | 159.8 | 167.7 | 95% |
| NVIDIA GB10Grace-Blackwell · CUDA | Qwen-7B | 3927 | 3333 | 118% | 47.5 | 50.8 | 93% |
When agents arrive at once Demerzel reads each weight once for the whole batch — batching built in, one binary, nothing to bolt on. Run against llama‑server with equal --parallel slots, same box, same model, same prompts, every response coherence‑checked:
| Llama‑3.2‑3B · AMD Renoir iGPU (Vulkan) | Demerzel | llama‑server | |
|---|---|---|---|
| 8 concurrent agents | 47.4 tok/s | 36.5 | +30% |
| 16 concurrent agents | 71.0 tok/s | 48.6 | +46% |
Demerzel leads where it counts — the 8–16‑agent band a single node actually serves — and both converge at the box’s memory‑bandwidth ceiling above that. One engine, batched by default; no separate serving program to stand up.
The regime that pays off most: large Mixture‑of‑Experts models under concurrency, where reading each expert once for the whole batch amortizes hardest. Demerzel batched decode vs llama‑server (equal slots, flash‑attention on), aggregate tok/s, same box & model files — re‑measured 2026‑10‑06 on both GB10 cards with a byte‑identical binary:
| Model · GB10 CUDA | 8 agents | 16 agents | 24 agents | 32 agents |
|---|---|---|---|---|
| Qwen3‑30B‑A3BQ4 · mixture‑of‑experts | +30% | +29% | +12% | +4% |
| Coder‑30BQ6 · mixture‑of‑experts | +31% | +24% | tie | tie |
| Qwen‑7Bdense | −8% | +8% | tie | +2% |
Qwen3‑30B‑A3B leads the whole 8–32 band; both 30B mixtures win big at the 8–16 agents a node actually serves. Above 32 the gap closes to llama‑server’s high‑density matmul tuning. One large dense model under heavy concurrency is still behind — the open serving lever we’re on next.
Live and answering in ~0.38 s — llama‑server takes 1.6 s warm, 5.3 s cold.
First token in 1.55 s vs 3.2 s — 2× sooner. Demerzel mmaps and loads lazily; the alternative is still building graphs. For model‑swapping, autoscaling and serverless cold‑starts, that is the whole game.
| Device / model | Result |
|---|---|
| 2× AMD RX 470 · Qwen3-8B8 GB each | Runs split across the pair. An 8-billion model fits no single 8 GB card — llama included — so on one card it simply doesn't load; Demerzel tiles the layers across both. |
| AMD Renoir iGPU · Qwen-7B2.67 GB VRAM · 4.7 GB model | Serves a model 1.75× its VRAM. The AGNUS pager streams cold weights from host RAM so a 4.7 GB model runs on a 2.67 GB card — a capability, on hardware where a manual layer-split isn't an option. |
| AMD Ryzen 7 · Qwen3-30BCPU | Runs the 30-billion-parameter mixture on a laptop CPU, with byte-identical output to llama.cpp — bit-for-bit correctness on a model most single machines can't hold. |