← Back

DeepSeek-V4.1-Flash-1017 on AMD Instinct GPUs: 2.27× faster code generation per user with megakernels, no accuracy loss

Same weights, only the decode kernel changed: up to 2.27× faster per user on AMD Instinct MI325X, with no accuracy loss. Here is how we verify a megakernel before it goes into our serving stack.

Many small binary cubes fusing into one large block, a megakernel replacing many kernel launches

<p>We serve DeepSeek-V4.1-Flash-1017 on AMD Instinct MI355X and MI325X GPUs. On both, the decode step now runs through a megakernel: AMD's optimized megakernel on MI355X, and a separate megakernel optimized for MI325X. A faster kernel only goes into our serving stack if the model still answers the same way. This post walks through how we verify a megakernel, and what the checks showed.</p><p>On MI325X, with the same weights and only the decode kernel changed, P90 output tok/s per user on code went from 192.5 to 436.3 at one concurrent user: a 2.27× speedup. Accuracy evaluations on the same configuration showed no loss.</p><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc042428e53b79c3c46d0a_megakernel-article-hero.png" loading="lazy" alt="Megakernels on AMD Instinct MI325X: 2.27× faster code per user, no accuracy loss"></div></figure><h2>How we verify a megakernel</h2><ol><li><strong>Isolate the kernel.</strong> We serve the same DeepSeek-V4.1-Flash-1017 weights twice, once with the previous decode kernels and once with the megakernel. Nothing else changes, so any difference in speed is the kernel.</li><li><strong>Keep everything else the same.</strong> Both runs use the same GPUs, the same workload and the same benchmark harness, and run against the serving stack the kernel would go into.</li><li><strong>Measure at one user.</strong> A decode megakernel targets the time per output token. We report P90 output tok/s per user, 1000 divided by the P90 time per output token, at one concurrent user, where decode latency sets the speed a user sees.</li><li><strong>Repeat across workloads.</strong> Code, math, writing and general Q&amp;A, so the gain does not depend on one kind of prompt.</li><li><strong>Check accuracy on the build we benchmarked.</strong> Accuracy evaluations run on the same configuration that produced the speed numbers, on a separate copy of the deployment so the evaluation never slows the speed run. Often the evaluation runs first, and the speed test only happens if accuracy holds.</li></ol><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc04258143f337ff3fd893_megakernel-article-fig-01.png" loading="lazy" alt="Three checks every megakernel passes before it enters our serving stack"></div></figure><h2>Results on MI325X: megakernels, kernel alone</h2><p>On 4 MI325X GPUs, with the same weights and only the decode kernel changed, megakernels are 2.14× to 2.27× faster per user across all four workloads at one concurrent user:</p><ul><li><strong>Code</strong> (internal dataset built on RepoBench): 192.5 to 436.3 P90 output tok/s per user, 2.27×</li><li><strong>Math</strong> (internal dataset built on OlympiadBench): 187.4 to 423.1, 2.26×</li><li><strong>Writing</strong> (internal dataset built on WritingBench): 162.4 to 354.7, 2.18×</li><li><strong>General Q&amp;A</strong> (internal dataset built on MMLU-Pro): 181.2 to 388.3, 2.14×</li></ul><p>For reference, the setup we served before, with older weights and kernels, measured 200.6 on code.</p><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc0426195b1dcaf4a33dc0_megakernel-article-fig-02.png" loading="lazy" alt="On MI325X, megakernels are 2.14× to 2.27× faster per user across four workloads"></div></figure><h2>Throughput at every load</h2><p>A faster single-user path only helps if it does not cost throughput under load. It does not. On 4 MI325X GPUs with the same weights, total tok/s is higher with megakernels at every load we measured: 2.07× at 1 user, 1.66× at 4 users, then 1.10× to 1.17× from 8 to 64 users. P90 output tok/s per user is also higher at every one of those loads.</p><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc0426981a711feca57060_megakernel-article-fig-03.png" loading="lazy" alt="Megakernels raise MI325X throughput at every load we measured, up to 2.07×"></div></figure><h2>Results on MI355X: AMD's optimized megakernel in our stack</h2><p>On MI355X our new serving stack pairs AMD's optimized megakernel with updated weights and a longer speculative draft. On 4 GPUs at one concurrent user it decodes at 487.3 P90 output tok/s per user, against 333.3 for our previous serving stack: 1.46×. On 2 GPUs it reaches 434.9, 1.30× our previous serving stack on 4 GPUs. This comparison covers the whole serving stack; the MI325X runs above are the kernel-alone test.</p><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc04261ef05c67b079cfd6_megakernel-article-fig-04.png" loading="lazy" alt="MI355X with AMD's optimized megakernel: 487.3 tok/s per user, 1.46× our previous stack"></div></figure><h2>What the megakernel does</h2><p>In standard LLM serving, each forward pass is split into many small kernels that the GPU launches one after another, and every launch adds time to each decode step. A megakernel fuses that work into a single kernel launch. On both MI355X and MI325X, the megakernels we serve fuse large sections of DeepSeek-V4.1-Flash-1017 (MoE, attention and allreduce) that previously ran as separate kernels, removing the cost of launching each one. The result is faster tokens per user: 2.27× on MI325X with the same weights.</p><figure class="w-richtext-align-fullwidth w-richtext-figure-type-image"><div><img src="https://cdn.prod.website-files.com/69983d45217658e64b234512/6adc0427c4a4351de791e87d_megakernel-article-fig-05.png" loading="lazy" alt="Megakernels turn many kernel launches into one, so tokens come out faster"></div></figure><h2>Try it</h2><p>DeepSeek-V4.1-Flash runs on Sciforium, on our own AMD hardware. <a href="https://console.sciforium.com/signup">Get Started</a> at <a href="https://sciforium.ai">sciforium.ai</a>, or <a href="https://www.sciforium.ai/contact">Contact Sales</a> to talk through your workload.</p><p><em>Test details: DeepSeek-V4.1-Flash-1017. Code: internal dataset built on RepoBench (write tests for given functions; find and fix errors with an explanation). Math: internal dataset built on OlympiadBench. Writing: internal dataset built on WritingBench. General Q&amp;A: internal dataset built on MMLU-Pro. 64K context, with 98% of each prompt served from the prefix cache. One concurrent user, 8 requests per point. P90 output tok/s per user is 1000 divided by the P90 time per output token. MI355X runs on October 20, 2026; MI325X runs in October 2026.</em></p>