Early tester loktar00 revealed on X that a new open-source model called Hy4-Flash can run locally, using a MoE architecture with 224B total parameters and only 12B active parameters. The tester has early access and has completed several tests, including a single-pass generation using the prompt "bees on a sunflower" that took 32 minutes. He called the performance "absolutely insane" for a model of this size, said the overall quality surprised him, and promised to release more test results later.
INT4 optimization lands: int8-vllm's first INT4 model
Meanwhile, the int8-vllm project announced it has added optimized support for Hy4-Flash, marking its first INT4 model. According to the project, at INT4 precision the model fits fully into 128GB of VRAM with about 1.3 million tokens of shared context space left over. The INT4 checkpoint was produced using NeUQI calibration and has been released on Hugging Face.
In benchmarks, on a 4x MI100, C8 configuration, tg128 reached 222.4 tok/s and pp512 reached 1655 tok/s; with concurrency 8 and 2048 context, tg256 was 96.7 tok/s. Another user ran Qwen3.8 27B on 2x MI210 with decoding speed above 1000 tok/s.
| Configuration | Metric | Value |
|---|---|---|
| 4x MI100, C8 | tg128 | 222.4 tok/s |
| 4x MI100, C8 | pp512 | 1655 tok/s |
| Concurrency 8, 2048 context | tg256 | 96.7 tok/s |
| 2x MI210 | Qwen3.8 27B decode speed | Above 1000 tok/s |
Targeting older AMD cards: the value proposition of used MI100s
The int8-vllm project focuses on older AMD cards, covering MI100, MI210, MI50, MI60 and RDNA1-3 architectures, and provides docker compose configurations. The author also made the MI100's vector ALUs and matrix cores run at full utilization simultaneously by deferring and merging epilogue work into subsequent fused kernels, delivering a 15% gain on most GEMMs; W4A8 GEMM support has been added to the companion AITER fork.
The author notes that used MI100s currently cost about $1,300. At the same price, new cards offer roughly double the compute but half the memory bandwidth, giving the MI100 a unique price-performance edge as an inference card.
That view offers budget-constrained local deployment users a new hardware reference point.
What to watch
Full evaluation data for Hy4-Flash has not yet been widely released. The additional test results loktar00 has promised, the model's real-world performance on more hardware configurations, and the INT4 version's stability in long-context scenarios all warrant continued attention.
