Inkling-2 Mini, a roughly 25B-parameter model released by thinkymachines, has sparked hands-on testing in the local AI community. One user ran the model on a single RTX 3090 using the Unsloth UD-Q4_K_XL quantization and said it outperforms larger MoE models. Meanwhile, the inference engine capy-flash v0.16 added support for the model, running 254k context on two 16GB 50-series cards and publishing performance benchmarks.
Single-card RTX 3090 test and generation demo
According to X user LiTianleli, Inkling-2 Mini has about 25B parameters and runs on a single RTX 3090 after Unsloth UD-Q4_K_XL quantization. The user claims it outperforms larger MoE models. Another user used the Q4 quantized version to generate a complex 3D floating-island scene with a windmill, grass and a cabin from pure code with no external assets, showing lightweight usability and generation capability close to that of 3B-class local models.
capy-flash v0.16 runs 254k context on two cards
Reddit user FeydRowan released capy-flash v0.16, which adds Inkling 2 Mini support to the inference engine; it previously supported Qwen3.8-27B and Qwen3.8-Flash-Next. The release is optimized for tensor parallelism across two 16GB Blackwell cards and runs on Linux; a single 5090 and Windows WSL2/Docker are theoretically usable but have not been tested.
On a 2×RTX 5070 Ti 16GB configuration, the benchmark results are as follows:
| Test | Value |
|---|---|
| Chat decode | 149 tok/s |
| Code decode | 194 tok/s |
| 4k prompt prefill | 5,032 tok/s |
| 250k context decode | 53 tok/s |
| 250k context prefill | 1,413 tok/s |
The full 254k context fits in VRAM, but because the original NVFP4 weights and activations are kept unquantized, the KV cache is also unquantized, and the model uses global attention, it cannot reach 1M context. Long-context degradation is pronounced: at 250k, decode drops to 53 tok/s and prefill to 1,413 tok/s.
The author plans to measure current performance first, then consider targeted quantization for speed. Both code and weights are open source.
Community impact and what to watch
Together, the two threads show that the cost barrier to local deployment of small models is falling. A single RTX 3090 can run a roughly 25B-parameter model, and two 16GB 50-series cards can support 254k context, which is practical value for the local AI community. What to watch next: the capy-flash author's quantization plan to address long-context degradation, and more user test reports on different hardware configurations.
