All models
Google
Gemma 4 26B A4B
The mixture-of-experts Gemma 4. Nearly the memory of the 31B but with only 3.8B active parameters, so it serves many more tokens per second.
moechatvisionfast
Overview
Gemma 4 26B A4B uses 128 experts plus one shared expert, with 8 active per token, for 3.8B active parameters out of 26B. Google describes it as running almost as fast as a 4B model.
Hardware fit
MXFP4 and UD-Q4_K_XL are around 17 GB, and Q8_0 is 27.3 GB. It is a good choice for high-traffic chat on 24 to 48 GB cards.
| Quantization | Weights |
|---|---|
| MXFP4 | 16.6 GB |
| UD-Q4_K_XL | 17.0 GB |
| Q8_0 | 27.3 GB |
Sizes are the GGUF file sizes from unsloth/gemma-4-26B-A4B-it-GGUF. Budget extra VRAM for the KV cache, which grows with context size and parallel requests.
Deploying on Loadcore
- In the console, open Models and search for
unsloth/gemma-4-26B-A4B-it-GGUF. - Pick the quantization that fits the target server and start the download.
- Start an instance with the context size you need and GPU layers set to offload the full model.
- Point your tools at
http://your-server:9090/v1with your personal key.
Recommended settings
| Parameter | Value |
|---|---|
| temperature | 1.0 |
| top_p | 0.95 |
| top_k | 64 |
| repeat_penalty | 1.0 |
| min_p | 0.0 |
Same recommended defaults as Gemma 4 31B.
Source: google/gemma-4-26B-A4B-it.