Instella-MoE

AMD🇺🇸 USMajor lab· Jul 23, 2026

Textopen weights

INT4 weight-only quantized (W4A16) SFT variant for CPU inference on AMD EPYC with vLLM and ZenDNN. Only dense and shared-expert layers are quantized, while routed experts stay BF16, so the size drops only from 29.5 to 28.9 GiB.

Summary of the model card

License: other
Modalities: text → text

8 variants

16B A3B SFT W4A16 Llmcompressor	16B-A3B	HF
16B A3B SFT W8A8 Llmcompressor	16B-A3B	HF
16B A3B Think	16B-A3B	HF
16B A3B SFT	16B-A3B	HF
16B A3B Pretrain	16B-A3B	HF
16B A3B Midtrain	16B-A3B	HF
16B A3B DPO	16B-A3B	HF
16B A3B Base	16B-A3B	HF

Sources

HuggingFace Paper

Deutsch

INT4-gewichtsquantisierte Variante (W4A16) des SFT-Modells für CPU-Inferenz auf AMD EPYC mit vLLM und ZenDNN. Quantisiert sind nur Dense- und Shared-Expert-Schichten, die gerouteten Experten bleiben BF16, daher sinkt die Größe nur von 29,5 auf 28,9 GiB.

Provenance

Detected: Sep 27, 2026
First source: huggingface
Description: Summary of the model card
Editorially reviewed: —