Supported Models - SGLang Documentation

This section describes the models supported on the Ascend NPU, including Large Language Models, Multimodal Language Models, Diffusion Language Models, Embedding Models, Reward Models and Rerank Models. Mainstream DeepSeek/Qwen/GLM series are included. You are welcome to enable various models based on your business requirements.

Large Language Models

Models	Model Family	A2 Supported	A3 Supported
Eco-Tech/DeepSeek-V4-Pro-w4a8-mtp	DeepSeek	✅	✅
Eco-Tech/DeepSeek-V4-Flash-w8a8-mtp	DeepSeek	✅	✅
sgl-npu/DeepSeek-V3.1-w8a8	DeepSeek	✅	✅
sgl-npu/DeepSeek-V3.2-W8A8	DeepSeek	✅	✅
sgl-npu/DeepSeek-R1-0528-W8A8	DeepSeek	✅	✅
sgl-npu/DeepSeek-V2-Lite-W8A8	DeepSeek	✅	✅
Qwen/Qwen3.8-2.4T-A95B	Qwen3.8	✅	✅
Qwen/Qwen3.6-35B-A3B	Qwen3.6	✅	✅
Eco-Tech/Qwen3.6-27B-w8a8	Qwen3.6	✅	✅
Eco-Tech/Qwen3.5-397B-A17B-w4a8-mtp	Qwen3.5	✅	✅
Eco-Tech/Qwen3.5-122B-A10B-w8a8-mtp	Qwen3.5	✅	✅
Eco-Tech/Qwen3.5-35B-A3B-w8a8-mtp	Qwen3.5	✅	✅
Eco-Tech/Qwen3.5-27B-w8a8-mtp	Qwen3.5	✅	✅
Qwen/Qwen3.5-9B	Qwen3.5	✅	✅
Qwen/Qwen3.5-4B	Qwen3.5	✅	✅
Qwen/Qwen3.5-0.8B	Qwen3.5	✅	✅
Qwen/Qwen3-30B-A3B-Instruct-2507	Qwen3	✅	✅
Qwen/Qwen3-32B	Qwen3	✅	✅
Qwen/Qwen3-0.6B	Qwen3	✅	✅
sgl-npu/Qwen3-235B-A22B-W8A8	Qwen3	✅	✅
Qwen/Qwen3-Next-80B-A3B-Instruct	Qwen3	✅	✅
Eco-Tech/Qwen3-Coder-480B-A35B-Instruct-w8a8-QuaRot	Qwen3	✅	✅
Qwen/Qwen2.5-7B-Instruct	Qwen2.5	✅	✅
sgl-npu/QwQ-32B-W8A8	QWQ	✅	✅
LLM-Research/Llama-4-Scout-17B-16E-Instruct	Llama	✅	✅
AI-ModelScope/Llama-3.1-8B-Instruct	Llama	✅	✅
LLM-Research/llama-2-7b	Llama	✅	✅
LLM-Research/Llama-3.2-1B-Instruct	Llama	✅	✅
mistralai/Mistral-7B-Instruct-v0.2	Mistral	✅	✅
google/gemma-3-4b-it	Gemma	✅	✅
microsoft/Phi-4-multimodal-instruct	Phi	✅	✅
allenai/OLMoE-1B-7B-0924	OLMoE	✅	✅
stabilityai/stablelm-2-1_6b	StableLM	✅	✅
CohereForAI/c4ai-command-r-v01	Command-R	✅	✅
ZhipuAI/chatglm2-6b	ChatGLM	✅	✅
LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct	ExaONE 3	✅	✅
xverse/XVERSE-MoE-A36B	XVERSE	✅	✅
HuggingFaceTB/SmolLM-1.7B	SmolLM	✅	✅
Eco-Tech/GLM-5.2-w8a8	GLM-5	✅	✅
Eco-Tech/GLM-5.1-w4a8	GLM-5	✅	✅
Eco-Tech/GLM-5-w4a8	GLM-5	✅	✅
ZhipuAI/glm-4-9b-chat	GLM-4	✅	✅
iridiumine/MiMo-V2-Flash-W8A8	MiMo	✅	✅
XiaomiMiMo/MiMo-7B-RL	MiMo	✅	✅
arcee-ai/AFM-4.5B-Base	Arcee AFM-4.5B	✅	✅
Howeee/persimmon-8b-chat	Persimmon	✅	✅
inclusionAI/Ling-lite	Ling	✅	✅
ibm-granite/granite-3.1-8b-instruct	Granite	✅	✅
AI-ModelScope/dbrx-instruct	DBRX (Databricks)	✅	✅
baichuan-inc/Baichuan2-13B-Chat	Baichuan 2 (7B, 13B)	✅	✅
PaddlePaddle/ERNIE-4.5-21B-A3B-PT	ERNIE-4.5 (4.5, 4.5MoE series)	✅	✅
OpenBMB/MiniCPM3-4B	MiniCPM (v3, 4B)	✅	✅
sgl-npu/Kimi-K3-W4A8	Kimi	✅	✅
Eco-Tech/Kimi-K2.6-w4a8	Kimi	✅	✅
Eco-Tech/Kimi-K2.5-w4a8	Kimi	✅	✅
moonshotai/Kimi-K2-Thinking	Kimi	✅	✅
eigen-ai-labs/gpt-oss-120b-bf16	GPTOSS	✅	✅
allenai/OLMo-2-1124-7B-Instruct	OLMo	✅	✅
Eco-Tech/MiniMax-M2.5-w8a8-QuaRot	MiniMax-M2.5	✅	✅
cyankiwi/MiniMax-M2-BF16	MiniMax-M2	✅	✅
upstage/SOLAR-10.7B-Instruct-v1.0	Solar	✅	✅
FLM/Tele-FLM	Tele FLM (52B-1T)	✅	✅
bigcode/starcoder2-7b	StarCoder2	✅	✅
arcee-ai/Trinity-Mini	Trinity (Nano, Mini)	✅	✅
OrionStarAI/Orion-14B-Base	Orion (14B)	✅	✅
EleutherAI/gpt-j-6b	GPT-J (6B)	✅	✅
LiquidAI/LFM2.5-1.2B-Instruct	LFM2 (230M, 350M, 1.2B)	✅	✅
LiquidAI/LFM2-8B-A1B	LFM2-MoE (8B-A1B, 24B-A2B)	✅	✅
tiiuae/Falcon-H1-34B-Instruct	Falcon-H1 (0.5B-34B)	✅	✅
ibm-granite/granite-4.0-h-micro	IBM Granite 4.0 (Hybrid, Dense)	✅	✅
ibm-granite/granite-4.0-micro	IBM Granite 4.0 (Hybrid, Dense)	✅	✅
poolside/Laguna-XS.2	Laguna XS.2 (poolside)	✅	✅
Tencent-Hunyuan/Hy3	Hunyuan	✅	✅

Multimodal Language Models

Models	Model Family (Variants)	A2 Supported	A3 Supported
Qwen/Qwen2.5-VL-3B-Instruct	Qwen2.5-VL	✅	✅
Qwen/Qwen2.5-VL-72B-Instruct	Qwen2.5-VL	✅	✅
Qwen/Qwen3-VL-30B-A3B-Instruct	Qwen3-VL	✅	✅
Qwen/Qwen3-VL-8B-Instruct	Qwen3-VL	✅	✅
Qwen/Qwen3-VL-4B-Instruct	Qwen3-VL	✅	✅
Qwen/Qwen3-VL-235B-A22B-Instruct	Qwen3-VL	✅	✅
deepseek-ai/deepseek-vl2	DeepSeek-VL2	✅	✅
deepseek-ai/Janus-Pro-1B	Janus-Pro (1B, 7B)	✅	✅
deepseek-ai/Janus-Pro-7B	Janus-Pro (1B, 7B)	✅	✅
OpenBMB/MiniCPM-V-2_6	MiniCPM-V / MiniCPM-o	✅	✅
OpenBMB/MiniCPM-o-2_6	MiniCPM-V / MiniCPM-o	✅	✅
google/gemma-3-4b-it	Gemma 3 (Multimodal)	✅	✅
mistralai/Mistral-Small-3.1-24B-Instruct-2503	Mistral-Small-3.1-24B	✅	✅
microsoft/Phi-4-multimodal-instruct	Phi-4-multimodal-instruct	✅	✅
XiaomiMiMo/MiMo-VL-7B-RL	MiMo-VL (7B)	✅	✅
AI-ModelScope/llava-v1.6-34b	LLaVA (v1.5 & v1.6)	✅	✅
lmms-lab/llava-next-72b	LLaVA-NeXT (8B, 72B)	✅	✅
moonshotai/Kimi-VL-A3B-Instruct	Kimi-VL (A3B)	✅	✅
ZhipuAI/GLM-4.5V	GLM-4.5V (106B)	✅	✅
LLM-Research/Llama-3.2-11B-Vision-Instruct	Llama 3.2 Vision (11B)	✅	✅
rednote-hilab/dots.ocr	DotsVLM-OCR	✅	✅
Qwen/Qwen3-Omni-30B-A3B-Instruct	Qwen3-Omni	✅	✅
stepfun-ai/Step3-VL-10B	Step3-VL (10B)	✅	✅
deepseek-ai/DeepSeek-OCR-2	DeepSeek-OCR / OCR-2	✅	✅
lmms-lab/LLaVA-NeXT-Video-7B	LLaVA video variants (LLaVA-NeXT-Video, LLaVA-OneVision)	✅	✅
ZhipuAI/GLM-OCR	GLM-OCR	✅	✅
AI-ModelScope/whisper-large-v3	Whisper	✅	✅
Qwen/Qwen3-ASR-1.7B	Qwen3-ASR (0.6B, 1.7B)	✅	✅
LiquidAI/LFM2.5-VL-1.6B	LFM2-VL	✅	✅

Embedding Models

Models	Model Family	A2 Supported	A3 Supported
intfloat/e5-mistral-7b-instruct	E5 (Llama/Mistral based)	✅	✅
iic/gte-Qwen2-1.5B-instruct	GTE-Qwen2	✅	✅
Qwen/Qwen3-Embedding-8B	Qwen3-Embedding	✅	✅
iic/gme-Qwen2-VL-2B-Instruct	GME (Multimodal)	✅	✅
AI-ModelScope/clip-vit-large-patch14-336	CLIP	✅	✅
BAAI/bge-large-en-v1.5	BGE	✅	✅

Reward Models

Models	Model Family	A2 Supported	A3 Supported
Skywork/Skywork-Reward-Llama-3.1-8B-v0.2	Llama3.1 Reward	✅	✅
Shanghai_AI_Laboratory/internlm2-7b-reward	InternLM 2 Reward	✅	✅
Qwen/Qwen2.5-Math-RM-72B	Qwen2.5 Reward - Math	✅	✅
Howeee/Qwen2.5-1.5B-apeach	Qwen2.5 Reward - Sequence	✅	✅

Rerank Models

Models	Model Family	A2 Supported	A3 Supported
BAAI/bge-reranker-v2-m3	BGE-Reranker	✅	✅
Qwen/Qwen3-Reranker-8B	Qwen3-Reranker (decoder-only yes/no)	✅	✅
Qwen/Qwen3-VL-Reranker-2B	Qwen3-VL-Reranker (multimodal yes/no)	✅	✅

⌘I