Editing Qwen3.5


{{LLM Infobox
|name=Qwen 3.5
|developer=Alibaba Cloud
|release_date=February 15, 2026
|sizes=0.8B, 2B, 4B, 9B, 27B (dense), 35B-A3B (MoE), 122B-A10B (MoE), 397B-A17B (MoE)
|architecture=[[Decoder-only Transformer]]
|context_length=262,144 (up to 1M via API)
|modality=Image-Text-to-Text
|thinking=Yes (toggleable)
|license=Apache 2.0
|languages=201 languages and dialects
|hf_link=https://huggingface.co/Qwen
|paper_link=https://qwen.ai/blog?id=qwen3.5
}}

'''Qwen3.5''' is an open-weight and native vision-language foundation model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]]. The models support 201 languages and dialects, compared to 119 of their earlier [[Qwen3]] model series.
Smaller versions up to 27B parameters are available as dense models, whereas the sizes 35B-A3B, 122B-A10B as well as the flagship model 397B-A17B use a [[Mixture of Experts]] architecture.

== Benchmarks ==

Results for the flagship '''397B-A17B''' and '''9b''', '''4B''' as well as '''2B''' small models.

{| class="wikitable"
|-
! Benchmark !! Category !! 397B-A17B !! 9B !! 4B !! 2B !! Claude Opus 4.6
|-
| [[GPQA Diamond]]<ref>[https://artificialanalysis.ai/evaluations/gpqa-diamond?models=gemma-4-26b-a4b%2Cgemma-4-31b-non-reasoning%2Cgemma-4-e2b%2Cgemma-4-e4b-non-reasoning%2Cgemma-4-e4b%2Cgemma-4-e2b-non-reasoning%2Cclaude-opus-4-6-adaptive%2Cqwen3-5-2b%2Cqwen3-5-9b%2Cqwen3-5-397b-a17b%2Cqwen3-5-4b%2Cqwen3-5-2b-non-reasoning GPQA Diamond Benchmark Leaderboard: Results]. Artificial Analysis, April 2026.</ref> || style="text-align:right;" | Science || style="text-align:right;" | 89.3 || 80.6 || 77.1 || -- || style="text-align:right;" | 89.6
|-
| [[SWE-bench Verified]] || style="text-align:right;" | Coding || style="text-align:right;" | 76.4 || -- || -- || -- || style="text-align:right;" | 80.8
|-
| [[MMMU-Pro]] || style="text-align:right;" | Multimodal || style="text-align:right;" | 79.0 || 70.1 || 66.3 || 50.3 || style="text-align:right;" | 73.9
|-
| [[MMMLU]] || style="text-align:right;" | Multilingual || style="text-align:right;" | 88.5 || 81.2 || 76.1 || 63.1 ||style="text-align:right;" | 91.1
|}

== Tokenizer ==

The Qwen3.5 tokenizer has a total of 248,077 tokens (up from 151,669 in [[Qwen3]]), out of which 248,044 stem from the [[BPE]] vocabulary size, and 33 are added tokens. Note that Qwen3.5 uses 248,320 embeddings in its embedding table, slightly more than the total token number of the tokenizer. This might be due to performance improvements (it is a multiple of 512) and leaves room for adding additional tokens in the future.

Thinking can be '''enabled''' and '''disabled''' through the chat template, which either appends <syntaxhighlight inline=1><think>\n</syntaxhighlight> or <syntaxhighlight inline=1><think>\n\n</think>\n\n</syntaxhighlight>.

<syntaxhighlight lang="python">
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")

len(tokenizer)
# -> 248077

messages = [{"role": "user", "content": "Hi"}]
tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)
# -> '<|im_start|>user\nHi<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n'

tokenizer.apply_chat_template(messages, enable_thinking=True, tokenize=False, add_generation_prompt=True)
# -> '<|im_start|>user\nHi<|im_end|>\n<|im_start|>assistant\n<think>\n'

tokenizer.apply_chat_template(messages, enable_thinking=True, tokenize=True, add_generation_prompt=True)
# -> {'input_ids': [248045, 846, 198, 12675, 248046, 198, 248045, 74455, 198, 248068, 198], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
</syntaxhighlight>


== References ==
<references />