Qwen3.5 - Revision history

FKemeth at 13:19, 7 April 2026

2026-04-07T13:19:42Z

← Older revision		Revision as of 13:19, 7 April 2026
Line 16:		Line 16:

	'''Qwen3.5''' is an open-weight and native vision-language foundation model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]]. The models support 201 languages and dialects, compared to 119 of their earlier [[Qwen3]] model series.		'''Qwen3.5''' is an open-weight and native vision-language foundation model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]]. The models support 201 languages and dialects, compared to 119 of their earlier [[Qwen3]] model series.
			Smaller versions up to 27B parameters are available as dense models, whereas the sizes 35B-A3B, 122B-A10B as well as the flagship model 397B-A17B use a [[Mixture of Experts]] architecture.

	== Benchmarks ==		== Benchmarks ==

FKemeth at 13:15, 7 April 2026

2026-04-07T13:15:09Z

← Older revision		Revision as of 13:15, 7 April 2026
Line 37:		Line 37:

	The Qwen3.5 tokenizer has a total of 248,077 tokens (up from 151,669 in [[Qwen3]]), out of which 248,044 stem from the [[BPE]] vocabulary size, and 33 are added tokens. Note that Qwen3.5 uses 248,320 embeddings in its embedding table, slightly more than the total token number of the tokenizer. This might be due to performance improvements (it is a multiple of 512) and leaves room for adding additional tokens in the future.		The Qwen3.5 tokenizer has a total of 248,077 tokens (up from 151,669 in [[Qwen3]]), out of which 248,044 stem from the [[BPE]] vocabulary size, and 33 are added tokens. Note that Qwen3.5 uses 248,320 embeddings in its embedding table, slightly more than the total token number of the tokenizer. This might be due to performance improvements (it is a multiple of 512) and leaves room for adding additional tokens in the future.

			Thinking can be '''enabled''' and '''disabled''' through the chat template, which either appends <syntaxhighlight inline=1><think>\n</syntaxhighlight> or <syntaxhighlight inline=1><think>\n\n</think>\n\n</syntaxhighlight>.

	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">

FKemeth at 13:09, 7 April 2026

2026-04-07T13:09:27Z

← Older revision		Revision as of 13:09, 7 April 2026
Line 35:		Line 35:

	== Tokenizer ==		== Tokenizer ==

			The Qwen3.5 tokenizer has a total of 248,077 tokens (up from 151,669 in [[Qwen3]]), out of which 248,044 stem from the [[BPE]] vocabulary size, and 33 are added tokens. Note that Qwen3.5 uses 248,320 embeddings in its embedding table, slightly more than the total token number of the tokenizer. This might be due to performance improvements (it is a multiple of 512) and leaves room for adding additional tokens in the future.

	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">
Line 40:		Line 42:

	tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")		tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")

			len(tokenizer)
			# -> 248077

	messages = [{"role": "user", "content": "Hi"}]		messages = [{"role": "user", "content": "Hi"}]

FKemeth at 12:57, 7 April 2026

2026-04-07T12:57:10Z

← Older revision		Revision as of 12:57, 7 April 2026
Line 41:		Line 41:
	tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")		tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")

			messages = [{"role": "user", "content": "Hi"}]
	tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)		tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)
			# -> '<\|im_start\|>user\nHi<\|im_end\|>\n<\|im_start\|>assistant\n<think>\n\n</think>\n\n'

			tokenizer.apply_chat_template(messages, enable_thinking=True, tokenize=False, add_generation_prompt=True)
			# -> '<\|im_start\|>user\nHi<\|im_end\|>\n<\|im_start\|>assistant\n<think>\n'

			tokenizer.apply_chat_template(messages, enable_thinking=True, tokenize=True, add_generation_prompt=True)
			# -> {'input_ids': [248045, 846, 198, 12675, 248046, 198, 248045, 74455, 198, 248068, 198], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}
	</syntaxhighlight>		</syntaxhighlight>
	~~'''Output:'''~~
	~~<pre style="background:#f0f0f0; border-left:3px solid #ccc; padding:8px;">~~
	~~'<\|im_start\|>user\nHi<\|im_end\|>\n<\|im_start\|>assistant\n<think>\n\n</think>\n\n'~~
	~~</pre>~~

	== References ==		== References ==
	<references />		<references />

FKemeth at 12:52, 7 April 2026

2026-04-07T12:52:30Z

← Older revision		Revision as of 12:52, 7 April 2026
Line 40:		Line 40:

	tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")		tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")
	~~</syntaxhighlight>~~

	tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)		tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)
			</syntaxhighlight>
			'''Output:'''
			<pre style="background:#f0f0f0; border-left:3px solid #ccc; padding:8px;">
			'<\|im_start\|>user\nHi<\|im_end\|>\n<\|im_start\|>assistant\n<think>\n\n</think>\n\n'
			</pre>

	== References ==		== References ==
	<references />		<references />

FKemeth at 12:51, 7 April 2026

2026-04-07T12:51:45Z

← Older revision		Revision as of 12:51, 7 April 2026
Line 34:		Line 34:
	\|}		\|}

			== Tokenizer ==

			<syntaxhighlight lang="python">
			from transformers import AutoTokenizer

			tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-0.8B")
			</syntaxhighlight>

			tokenizer.apply_chat_template(messages, enable_thinking=False, tokenize=False, add_generation_prompt=True)

	== References ==		== References ==
	<references />		<references />

FKemeth at 09:03, 7 April 2026

2026-04-07T09:03:16Z

← Older revision		Revision as of 09:03, 7 April 2026
Line 25:		Line 25:
	! Benchmark !! Category !! 397B-A17B !! 9B !! 4B !! 2B !! Claude Opus 4.6		! Benchmark !! Category !! 397B-A17B !! 9B !! 4B !! 2B !! Claude Opus 4.6
	\|-		\|-
	\| [[GPQA Diamond]]<ref>[https://artificialanalysis.ai/evaluations/gpqa-diamond?models=gemma-4-26b-a4b%2Cgemma-4-31b-non-reasoning%2Cgemma-4-e2b%2Cgemma-4-e4b-non-reasoning%2Cgemma-4-e4b%2Cgemma-4-e2b-non-reasoning%2Cclaude-opus-4-6-adaptive%2Cqwen3-5-2b%2Cqwen3-5-9b%2Cqwen3-5-397b-a17b%2Cqwen3-5-4b%2Cqwen3-5-2b-non-reasoning GPQA Diamond Benchmark Leaderboard: Results]. Artificial Analysis, April 2026.</ref> \|\| Science \|\| 89.3 \|\| 80.6 \|\| 77.1 \|\| -- \|\| 89.6		\| [[GPQA Diamond]]<ref>[https://artificialanalysis.ai/evaluations/gpqa-diamond?models=gemma-4-26b-a4b%2Cgemma-4-31b-non-reasoning%2Cgemma-4-e2b%2Cgemma-4-e4b-non-reasoning%2Cgemma-4-e4b%2Cgemma-4-e2b-non-reasoning%2Cclaude-opus-4-6-adaptive%2Cqwen3-5-2b%2Cqwen3-5-9b%2Cqwen3-5-397b-a17b%2Cqwen3-5-4b%2Cqwen3-5-2b-non-reasoning GPQA Diamond Benchmark Leaderboard: Results]. Artificial Analysis, April 2026.</ref> \|\| style="text-align:right;" \| Science \|\| style="text-align:right;" \| 89.3 \|\| 80.6 \|\| 77.1 \|\| -- \|\| style="text-align:right;" \| 89.6
	\|-		\|-
	\| [[SWE-bench Verified]] \|\| Coding \|\| 76.4 \|\| -- \|\| -- \|\| -- \|\| 80.8		\| [[SWE-bench Verified]] \|\| style="text-align:right;" \| Coding \|\| style="text-align:right;" \| 76.4 \|\| -- \|\| -- \|\| -- \|\| style="text-align:right;" \| 80.8
	\|-		\|-
	\| [[MMMU-Pro]] \|\| Multimodal \|\| 79.0 \|\| 70.1 \|\| 66.3 \|\| 50.3 \|\| 73.9		\| [[MMMU-Pro]] \|\| style="text-align:right;" \| Multimodal \|\| style="text-align:right;" \| 79.0 \|\| 70.1 \|\| 66.3 \|\| 50.3 \|\| style="text-align:right;" \| 73.9
	\|-		\|-
	\| [[MMMLU]] \|\| Multilingual \|\| 88.5 \|\| 81.2 \|\| 76.1 \|\| 63.1 \|\| 91.1		\| [[MMMLU]] \|\| style="text-align:right;" \| Multilingual \|\| style="text-align:right;" \| 88.5 \|\| 81.2 \|\| 76.1 \|\| 63.1 \|\|style="text-align:right;" \| 91.1
	\|}		\|}

FKemeth at 08:57, 7 April 2026

2026-04-07T08:57:07Z

@@ Line 15: / Line 15: @@
 }}
-'''Qwen3.5''' is an open-weight and native vision-language model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]].
+'''Qwen3.5''' is an open-weight and native vision-language foundation model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]]. The models support 201 languages and dialects, compared to 119 of their earlier [[Qwen3]] model series.
 == References ==
 <references />

FKemeth at 08:14, 7 April 2026

2026-04-07T08:14:48Z

← Older revision		Revision as of 08:14, 7 April 2026
Line 15:		Line 15:
	}}		}}

	'''Qwen3.5''' is an open-weight and native vision-language model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref>		'''Qwen3.5''' is an open-weight and native vision-language model series developed by Alibaba and released on February 15, 2026.<ref name="qwen3.5-blog">[https://qwen.ai/blog?id=qwen3.5 Qwen3.5: Towards Native Multimodal Agents]. Qwen Team, February 2026.</ref> It is build on a hybrid architecture using linear attention with [[Gated Delta Networks]] as well as sparse [[Mixture of Experts]].


	== References ==		== References ==
	<references />		<references />

FKemeth at 08:12, 7 April 2026

2026-04-07T08:12:38Z

← Older revision		Revision as of 08:12, 7 April 2026
Line 5:		Line 5:
	\|release_date=February 15, 2026		\|release_date=February 15, 2026
	\|sizes=0.8B, 2B, 4B, 9B, 27B (dense), 35B-A3B (MoE), 122B-A10B (MoE), 397B-A17B (MoE)		\|sizes=0.8B, 2B, 4B, 9B, 27B (dense), 35B-A3B (MoE), 122B-A10B (MoE), 397B-A17B (MoE)
	\|architecture=[[Decoder-only Transformer~~]] with [[Gated Delta Networks]] + sparse [[Mixture of Experts\|MoE~~]]		\|architecture=[[Decoder-only Transformer]]
	\|context_length=262,144 (up to 1M via API)		\|context_length=262,144 (up to 1M via API)
	\|modality=Image-Text-to-Text		\|modality=Image-Text-to-Text