ModelsVision-languageOct 20, 2026

Mtrini-SVL-1.2

The third SVL release on Qwen3-VL-8B-Instruct, with “Identity v3” trained into the weights. Published as merged weights and a LoRA adapter.

View on Hugging Face Try in Playground

Mtrini-SVL-1.2 is a fine-tune of Qwen3-VL-8B-Instruct. The release incorporates Identity v3 into the model weights; the merged repository is a full Transformers model, and the original LoRA adapter is published separately.

The model cards describe identity behavior as experimental: in evaluation the model could still misidentify its developer, and the adapter card notes it may invent unverified company facts.

Benchmarks

Mtrini-SVL-1.2 benchmark scores
Benchmark	Metric	Split	Score
AI2DAccuracy · Test · 2,588 of 3,088 correct	Accuracy	Test	83.81%
DocVQAANLS · Validation · 5,349 questions. Test-split answers are not public.	ANLS	Validation	95.60
OCRBenchFinal score · Test · 834 of 1,000 correct	Final score	Test	834 / 1000
MMMUAccuracy · Validation · Official Compiwer AI result	Accuracy	Validation	73.9%
CodeMMLUAccuracy · Full, 19,878 questions · 11,996 of 19,878 correct · official Compiwer AI result	Accuracy	Full, 19,878 questions	60.35%
MMECombined score · Perception + cognition · Perception 1,759.11 / 2,000 · Cognition 625.71 / 800 · official Compiwer AI result	Combined score	Perception + cognition	2,384.82 / 2,800
MATH-500Accuracy · 500 problems · 266 of 500 correct · official Compiwer AI result	Accuracy	500 problems	53.2%
ATU-1.1ATU-Score · 20 tasks · Rank #5 of 7 in the official standings	ATU-Score	20 tasks	40.25%

AI2D, DocVQA, OCRBench and MMMU were run with lmms-eval on the merged weights, then re-checked against each benchmark’s official answer key and scoring rule. Verified Oct 22, 2026. HallusionBench and MathVista runs are not complete yet, so no scores are shown for them. Scores use different metrics and are not averaged.

How it compares

Mtrini-SVL-1.2 compared with other models on AI2D, DocVQA, OCRBench, MMMU
Model	AI2D	DocVQA*	OCRBench	MMMU
Mtrini-SVL-1.2This model	83.8	95.6val	834	73.9
Qwen3-VL-8B-InstructBase model	85.7	96.1	896	69.6
Qwen3-VL-4B-Instruct	84.1	95.3	881	67.4
Qwen2.5-VL-7B-Instruct	83.9	95.7	864	58.6
Qwen2.5-VL-72B-Instruct	88.7	96.4	885	70.2
Gemini 2.5 Flash-LiteWithout thinking	84.8	92.0	813	72.7
GPT-5 NanoMinimal reasoning	65.7	78.3	701	57.6

Other models’ scores are quoted from their developers’ published results, using their own evaluation setups. Prompts and harnesses differ, which can move a score by a point or more.

* DocVQA: the published scores use the test split. Mtrini-SVL-1.2’s score uses the validation split, because test answers are not public, so that column is not a like-for-like comparison.

AI2D, DocVQA and MMMU are on a 0–100 scale. OCRBench counts correct answers out of 1,000. MMMU uses the validation split for every model.

The best published score in each column is in bold.

Sources: Qwen3-VL-8B-Instruct model card (opens in new tab) · Qwen2.5-VL technical report, Table 5, and model card (opens in new tab) · Qwen3-VL-8B-Instruct model card (evaluated by Qwen) (opens in new tab)

CodeMMLU: how it compares

Would rank #4 of 68 on the source leaderboard.

Mtrini-SVL-1.2 compared with other models on CodeMMLU
Model	CodeMMLU accuracy
GPT-4o	67.00%
GPT o3-mini	62.36%
Claude 3.7 Sonnet	61.65%
Mtrini-SVL-1.2This model	60.35%
Llama 3.1 70B Instruct	60.00%
Claude 3.5 Sonnet	59.81%
Qwen2.5-Coder 32B Instruct	56.40%
Qwen2.5 14B Instruct	51.38%
DeepSeek V3	49.08%
Llama 3.1 8B Instruct	45.22%
GPT-4o mini	38.43%

Other models’ scores were run by the CodeMMLU authors with their own prompts. The rank counts Mtrini-SVL-1.2 among the 67 models on the official leaderboard, which does not yet include newer models such as Qwen3. Source: Official CodeMMLU leaderboard (data updated April 2025).

CodeMMLU by category

CodeMMLU accuracy by category
Category	Questions	Accuracy
API frameworks	701	85.16%
Code completion	164	76.22%
DBMS / SQL	389	72.49%
Fill in the middle	2,129	72.10%
Other coding topics	1,371	68.64%
Software principles	2,826	66.95%
Programming syntax	6,216	62.18%
Execution prediction	6,006	45.54%
Code repair	76	31.58%

Overall 60.35% (11,996 of 19,878 correct). Model revision 348e871c. 99.99% of answers were in a valid format. Runtime 15 min 47 s.

MME: how it compares

Mtrini-SVL-1.2 compared with other models on MME
Model	MME score (max 2,800)
Mtrini-SVL-1.2This model	2,384.82
MiniCPM-V 2.6	2,348.4
InternVL2.5-8B	2,344.1
Qwen2-VL-7B	2,326.8
InternVL2-8B	2,210.3
GPT-4o mini	2,003.4

MME adds perception (out of 2,000) and cognition (out of 800), so the maximum is 2,800. It is a points total, not an accuracy. Other models’ scores are as published by their developers. These are 2024 models: newer ones such as Qwen2.5-VL and Qwen3-VL do not publish MME. Sources: Qwen2-VL-7B-Instruct model card · InternVL2.5 blog.

MATH-500: how it compares

Would rank #169 of 203 on the source leaderboard.

Mtrini-SVL-1.2 compared with other models on MATH-500
Model	MATH-500 accuracy
GPT-5 (high)	99.4%
DeepSeek R1 0528	98.3%
GPT-4o mini	78.9%
Llama 3.3 70B	77.3%
Gemma 3 4B	76.6%
Mistral Small 3.1	70.7%
Phi-4 Mini	69.6%
Llama 3.1 70B	64.9%
GPT-4	56.8%
Mixtral 8x22B	54.5%
Mtrini-SVL-1.2This model	53.2%
Llama 3.1 8B	51.9%
Llama 3.2 11B Vision	51.6%
GPT-3.5 Turbo	44.1%
Claude 3 Haiku	39.4%

Other models’ scores were measured independently by Artificial Analysis with one shared setup. The rank places Mtrini-SVL-1.2 among the 202 models on that leaderboard, which Artificial Analysis now lists as a legacy benchmark. Many top entries are reasoning models that think at length before answering. Source: Artificial Analysis MATH-500 leaderboard.

Specifications

Status: Experimental
Base model: Qwen3-VL-8B-Instruct (opens in new tab)
Release: Mtrini-SVL-1.2 with Identity v3
Formats: Merged Transformers weights, LoRA adapter
Focus: Vision-language, Moroccan Darija, Model identity

From the model card. Details the card doesn’t state are left out.

Downloads

Mtrini-SVL-1.2 (opens in new tab)
Merged · Merged Transformers weights · License not declared
114 downloads last 30 days
Mtrini-SVL-1.2-LoRA-Adapter (opens in new tab)
Adapter · LoRA adapter (safetensors) · License not declared
12 downloads last 30 days

Download counts from the Hugging Face API, updated hourly.

Limitations

Identity behavior is experimental: the model can still misidentify its developer.
Identity responses may include invented, unverified company facts. Do not rely on them.
No license is declared in the repository metadata. Review the base model’s license and redistribution terms before use.