VibeThinker-3B：基於Qwen2.5-Coder-3B與頻譜至訊號後訓練管線的30億參數濃密推理模型

While recent breakthroughs in AI reasoning have largely been driven by massive scale, pouring in billions of parameters to cross complex cognitive thresholds—VibeThinker-3B is charting a completely different path. Created by researchers from Sina Weibo Inc (China), this 3-billion-parameter model proves that efficiency can punch far above its weight class. Released under an open-source MIT license, VibeThinker-3B matches the performance of models hundreds of times its size on verifiable tasks like mathematics, coding, and STEM disciplines. What is VibeThinker-3B VibeThinker-3B is a compact dense model built on the Qwen2.5-Coder-3B base. It is post-trained, not pretrained from scratch. The research team applies supervised fine-tuning, reinforcement learning, and self-distillation on top. The training framework continues the Spectrum-to-Signal Principle (SSP) from the earlier VibeThinker-1.5B. SFT (Supervised Fine-Tuning) builds a broad space of valid reasoning paths, the ‘Spectrum.’ RL then amplifies the correct paths, the ‘Signal.’ The model targets one job: reasoning where a verifier can confirm the answer. The research team recommends larger general models for open-domain knowledge tasks. VibeThinker-3B is a specialist by design. It runs on standard stacks. The model weights require transformers>=4.54.0. For faster inference it recommends vLLM==0.10.1 or SGLang>=0.4.9.post6. The BF16 weights are roughly 6 GB, small enough for a single GPU. https://arxiv.org/pdf/2606.16140v1 Benchmark On AIME26, VibeThinker-3B scores 94.3. According to the research paper, this is comparable to DeepSeek V3.2 (671B) and Kimi K2.5 (1T). On LiveCodeBench v6, it reaches 80.2 Pass@1. On OJBench, another code benchmark, it scores 38.6, below the largest models. On HMMT25 it scores 89.3, and on BruMO25 it reaches 93.8. On IMO-AnswerBench, a 400-problem IMO-level set, it scores 76.4. The table below compares it against much larger reasoning models. The ‘+CLR’ row uses test-time scaling. It stands for Claim-Level Reliability Assessment ModelParamsAIME26HMMT25IMO-AnsLCBv6GPQA-DVibeThinker-3B3B94.389.376.480.270.2VibeThinker-3B +CLR3B97.195.480.6—72.9GPT-OSS (high)120B93.290.075.681.980.1DeepSeek V3.2671B94.290.278.380.882.4GLM-5744B95.897.982.585.586.0Kimi K2.51T93.395.481.885.087.6Source: VibeThinker-3B Technical Report, Table 2. GPQA-D is GPQA-Diamond. The pattern is consistent. On verifiable math and code, the 3B model sits near the top cluster. On GPQA-Diamond, a knowledge-heavy benchmark, the gap to large models stays visible. The research team also ran an out-of-distribution coding test. It used recent LeetCode weekly and biweekly contests, from Apr 25 to May 31, 2026. The model passed 123 of 128 first-attempt Python submissions. That is a 96.1% acceptance rate on unseen problems. Inside the Spectrum-to-Signal Pipeline The post-training pipeline runs in four stages. Each one targets a different weakness of small reasoning models. First comes curriculum-based two-stage SFT. Stage 1 covers math, code, STEM, dialogue, and instruction following broadly. Stage 2 shifts to harder, longer-horizon samples filtered by reasoning length and difficulty. Diversity-Exploring Distillation preserves multiple valid solution paths through both stages. Second comes multi-domain Reasoning RL. The research team reuses MaxEnt-Guided Policy Optimization (MGPO). MGPO weights prompts near the model’s current capability boundary, where correct and incorrect rollouts coexist. Training runs sequentially across Math, Code, and STEM. A notable detail: VibeThinker-3B drops progressive context expansion. The research team found high-truncation warm-up hurt long reasoning at this scale. So RL uses a single 64K long-context window throughout. Math RL adds a Long2Short stage. It redistributes reward among correct trajectories by length. Shorter correct answers get higher reward, longer ones lower, with the group mean unchanged. The goal is fewer redundant tokens without losing accuracy. Third, Offline Self-Distillation merges the RL checkpoints back into one student model. Fourth, Instruct RL improves instruction adherence. That stage explains the 93.4 IFEval and 74.5 IFBench scores. Both show reasoning tuning did not break controllability. CLR: Scaling at Test Time, Not Parameter Count Claim-Level Reliability Assessment (CLR) is the report’s test-time scaling method. It runs on answer-verifiable tasks and adds no parameters. The procedure has two steps. The model first generates K = 32 trajectories per problem. From each, it extracts M = 5 decision-relevant claims plus a final answer. The model then acts as its own verifier. It validates or falsifies each claim, producing binary verdicts. CLR maps these into a nonlinear trajectory reliability score, where one weak claim sharply lowers the weight. Answers are clustered by equivalence, and the highest reliability-weighted answer wins. The full flow runs 8 times, and the averaged Pass@1 is reported. CLR lifts AIME26 to 97.1 and BruMO25 to 99.2. The interactive demo below lets you flip claims and watch the score collapse. It also lets you switch benchmarks and compare against larger models. (function(){ window.addEventListener('message', function(e){ if(e && e.data && e.data.vt3bHeight){ var f=document.getElementById('vt3b-frame'); if(f){ f.style.height = e.data.vt3bHeight + 'px'; } } }); })(); Use Cases With Examples The research team frames VibeThinker-3B as a specialist, so use cases follow the verifiable-reasoning boundary. Competitive math tutoring: It solves AIME and HMMT-style problems with full chains of reasoning. A study tool could generate worked solutions and self-check answers locally. Algorithmic coding help: The 96.1% LeetCode acceptance rate suggests strong one-shot Python generation. An IDE assistant could draft contest-style solutions and run hidden tests. Cost-sensitive RL or agent backends: A 3B model is cheap to serve at scale. Teams running many verifiable subtasks could route them here instead of a 600B+ model. On-device reasoning. BF16 weights fit one consumer GPU. Edge or offline deployments gain a reasoning engine without cloud calls. Running It: Quick Start Serving with vLLM exposes an OpenAI-compatible endpoint: Copy CodeCopiedUse a different Browserpip install vllm vllm serve "WeiboAI/VibeThinker-3B" curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "WeiboAI/VibeThinker-3B", "messages": [{"role":"user","content":"Prove there are infinitely many primes."}], "temperature": 1.0, "top_p": 0.95 }' Direct Transformers usage mirrors the official card: Copy CodeCopiedUse a different Browserfrom transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("WeiboAI/VibeThinker-3B", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( "WeiboAI/VibeThinker-3B", torch_dtype="bfloat16", device_map="auto") msgs = [{"role": "user", "content": "Your prompt"}] text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) inputs = tok([text], return_tensors="pt").to(model.device) out = model.generate(**inputs, max_new_tokens=102400, do_sample=True, temperature=1.0, top_p=0.95) print(tok.decode(out[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True)) The high max_new_tokens matters. The model produces long reasoning traces, so short caps can truncate answers. Key Takeaways VibeThinker-3B is a 3B dense model, MIT-licensed, built on Qwen2.5-Coder-3B for verifiable reasoning. It scores 94.3 on AIME26, comparable to DeepSeek V3.2 (671B) and Kimi K2.5 (1T). CLR test-time scaling lifts AIME26 to 97.1 and BruMO25 to 99.2, with no extra parameters. On unseen LeetCode contests, it passed 123 of 128 first-attempt Python submissions (96.1%). The gain is narrow: it trails large models on GPQA-Diamond and broad open-domain knowledge. Check out the Paper, Model weight and Repo. Also, feel free to follow us on Twitter and don