Fastino 發布 GLiNER2.5-Decide：可在 CPU 上運行的 340M 開放權重決策模型

Fastino Labs has released GLiNER2.5-Decide, a 340M-parameter open-weight decision model.It takes text and a schema of typed questions and returns structured answers.Each answer comes with a probability distribution, a confidence score, and constraint-feasibility metadata.

It targets the frequent judgment calls inside agent pipelines: routing, triage, tool selection, and guardrails.Is it deployable?Yes, the weights ship under Apache 2.0 and install with pip install gliner2.They run on CPU, GPU, or in air-gapped environments.

Fastino team also offers hosted inference and fine-tuning through its GLiNER API.What GLiNER2.5-Decide Actually Does GLiNER2.5-Decide is a non-generative classifier.It uses a DeBERTa-v3-large encoder and is fine-tuned from gliner2-large-v1.

It produces no generated tokens and needs no prompt template.Label sets are passed at call time.Each question in the schema declares its permitted answers.It also declares whether it expects one answer, multiple answers, or an ordered value.

Schemas can carry instructions, examples, label descriptions, and rules linking answers across questions.The pipeline has 2 stages.The encoder reads the text and schema together and scores every permitted answer.

A constrained decoder then searches for the highest-scoring joint assignment the declared rules allow.Fastino is explicit about scope.The model does not reason, explain, or answer open questions.It is a specialist for operational decisions.

Why Joint Decoding Matters Fastino team illustrates the value with a guardrail example.Decoded independently, the model flagged prompt injection at 0.82.It also labeled the same prompt safe at 0.52.The attack was detected, but the 2 outputs conflicted.

Joint decoding applies a rule that any detected harm requires an unsafe verdict.The model then returns safety=unsafe and harmtype=promptinjection together.Downstream code can use those scores to block, route, or escalate.

Schemas can express implications, exclusions, cardinality limits, and ordinal bounds.The same encoder can also extract entities, relations, and structured records with character-level offsets in 1 forward pass.Classification answers do not return evidence spans.window.

addEventListener("message",function(e){if(e.data&&e.data.mtpFrame==="gliner-decide"){var f=document.getElementById("mtp-gliner-decide-frame");if(f){f.style.height=e.data.

h+"px";}}}); Benchmark Results on Fast Decisions Fastino team evaluated the model on Fast Decisions, an internally generated, held-out suite.It contains 5,100 test examples across 17 datasets.

The tasks cover customer operations, domain routing (banking, clinical, travel, benefits), and general content understanding.The metric is exact-match accuracy: a prediction counts only if its label set matches the reference exactly.ModelTypeAverageGLiNER2.5-Decide340M encoder60.

1%JevK54B-class Qwen3.5 decoder57.5%SemIfQwen3.5-4B decoder56.4%GLiFormer large-v1Single-pass encoder49.0%Laya421M ModernBERT encoder46.6% GLiNER2.5-Decide led 9 of the 17 datasets.Intent routing was its strongest area.It scored 75.3% on support intent and 64.3% on banking intent.Those are 18.

6 and 8.6 points ahead of the next-best models.Latency: Practical on CPU Fastino team benchmarked the checkpoint end to end at batch 1 with a 2-head, 15-label schema.At 64 tokens, p50 latency was: 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C 43.6 ms on an NVIDIA T4 43.4 ms on an NVIDIA L4 38.

3 ms on an NVIDIA V100 47.3 ms on an NVIDIA A100 Short requests are dominated by fixed preprocessing and kernel-launch overhead.That keeps the GPUs within 9 ms of each other.At 1,024 tokens, the A100 pulls ahead at 52.6 ms, versus 75.6 ms on V100 and 131.4 ms on L4.

Using It in Code This multi-head call is adapted from the model card: Copy CodeCopiedUse a different Browserfrom gliner2 import AutoExtractor model = AutoExtractor.frompretrained("fastino/GLiNER2.5-Decide") model.classifytext( "Please confirm the new retention rule is applied before Friday's audit.

", { "intent": ["fyi", "request", "approval", "complaint"], "urgency": ["low", "normal", "high", "critical"], "route": ["support", "billing", "legal", "security"], }, ) Single-label heads return one string.Multi-label heads return every label above a cls_threshold.

Labels can carry descriptions, and ordinal scales are passed as ordinary strings like “0” to “10”.Use Cases and the Model Family Fastino targets model routing, tool calling, browser and computer use, guardrails, context pruning, LLM-as-a-judge, and simulations.

Fine-tuning works locally, fully or with LoRA, through the GLiNER2 trainer.A SKILL.md file exposes the hosted workflow to coding agents.Fastino also published GLiNER2.5-Decide-1B, built on the Ettin 1B encoder.It scores 59.6% on the same suite, just under the 340M model.

For multilingual input, Fastino points to GLiNER2.5-multi-Decide, a 287M model scoring 56.7%.The underlying architecture is described in the GLiNER2 paper.Key Takeaways 340M open-weight encoder, Apache 2.0, runs on CPU or air-gapped.

Returns answers, probabilities, confidence, and feasibility metadata.Joint decoding enforces rules across related answers.60.1% average on Fastino’s internal 17-dataset suite, leading 9.167.3 ms p50 on CPU, 38.3 ms on V100 for short inputs.Check out the Technical Details and Model Card.

All credit goes to the researcher of this project.Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter.Wait!are you on telegram?now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?Connect with us The post Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU appeared first on MarkTechPost.