Mistral發佈開源安全小模型

Shieldstral, a 3-billion-parameter model from French AI company Mistral, matches models three times its size on standard text safety benchmarks, according to the paper. Mistral says the model also sets a new high score for joint text and image classification. Runtime rules let operators tailor safety checks Many guardrail models sort content using fixed taxonomies. The paper's authors, including Mistral co-founder Guillaume Lample, point to two problems with this approach: public safety datasets group risks too differently to support one common taxonomy, and the same rules don't fit every use case. Content suitable for a cybersecurity tool could be harmful on a mental health platform.Ad Operators write the review criteria in plain language. Shieldstral returns one token, which produces a safety score between zero and one. | Image: Mistral Operators tell Shieldstral what to check with plain-language questions such as "Does this content promote violence?" The model answers only "yes" or "no," and the system uses the probability of each response to calculate a safety score between zero and one.AdDEC_D_Incontent-1 Synthetic data helps the model handle new rules The researchers combined about 54.1 million examples covering safety, harmful content, and manipulation attempts into one format. They applied strict standards to targeted manipulation, moderate standards to general safety data, and lenient standards to response quality. Each training example includes task instructions, a specific yes or no question, and the content being reviewed. The answer is a single token. | Image: Mistral To teach Shieldstral finer distinctions, the team used another language model to rewrite safe text into unsafe variants. Each example also included a similar but different category that had to be rejected, which trained the model to separate closely related rules rather than make only a broad safe or unsafe judgment.Ad The authors created the adaptability test categories separately from the training set, using different names and levels of detail. None of the fine-grained test categories directly matches a training category, though 10 of the 12 broader classes have rough counterparts, they say. The 3B model ties one nearly seven times larger Across the combined text benchmarks, Shieldstral posts an F1 score of 84.9 percent. F1 combines precision and recall into one metric, with 100 percent representing a perfect score. That result ties OpenAI's GPT-OSS-Safeguard-20B, which is about seven times larger, and beats Qwen3Guard-8B at 84.0 percent, Nemotron-3.5-Safety-4B at 83.3 percent, and LlamaGuard-4-12B at 69.1 percent.AdDEC_D_Incontent-2 On images and image-text combinations, Shieldstral scores 83.8 percent, ahead of OmniGuard-7B at 77.6 percent and LlavaGuard-7B at 71.6 percent.Ad Across the text benchmarks, Shieldstral ties an OpenAI model roughly seven times its size. | Image: Mistral GPT-OSS-Safeguard-20B leads the adaptability benchmark with 94.1 percent, compared with Shieldstral's 91.3 percent. This test uses rules that differ from the training categories or are entirely new. The authors still consider Shieldstral more practical than GPT-OSS-Safeguard-20B and Nemotron-3.5-Safety, since both generate long intermediate reasoning sequences that raise compute costs, while Shieldstral returns a single word. On policies absent from training, Shieldstral trails the two models that generate intermediate reasoning before answering. | Image: Mistral Shieldstral is based on Mistral's Ministral-3B with the Pixtral vision encoder. In a validation test with fine-grained categories, synthetic category data raised the F1 score by 23.3 percentage points, which the researchers say was the main driver of the model's ability to adapt to new rules. Shieldstral is available as an open-weight model under the Apache 2.0 license. Custom rules could limit costly filtering mistakes Safety classifiers sit on either side of the main language model, screening prompts before processing and responses before they reach users. Operators can update these rules without retraining the main model. But because every request passes through the classifier, its size, speed, and cost add up quickly. Anthropic's Claude Fable 5 showed how poorly tuned filters can affect real use. Artificial Analysis found that the system automatically routed eight to nine percent of tasks to a weaker model. One medical physicist called Fable 5 unusable because his work often includes the word "nuclear." Other users reported that the system flagged MRI analysis as bioterrorism. Anthropic tightened the filter after locating a safety issue and says it has since blocked harmless coding tasks more often. Shieldstral gives operators more control over that tradeoff. They can write screening criteria at runtime and tailor the filter to a specific app instead of adopting someone else's categories. These classifiers already play a growing role across the industry. OpenAI uses them for automatic age detection in ChatGPT and routes emotional requests through a safety filter to stricter models. Claude Code uses a classifier to block external scripts, production deployments, and force pushes. Anthropic's Fable 5 review also requires the company to store inputs and outputs for up to 30 days, or up to two years after rule violations. AI News Without the Hype – Curated by Humans Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section. Subscribe now Source: Arxiv | Mistral BETA-TEST × wpDiscuzInsert BETA-TEST × wpDiscuzInsert