如何在生產環境中保護 AI 代理、MCP 伺服器與 LLM 應用程式
重點摘要
代理、MCP 整合以及 LLM 驅動的應用程式正以超越多數安全團隊追蹤速度的速度湧入程式碼庫。Mend.io 的新實務指南《保護 AI 代理、MCP 伺服器與 LLM 應用程式:實用框架》正是針對此缺口。該指南圍繞三大步驟展開:看清重點、更快修復重點、在生產環境中保護 AI,並提供七個可重複使用的工件。為何傳統應用程式安全(AppSec)會失效?AppSec 建立在一個假設上:應用程式會按照其程式碼的指示執行。但代理式 AI 打破了這個假設。代理行為來自模型、系統提示、檢索到的上下文、使用者輸入以及它可能呼叫的工具。兩個完全相同的部署可能表現出截然不同的行為。故障模式也是全新的。提示注入透過數據而非程式碼進入;權限過大的代理可能造成危害。
Agents, MCP integrations, and LLM-powered applications are entering codebases faster than most security programs can track them.Mend.io’s new practitioner guide, ‘Securing AI agents, MCP servers & LLM apps: A practical framework’, targets that gap.
It is organized around three moves: see what matters, fix what matters faster, protect AI in production and ships seven reusable artifacts.Why traditional AppSec breaks AppSec was built on one assumption: applications do what their code says.Agentic AI breaks it.
Agent behavior emerges from a model, a system prompt, retrieved context, user input, and the tools it may call.Two identical deployments can behave differently.The failure modes are new too.Prompt injection arrives through data, not code.
An over-permissioned agent can take harmful actions without any vulnerability being exploited.A deprecated model keeps serving predictions after its maintainer stops patching it.A poisoned tool description on an MCP server can redirect an agent’s behavior without touching the application.
None appear in a CVE feed.The mandate is two-sided: shift left, and protect right.Artifact 1.
1: the five-layer attack surface map Interaction: user inputs, retrieved documents, inter-agent messages → prompt injection, context poisoning, data exfiltration Agent: system prompts, configs, memory, autonomy settings → over-permissioned tools, unsafe defaults, goal hijacking Integration: MCP servers, tool definitions, plugins, APIs → poisoned tool descriptions, unscoped credentials, shadow servers Model: foundation and fine-tuned models, embeddings → EOL models, supply chain risk, unsafe generations Code: AI-generated code, AI frameworks, SDKs → vulnerable code, framework CVEs, malicious packages See: agent and MCP discovery Agents rarely arrive through procurement.
Three categories to hunt: shadow agents, unregistered MCP servers, and embedded AI frameworks.Every MCP server needs an owner, an access scope, and a review.There are five discovery methods.First, scan repositories for agentic signatures.Second, watch network egress for calls to model API endpoints.
Third, audit service accounts and API keys.Fourth, make declaration cheap via lightweight registration.Finally, automate continuously, since point-in-time discovery goes stale fast.Artifact 2.
1 extends the AI-BOM with nine fields per agent or MCP server: identity, model dependency, autonomy level, tool permissions, credential scope, data reach, MCP endpoints, prompt location, last review.Artifact 2.
2 is a 12-point misconfiguration checklist: credentials scoped to specific resources, not broad service-level access; no shared credentials between agents; high-impact tools requiring human approval; system prompts in version control, not editable in production; MCP servers authenticating clients; tool descriptions reviewed for injection-bearing content before adoption (tool poisoning); model versions pinned with EOL monitoring and an owner.
Fix: prioritization and triage AI expanded the finding surface, not just the attack surface.The pipeline is enrich → prioritize → triage.Prioritization signals, in order of value: reachability, exploitability context, business context, agentic amplification, fix availability.Artifact 3.
1 draws the automation line: DecisionDispositionReachability/dataflow, well-understood classesAutomateFP/TP assessment with evidence trailsAutomate, with samplingTier-3/high-risk applicationsAI-assist, human decidesNovel classes, AI behaviors, no evidenceHuman onlyAccepting risk or deferring a fixHuman only, documented Two rules govern it.
Every automated closure carries evidence; if the system cannot show why something is a false positive, it goes to a human.Error rates get sampled review, with thresholds triggering retraining.
Protect: runtime security Runtime protection involves guardrails, prompt hardening, policy enforcement, and monitoring.Operating as a loop with AI red teaming, red team discoveries improve guardrails, while guardrail logs guide subsequent red teaming.
Guardrails deploy in two ways: via an in-app Python SDK (supporting Online or isolated Offline modes) or as a standalone API Server (Docker) requiring no code changes or Python dependencies.
The minimal viable setup includes inbound guardrails catching prompt injections, out-of-policy requests, and jailbreaks, alongside outbound guardrails catching credentials, PII, proprietary code, unsafe content, and policy violations.
System prompt hardening follows five patterns: assuming disclosure, separating instructions from data, constraining the blast radius, versioning/reviewing, and adversarial testing.
Setting strict permissions is more effective than prompt instructions—preventing tool access removes the need to instruct against dangerous actions.Artifact 4.1 contains seven validation checks.The maturity roadmap Four stages: Emerging, Developing, Controlling, Leading.
It is aligned to NIST AI RMF, OWASP AIMA, ISO/IEC 42001, and the EU AI Act.Artifact 5.1 is a 15-question self-assessment: 0–5 Emerging, 6–10 Developing, 11–13 Controlling, 14–15 Leading.Key takeaways Agent behavior emerges from model, prompt, context, input, and tools — not code alone.
Five risk layers: interaction, agent, integration, model, code.Hunt shadow agents, unregistered MCP servers, embedded AI frameworks.Automate evidence-backed triage; keep risk acceptance and novel findings human-only.Guardrails ship as an embedded Python SDK or a standalone Docker API Server.
Check out the full guide here.Thanks to the Mend.io team for the thought leadership / resources for this article.This article is sponsored by Mend.io.The post How to Secure AI Agents, MCP Servers, and LLM Apps in Production appeared first on MarkTechPost.
Related
相關文章

曝字節訓10億參數大模型,或超Mythos 5,張一鳴、梁汝波先後發聲
字節跳動正在訓練一個參數量高達10萬億的AI模型,規模可能超越Anthropic的Mythos 5。創辦人張一鳴在內部會議中強調編程的關鍵地位,並反對模型蒸餾,認為這只能複製而非超越對手。字節跳動在AI領域持續加大投入,同時在產品端與訓練端採取雙線進攻策略。

AI 需求擠爆雲計算,消息稱 AWS 要求工程師關閉閒置服務器減少資源浪費
因AI需求導致算力緊缺,亞馬遜AWS要求工程師關閉閒置的EC2實例,以減少資源浪費。數據顯示約65%的EC2實例在30天內平均CPU利用率低於20%,AWS因此升級計算優化器自動標記低使用率虛擬機。此外,AWS過去一年新增3.8吉瓦電力容量,仍難以應對GPU雲端實例的龐大需求。
使用 NVIDIA NeMo Retriever、託管 NIM、LanceDB、重新排序與基於事實生成建立多模態 RAG 管線
在本教學中,我們將使用 NVIDIA NeMo Retriever 建立一個先進的多模態檢索增強生成管線。首先設定 Python 3.12 環境、安裝必要套件,並在無需 GPU 或外部 API 金鑰的情況下進行離線 PDF 文字提取。接著,我們透過託管的 NVIDIA NIM 端點來偵測頁面元素、提取表格、圖表與資訊圖形、產生稠密向量嵌入,並將處理後的內容儲存至 LanceDB。最後,我們實作了稠密檢索、視覺語言重新排序、後設資料過濾搜尋、附行內引用的基於事實回應生成,以及輕量級的 recall-at-k 評估,以驗證跨多模態文件內容的檢索品質。
NVIDIA AI 推出 NOOA:將 AI 代理轉化為單一 Python 類別的物件導向框架
NVIDIA 實驗室開源了 NOOA(NVIDIA 物件導向代理),這是一個與模型無關的 Python 框架,用於建構 AI 代理。傳統的代理開發分散在提示模板、工具架構、回呼程式碼和工作流程圖中,而 NOOA 將所有這些整合到一個 Python 類別中:方法代表模型可採取的動作,欄位代表代理狀態,文件字串作為提示,型別註解則是執行時期強制執行的合約。主體為「...」的方法由 LLM 驅動的迴圈在執行時期完成,而具有正常主體的方法則保持確定性的 Python 程式碼。開發者與模型因此共享同一介面,使代理行為能像一般軟體一樣進行測試、追蹤、重構和版本控制。NVIDIA 報告在 SWE-bench Verified 上達到 82.2%,在 CyberGym L1 上達到 86.8%,平均 RHAE 為 85.1%。

六巨頭定AI插件新標準,撞臉Claude,Anthropic沒上桌
六大科技巨頭(AWS、Anysphere、GitHub、微軟、OpenAI、Vercel)聯合發布AI智能體插件統一開放規範Agent Plugins 1.0.0,旨在統一插件打包格式,減少開發者重複勞動。該規範的結構與Anthropic的Claude Code插件系統高度相似,但Anthropic並未參與制定,而是繼續經營自己的封閉生態。

DeepSeek重啟融資,三年市值對齊騰訊?
DeepSeek重啟第二輪融資,以5000億元人民幣估值尋求籌集80億美元,但網傳一份由小型醫藥私募發起的專項基金募資材料引發網友質疑,後經DeepSeek員工證實部分數據屬實。該公司近期宣布API大幅漲價,可能打破其以低價換規模的估值邏輯,面臨客戶流失風險。市場關注其能否從「價格屠夫」轉型為價值提供商,以及三年內市值能否對齊騰訊等巨頭。