MarkTechPost AI生成式AI

如何在生產環境中保護 AI 代理、MCP 伺服器與 LLM 應用程式

2026年8月3日 20:16

重點摘要

代理、MCP 整合以及 LLM 驅動的應用程式正以超越多數安全團隊追蹤速度的速度湧入程式碼庫。Mend.io 的新實務指南《保護 AI 代理、MCP 伺服器與 LLM 應用程式:實用框架》正是針對此缺口。該指南圍繞三大步驟展開:看清重點、更快修復重點、在生產環境中保護 AI,並提供七個可重複使用的工件。為何傳統應用程式安全(AppSec)會失效?AppSec 建立在一個假設上:應用程式會按照其程式碼的指示執行。但代理式 AI 打破了這個假設。代理行為來自模型、系統提示、檢索到的上下文、使用者輸入以及它可能呼叫的工具。兩個完全相同的部署可能表現出截然不同的行為。故障模式也是全新的。提示注入透過數據而非程式碼進入;權限過大的代理可能造成危害。

站內 AI 整理稿

Agents, MCP integrations, and LLM-powered applications are entering codebases faster than most security programs can track them. Mend.io’s new practitioner guide, ‘Securing AI agents, MCP servers & LLM apps: A practical framework’, targets that gap. It is organized around three moves: see what matters, fix what matters faster, protect AI in production and ships seven reusable artifacts. Why traditional AppSec breaks AppSec was built on one assumption: applications do what their code says. Agentic AI breaks it. Agent behavior emerges from a model, a system prompt, retrieved context, user input, and the tools it may call. Two identical deployments can behave differently. The failure modes are new too. Prompt injection arrives through data, not code. An over-permissioned agent can take harmful actions without any vulnerability being exploited. A deprecated model keeps serving predictions after its maintainer stops patching it. A poisoned tool description on an MCP server can redirect an agent’s behavior without touching the application. None appear in a CVE feed. The mandate is two-sided: shift left, and protect right. Artifact 1.1: the five-layer attack surface map Interaction: user inputs, retrieved documents, inter-agent messages → prompt injection, context poisoning, data exfiltration Agent: system prompts, configs, memory, autonomy settings → over-permissioned tools, unsafe defaults, goal hijacking Integration: MCP servers, tool definitions, plugins, APIs → poisoned tool descriptions, unscoped credentials, shadow servers Model: foundation and fine-tuned models, embeddings → EOL models, supply chain risk, unsafe generations Code: AI-generated code, AI frameworks, SDKs → vulnerable code, framework CVEs, malicious packages See: agent and MCP discovery Agents rarely arrive through procurement. Three categories to hunt: shadow agents, unregistered MCP servers, and embedded AI frameworks. Every MCP server needs an owner, an access scope, and a review. There are five discovery methods. First, scan repositories for agentic signatures. Second, watch network egress for calls to model API endpoints. Third, audit service accounts and API keys. Fourth, make declaration cheap via lightweight registration. Finally, automate continuously, since point-in-time discovery goes stale fast. Artifact 2.1 extends the AI-BOM with nine fields per agent or MCP server: identity, model dependency, autonomy level, tool permissions, credential scope, data reach, MCP endpoints, prompt location, last review. Artifact 2.2 is a 12-point misconfiguration checklist: credentials scoped to specific resources, not broad service-level access; no shared credentials between agents; high-impact tools requiring human approval; system prompts in version control, not editable in production; MCP servers authenticating clients; tool descriptions reviewed for injection-bearing content before adoption (tool poisoning); model versions pinned with EOL monitoring and an owner. Fix: prioritization and triage AI expanded the finding surface, not just the attack surface. The pipeline is enrich → prioritize → triage. Prioritization signals, in order of value: reachability, exploitability context, business context, agentic amplification, fix availability. Artifact 3.1 draws the automation line: DecisionDispositionReachability/dataflow, well-understood classesAutomateFP/TP assessment with evidence trailsAutomate, with samplingTier-3/high-risk applicationsAI-assist, human decidesNovel classes, AI behaviors, no evidenceHuman onlyAccepting risk or deferring a fixHuman only, documented Two rules govern it. Every automated closure carries evidence; if the system cannot show why something is a false positive, it goes to a human. Error rates get sampled review, with thresholds triggering retraining. Protect: runtime security Runtime protection involves guardrails, prompt hardening, policy enforcement, and monitoring. Operating as a loop with AI red teaming, red team discoveries improve guardrails, while guardrail logs guide subsequent red teaming. Guardrails deploy in two ways: via an in-app Python SDK (supporting Online or isolated Offline modes) or as a standalone API Server (Docker) requiring no code changes or Python dependencies. The minimal viable setup includes inbound guardrails catching prompt injections, out-of-policy requests, and jailbreaks, alongside outbound guardrails catching credentials, PII, proprietary code, unsafe content, and policy violations. System prompt hardening follows five patterns: assuming disclosure, separating instructions from data, constraining the blast radius, versioning/reviewing, and adversarial testing. Setting strict permissions is more effective than prompt instructions—preventing tool access removes the need to instruct against dangerous actions. Artifact 4.1 contains seven validation checks. The maturity roadmap Four stages: Emerging, Developing, Controlling, Leading. It is aligned to NIST AI RMF, OWASP AIMA, ISO/IEC 42001, and the EU AI Act. Artifact 5.1 is a 15-question self-assessment: 0–5 Emerging, 6–10 Developing, 11–13 Controlling, 14–15 Leading. Key takeaways Agent behavior emerges from model, prompt, context, input, and tools — not code alone. Five risk layers: interaction, agent, integration, model, code. Hunt shadow agents, unregistered MCP servers, embedded AI frameworks. Automate evidence-backed triage; keep risk acceptance and novel findings human-only. Guardrails ship as an embedded Python SDK or a standalone Docker API Server. Check out the full guide here. Thanks to the Mend.io team for the thought leadership / resources for this article. This article is sponsored by Mend.io. The post How to Secure AI Agents, MCP Servers, and LLM Apps in Production appeared first on MarkTechPost.

Related

相關文章

智東西生成式AI

又一國產模型重磅開源!有聲視頻編輯全球第一,16家芯片及平臺首日適配

智東西(公眾號:zhidxcom) 作者 | 楊京麗 編輯 | 李水青 智東西8月3日報道,今日,MiniMax正式開源新一代通用視頻模型MiniMax H3。 MiniMax H3是一個通用型全模態生成系統,可統一理解文本、圖像、視頻和音頻組成的多模態上下文,生成最長15秒、最高2K分辨率並帶有原生立體聲音頻的視頻。 此前7月31日,MiniMax H3發佈。

1 小時前
MarkTechPost AI生成式AI

使用穩健數據載入與自動評判,以Moonshot PerceptionBench評估多模態視覺模型

本教學設計了一個端到端評估流程,用於Moonshot PerceptionBench,這是一個衡量多模態視覺模型細粒度感知能力的基準,涵蓋OCR、計數、定位、上下文推理等任務。流程包含穩健的數據載入、多種後端模型支援(如盲測基準、OpenAI相容API及本地Hugging Face模型),以及基於規則或LLM輔助的自動評判機制,最終產出可重現的預測與報告。

1 小時前

豆包輸入法撕開語音口子,扒開科大訊飛外衣

# 豆包輸入法撕開語音口子,科大訊飛的護城河為何失效? 大模型戰火正在燒向一個用戶習以為常、卻高頻剛需的角落——輸入法。這個被視為“最後一公里”的關鍵入口,迎來了新一轮殘酷洗牌。當搜狗、訊飛、百度、微信等老牌廠商佔據超過84%市場份額時,看似固若金湯的格局,卻被一款新產品迅速撕開裂口:豆包輸入法。 據MobTech研究院數據,2025年11月豆包輸入法上線首周下載量即突破5000萬,日活躍用戶衝上1200萬,刷新了輸入法新品增長紀錄。

3 小時前

Claude 焚書又“越獄”邊界在哪?

# Claude 焚书又“越獄”,Anthropic的邊界何在? **——從“巴拿馬計劃”到模型逃逸,AI巨頭的數據饑渴正同時撕開物理與數字邊界** 2026年7月底,人工智能公司Anthropic的兩件事同時被推至聚光燈下,分別觸及物理世界與數字世界的邊界。一方面,法庭解封的“巴拿馬計劃”內部文件顯示,這家公司通過二手書商大宗採購實體書籍,用液壓機切除書脊、掃描內容,再將紙張送入碎紙機,試圖將50萬至200萬冊書籍“破壞性掃描”為訓練數據。

3 小時前

靠做“中間商”生意估值20億美元,Liblib跑出了奇蹟還是泡沫?

在AI領域普遍追求「得大模型者得天下」的浪潮中,一家不走尋常路的應用層公司卻交出了截然不同的成績單。演語科技(Evoken)於6月18日宣布完成近3億美元的B+輪融資,投後估值一舉突破20億美元,成為中國AI應用層至今為止規模最大的單輪融資之一。多數人或許對演語科技這個集團品牌感到陌生,但其旗下產品LiblibAI以及Liblib這個名稱,在業界早已廣為人知。此次融資也是演語科技首次以集團身分對外發聲。 在大量AI初創公司仍依賴投資人耐心爭取時間窗口的當下,這家沒有自研大模型的企業卻成了最會賺錢的AI應用之一。

3 小時前

大模型公司估值,正在降權「SOTA敘事」

7月之后,资本市场对大模型公司的估值逻辑正在发生显著变化:单一大模型在榜单上的领先排名,已经不足以支撑长期估值溢价。8月3日,阿里巴巴发布Qwen 3.8-Max版正式版,港股当日上涨6.75%;而在此前半个月,Kimi K3的发布却让智谱和MiniMax股价坐上过山车。同样是竞争对手发布新模型,市场给出的反应截然不同,这背后折射出投资者对“SOTA叙事”的降权。 7月17日,Kimi K3发布,其在Artificial Analysis(AA)的编程评分一度位列第一。这一排名直接冲击了智谱依靠GLM-5.

3 小時前