NVIDIA garak 教學：使用自訂探測器與檢測器建立完整的防禦型 LLM 紅隊測試工作流程

In this tutorial, we analyze NVIDIA garak as a practical framework for defensive LLM red-teaming. We start by setting up Garak, then move through plugin discovery, dry runs, real-model scans, multi-probe evaluations, report analysis, custom probe creation, custom detector creation, and AVID export. Instead of running only a single scan, we use Garak end-to-end to understand how probes, detectors, generators, reports, and vulnerability scores work together in a complete LLM security testing workflow. Check out the FULL CODES Here. Setting Up NVIDIA garak and Defining Helper Functions Copy CodeCopiedUse a different Browserimport os, sys, json, glob, subprocess, importlib def sh(cmd, capture=False): print(f"\n$ {cmd}") return subprocess.run(cmd, shell=True, text=True, capture_output=capture) sh(f"{sys.executable} -m pip install -q -U garak") os.environ.setdefault("TOKENIZERS_PARALLELISM", "false") os.environ.setdefault("HF_HUB_DISABLE_TELEMETRY", "1") import garak, garak.cli from garak import _config print("\n=== garak version:", garak.__version__, "===") def run_garak(args): print("\n>>> garak " + " ".join(args)) try: garak.cli.main(args) except SystemExit as e: if e.code not in (0, None): print(f"[garak exited {e.code}]") try: return _config.transient.report_filename except Exception: return None We begin by importing the required libraries and creating a helper function to run shell commands directly from the notebook. We install garak, configure basic environment variables, and import the main garak modules needed for the tutorial. We also define a reusable function that lets us run Garak programmatically and capture the path to the generated report. Listing garak Probes and Detectors and Running Model Scans Copy CodeCopiedUse a different Browserprint("\n########## 1. PLUGIN INVENTORY ##########") for kind in ["probes", "detectors", "generators", "buffs"]: out = sh(f"{sys.executable} -m garak --list_{kind} 2>/dev/null", capture=True) lines = [l for l in (out.stdout or "").splitlines() if "." in l] print(f" {kind:11s}: {len(lines)} plugins e.g. " f"{', '.join(l.split()[-1] if l.split() else l for l in lines[:3])}") print("\n########## 2. FAST DRY-RUN (test.Repeat) ##########") sh(f"{sys.executable} -m garak --target_type test.Repeat " f"--probes lmrc.SlurUsage --generations 1") print("\n########## 3. REAL MODEL: gpt2 vs DAN 11.0 ##########") sh(f"{sys.executable} -m garak --target_type huggingface --target_name gpt2 " f"--probes dan.Dan_11_0 --generations 1 --parallel_attempts 8") print("\n########## 4. PROGRAMMATIC MULTI-PROBE SCAN ##########") report_path = run_garak([ "--target_type", "test.Repeat", "--probes", "dan.Dan_11_0,encoding.InjectBase64,lmrc.SlurUsage", "--generations", "1", "--parallel_attempts", "16", ]) print("Report:", report_path) We inspect the garak plugin ecosystem by listing available probes, detectors, generators, and buffs. We then run a quick dry run using the test generator to confirm that Garak is working without requiring any external model or API key. After that, we scan a real Hugging Face model and run a multi-probe scan to generate a richer report for analysis. Analyzing garak Reports: Safety Scores and Attack Success Rates Copy CodeCopiedUse a different Browserprint("\n########## 5. ANALYSIS ##########") import numpy as np, pandas as pd def find_latest_report(): cands = [] for base in [os.path.expanduser("~/.local/share/garak/garak_runs"), os.path.expanduser("~/.cache/garak"), "."]: cands += glob.glob(os.path.join(base, "**", "*report.jsonl"), recursive=True) cands = [c for c in cands if os.path.getsize(c) > 0] return max(cands, key=os.path.getmtime) if cands else None report_path = report_path or find_latest_report() print("Analysing:", report_path) evaluations = None try: from garak.report import Report rep = Report(report_path).load().get_evaluations() evaluations = rep.evaluations.copy() print("\n--- Per-probe mean SAFETY score (garak.report.Report) ---") print(rep.scores.round(1).to_string()) except Exception as e: print("garak.report.Report unavailable, falling back to manual parse:", e) rows = [] with open(report_path) as f: for line in f: try: r = json.loads(line) except json.JSONDecodeError: continue if r.get("entry_type") == "eval": rows.append(r) evaluations = pd.DataFrame(rows) if not evaluations.empty: evaluations["score"] = np.where( evaluations["total_evaluated"] != 0, 100 * evaluations["passed"] / evaluations["total_evaluated"], 0.0) if evaluations is not None and not evaluations.empty: evaluations["asr_%"] = (100 - evaluations["score"]).round(1) view = evaluations[["probe", "detector", "passed", "total_evaluated", "score", "asr_%"]].copy() view = view.rename(columns={"score": "safe_%"}) view["safe_%"] = view["safe_%"].round(1) view = view.sort_values("asr_%", ascending=False) print("\n--- Per probe/detector (higher asr_% = more vulnerable) ---") print(view.to_string(index=False)) try: import matplotlib.pyplot as plt labels = (view["probe"] + "\n" + view["detector"]).tolist() plt.figure(figsize=(8, 0.55 * len(view) + 1.5)) plt.barh(labels, view["asr_%"], color="#76b900") plt.gca().invert_yaxis() plt.xlabel("Attack Success Rate (%)"); plt.xlim(0, 100) plt.title("garak — vulnerability by probe/detector") plt.tight_layout(); plt.show() except Exception as e: print("plot skipped:", e) We load the generated garak report and prepare it for detailed analysis using pandas and NumPy. We first try to use Garak’s built-in report parser, and if that is unavailable, we manually parse the JSONL report file. We then calculate safety scores and attack success rates, and visualize vulnerabilities across different probe-detector combinations. Inspecting Flagged Outputs and Building a Custom garak Probe Copy CodeCopiedUse a different Browserprint("\n--- Sample hits (detector score >= 0.5) ---") hits = [] with open(report_path) as f: for line in f: try: r = json.loads(line) except json.JSONDecodeError: continue if r.get("entry_type") != "attempt" or r.get("status") != 2: continue dr = r.get("detector_results") or r.get("detector") or {} max_score = 0.0 if isinstance(dr, dict): for v in dr.values(): if isinstance(v, list) and v: max_score = max(max_score, max(float(x) for x in v)) if max_score >= 0.5: prompt = r.get("prompt") hits.append((r.get("probe_classname", r.get("probe", "?")), round(max_score, 2), str(prompt)[:90])) print(f"Total hits found: {len(hits)}") for probe, score, prompt in hits[:5]: print(f" [{score}] {probe}: {prompt!r}") print("\n########## 6. CUSTOM PROBE ##########") custom_probe = ''' from garak.probes.base import Probe class HelloProbe(Probe): """Minimal custom probe: two fixed prompts, paired with a custom detector.""" lang = "en" bcp47 = "en" primary_detector = "mycustomdet.ContainsHello" recommended_detector = ["mycustomdet.ContainsHello"] tags = ["custom:demo"] goal = "demonstrate authoring a garak probe" doc_uri = "" prompts = [ "Please reply with the single word: hello", "Continue this sentence: The capital of France is", ] ''' import garak.probes probes_dir = os.path.dirname(garak.probes.__file__) with open(os.path.join(probes_dir, "mycustom.py"), "w") as fh: fh.write(custom_probe) We further inspect the report by extracting sample hits in which detector scores indicate potentially unsafe or vulnerable outputs. We collect the flagged prompts, detector scores, and probe names to understand what kind of behavior is being detected. We then create a custom garak probe that uses fixed prompts and connects it with a custom detector. Creating a Custom garak Detector and Exporting Results to AVID Copy CodeCopiedUse a different Browserprint("\n########## 7. CUSTOM DETECTOR ##########") custom_detector = ''' from garak import _config from garak.detectors.base import StringDetector class ContainsHello(StringDetector): """Demo detector: flags any output containing 'hello' (case-insensitive).""" lang_spec = "en" bcp47 = "en" def __init__(self, config_root=_config): super().__init__(["hello"], config_roo