Qualcomm AI Hub 模型實作教學：分類、物件偵測與硬體感知部署

In this tutorial, we work through an end-to-end workflow for Qualcomm AI Hub Models. We start by setting up the required package, discovering the available model collection, and loading MobileNet-V2 for local PyTorch inference. We also handle an important input-shape issue by converting NHWC image tensors into the NCHW format expected by the model. From there, we run inference on both the model’s built-in sample input and a real image, inspect top predictions, execute the official Qualcomm AI Hub CLI demo, and extend the workflow with a YOLOv7 object detection example. Also, we include an optional cloud-device section where we compile, profile, and run the model on a real Qualcomm device when an API token is available. Copy CodeCopiedUse a different Browserimport subprocess, sys, os, glob, textwrap, traceback import numpy as np, torch from PIL import Image import matplotlib.pyplot as plt def pip_install(*pkgs): subprocess.run([sys.executable, "-m", "pip", "install", "-q", *pkgs], check=True) pip_install("qai_hub_models") OUT_DIR = "/content/qaihm_out"; os.makedirs(OUT_DIR, exist_ok=True) torch.set_grad_enabled(False) def to_nchw(value): arr = value[0] if isinstance(value, (list, tuple)) else value t = torch.from_numpy(np.asarray(arr, dtype=np.float32)) if t.ndim == 3: t = t.unsqueeze(0) if t.ndim == 4 and t.shape[1] != 3 and t.shape[-1] == 3: t = t.permute(0, 3, 1, 2).contiguous() return t We begin by importing libraries and setting up a helper function to install packages directly inside Colab. We install qai_hub_models, create an output directory, and disable gradient tracking since we only need inference. We also define the to_nchw() function to convert any input image tensor to the channel-first format expected by the model. Copy CodeCopiedUse a different Browserimport pkgutil, qai_hub_models.models as _m model_ids = sorted(n for _, n, p in pkgutil.iter_modules(_m.__path__) if p and not n.startswith("_")) print(f">>> {len(model_ids)} models available. First 40:\n") print(textwrap.fill(", ".join(model_ids[:40]), 100), "\n") from qai_hub_models.models.mobilenet_v2 import Model as MobileNetV2 model = MobileNetV2.from_pretrained().eval() spec = model.get_input_spec() input_name = list(spec.keys())[0] print(">>> Input:", input_name, spec[input_name].shape, spec[input_name].dtype) from torchvision.models import MobileNet_V2_Weights IMAGENET_CLASSES = MobileNet_V2_Weights.IMAGENET1K_V1.meta["categories"] def top5(logits): if logits.ndim == 1: logits = logits.unsqueeze(0) probs = torch.softmax(logits, dim=1)[0] conf, idx = probs.topk(5) return [(IMAGENET_CLASSES[i], float(c)) for c, i in zip(conf, idx)] We discover the available Qualcomm AI Hub model packages and print the first set of model IDs to understand what is accessible. We then load the pretrained MobileNet-V2 model, read its input specification, and identify the correct input name. We also prepare the ImageNet class labels and define a top5() function to convert model logits into readable top-5 predictions. Copy CodeCopiedUse a different Browsersample = model.sample_inputs() x = to_nchw(sample[input_name]) print(">>> fed tensor shape:", tuple(x.shape)) print("\n>>> Top-5 for the built-in sample input:") for label, conf in top5(model(x)): print(f" {conf:6.2%} {label}") from torchvision import transforms preprocess = transforms.Compose([ transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), ]) img = None try: import urllib.request p = os.path.join(OUT_DIR, "input.jpg") urllib.request.urlretrieve( "https://raw.githubusercontent.com/pytorch/hub/master/images/dog.jpg", p) img = Image.open(p).convert("RGB") except Exception as e: print(">>> photo download skipped:", e) if img is not None: preds = top5(model(preprocess(img).unsqueeze(0))) print("\n>>> Top-5 for the downloaded photo:") for label, conf in preds: print(f" {conf:6.2%} {label}") plt.figure(figsize=(5,5)); plt.imshow(img); plt.axis("off") plt.title(f"{preds[0][0]} ({preds[0][1]:.1%})"); plt.show() We first run inference using the model’s built-in sample input and use to_nchw() to fix the tensor shape before passing it to MobileNet-V2. We then download a real image, preprocess it using standard resizing, cropping, and tensor conversion steps, and run another prediction. We finally display the image with the top predicted label to visually connect the model output to the input photo. Copy CodeCopiedUse a different Browserdef run_demo(module, extra=None, timeout=900): cmd = [sys.executable, "-m", module, "--eval-mode", "fp", "--output-dir", OUT_DIR] + (extra or []) print(f"\n>>> {' '.join(cmd)}") try: r = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout) print("\n".join((r.stdout + r.stderr).strip().splitlines()[-25:])) except Exception as e: print(">>> demo skipped:", e) run_demo("qai_hub_models.models.mobilenet_v2.demo") try: pip_install("qai_hub_models[yolov7]") run_demo("qai_hub_models.models.yolov7.demo") imgs = sorted(glob.glob(OUT_DIR + "/*.png") + glob.glob(OUT_DIR + "/*.jpg"), key=os.path.getmtime) if imgs: plt.figure(figsize=(9,9)); plt.imshow(Image.open(imgs[-1]).convert("RGB")) plt.axis("off"); plt.title("YOLOv7 detections"); plt.show() else: print(">>> no output image found (results may have printed instead).") except Exception: print(">>> YOLOv7 section skipped:\n", traceback.format_exc()) We define a reusable run_demo() function that executes official Qualcomm AI Hub model demos from the command line. We use it to run the MobileNet-V2 demo and then install the YOLOv7 extras for object detection. We run the YOLOv7 demo, search for the generated output image, and visualize the detections if an image is created. Copy CodeCopiedUse a different Browsertry: import qai_hub as hub devices = hub.get_devices() print(f"\n>>> Authenticated. {len(devices)} cloud devices available.") device = hub.Device("Samsung Galaxy S24 (Family)") sample = model.sample_inputs() nchw = to_nchw(sample[input_name]) traced = torch.jit.trace(model, [nchw]) cloud_inputs = {input_name: [nchw.numpy()]} cj = hub.submit_compile_job(model=traced, device=device, input_specs=model.get_input_spec(), options="--target_runtime tflite") target = cj.get_target_model(); print(">>> compiled:", cj.url) pj = hub.submit_profile_job(model=target, device=device); print(">>> profiling:", pj.url) ij = hub.submit_inference_job(model=target, device=device, inputs=cloud_inputs) out = ij.download_output_data() dev_logits = torch.from_numpy(np.asarray(list(out.values())[0][0])) print(">>> Top-5 from the REAL device:") for label, conf in top5(dev_logits): print(f" {conf:6.2%} {label}") target.download(os.path.join(OUT_DIR, "mobilenet_v2.tflite")) print(">>> saved compiled .tflite to", OUT_DIR) except Exception as e: print("\n>>> Cloud (on-device) section skipped — no API token configured.") print(" Get one at workbench.aihub.qualcomm.com, then:") print(" !qai-hub configure --api_token YOUR_TOKEN") print(" detail:", (str(e).splitlines() or [type(e).__name__])[0]) print("\n>>> Tutorial complete. Outputs in:", OUT_DIR) We include an optional Qualcomm AI Hub cloud workflow that runs only when an API token is configured. We retrieve available cloud devices, trace the PyTorch model, compile it for TFLite, profile it on a Qualcomm device, and submit an inference job. We then download the device output, print the top predictions, save the compiled TFLite model, and finish by showing where all tutorial outputs are stored. In conclusion, we have a complete practical workflow for using Qualcomm AI Hub Models inside Colab. We learned how to load pretrained models, prepare inputs correctly, run local inference, visualize classification and detection results, and use the official demos as reproducible reference points. We also saw how the same model can move beyond local PyTorch execution into Qualcomm’s cloud-device pipeline for compilation, profiling, and real-device inference. It provides a path from simple experimentation to hardware-aware deployment with Q