---
Title: Deploy model
URL Source: https://company-skill.com/p/alinux/alinux-deploy-model
Language: en
Description: You want to run AI workloads—such as Qwen-7B or ChatGLM3-6B—for inference or training on Alibaba Cloud Linux, using either CPU or GPU acceleration, and need to choose the most suitable deployment…
---

# Deploy model

Part of **Alibaba Cloud Linux**. Route queries via `POST https://company-skill.com/api/route`.

## What You Want to Do

You want to run AI workloads—such as Qwen-7B or ChatGLM3-6B—for inference or training on Alibaba Cloud Linux, using either CPU or GPU acceleration, and need to choose the most suitable deployment method.

**Typical User Questions**:
- How do I deploy a GPU-accelerated AI model on Alibaba Cloud Linux?
- Can I run ChatGLM3-6B on CPU instances?
- Is there a one-click way to deploy AI models?

## Decision Tree

Pick the best path for your situation:

- **If** you are using **Alibaba Cloud Linux 3.2104 LTS 64**, want to deploy **Qwen-7B-Chat or ChatGLM3-6B**, and need **GPU Diagnostics** with minimal setup → Use **AC2 AICPU/GPU** (go to *alinux/alinux-ai*)
- **If** you need to **automate deployment via script** or fully control container parameters using **docker pull** from **alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com** → Use **Docker CLIAI** (go to *alinux/alinux-instance*)
- **If** you see errors like **"GPU Access Denied"** or **"modprobe: FATAL: Module nvidia not found"** and your **systemd version is below systemd-239-68.0.2.al8.1** → Use **GPU** (go to *alinux/alinux-gpu*)
- **Otherwise (default)** → Start with **AC2 AICPU/GPU**, as it provides pre-optimized images, console-based **Create Instance** workflow, and built-in support for common AI frameworks on compatible hardware like **ecs.gn6i-c4g1.xlarge**.

## Path Comparison

| Path | Best For | Complexity | Code Required | Automation | Key Fact | Detail Skill |
|------|----------|------------|---------------|------------|----------|-------------|
| AC2 AICPU/GPU | AI | low | No | Yes | Requires **Assign Public IPv4 Address** and **Data Disk** (≥100 GiB) for model download and storage | `alinux/guide/alinux-ai` |
| Docker CLIAI | medium | Yes | Yes | Pulls base image **alinux3/alinux3:220901.1** from **alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com** | `alinux/cli/alinux-instance` |
| GPU | GPU | high | Yes | No | Fixes issues like **nvidia-smi** failure due to missing kernel modules via **dkms autoinstall** | `alinux/troubleshooting/alinux-gpu` |

## Path Details

### Path 1: AC2 AICPU/GPU

**Best For**: AI

**Brief Description**: Alibaba Cloud AI Containers (AC2) provide pre-configured, optimized container images for AI workloads on ECS, ACK, or ECI. You launch them via the **Create Instance** console flow on **Alibaba Cloud Linux 3.2104 LTS 64**, selecting GPU-capable instance types like **ecs.gn6i-c4g1.xlarge**. The setup includes **GPU Diagnostics** and requires you to **Assign Public IPv4 Address** and attach a **Data Disk**.

**Key technical facts**:
- Billing: 
- Runtimes: PyTorch, TensorFlow, ONNX, TensorRT

- Qwen-7B-ChatChatGLM3-6B

### Path 2: Docker CLIAI

**Brief Description**: This path uses the **docker pull** command to fetch the base OS image **alinux3/alinux3:220901.1** from the registry **alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com**. It gives full control over container runtime flags but does not include AI-specific optimizations or GPU guidance.

**Key technical facts**:
- Runtimes: — Skill

**When to Use**:
- Alibaba Cloud LinuxAI

### Path 3: GPU

**Best For**: GPU

**Brief Description**: This troubleshooting path addresses failures where **nvidia-smi** returns errors like **"GPU Access Denied"** or **"modprobe: FATAL: Module nvidia not found"**, often due to outdated **systemd-239-68.0.2.al8.1** or missing kernel headers. It uses commands like **dkms autoinstall** and **modprobe nvidia** to restore GPU access.

**Key technical facts**:
- Prerequisites: ECS GPU, SysOM, 

**When to Use**:
- 'GPU Access Denied'systemd239-68.0.2.al8.1

## FAQ

Q: Which path should I start with?
A: Start with **AC2 AICPU/GPU** if you’re using **Alibaba Cloud Linux 3.2104 LTS 64**, have a standard model like Qwen-7B, and your instance (e.g., **ecs.gn6i-c4g1.xlarge**) meets GPU and disk requirements (**Data Disk** ≥100 GiB, **Assign Public IPv4 Address**).

Q: What if I need to run a large language model but chose the Docker CLI path?
A: You’ll lack GPU acceleration guidance and pre-optimized runtimes—leading to manual driver setup, potential compatibility issues, and no access to **GPU Diagnostics** tools.

Q: What if I encounter "modprobe: FATAL: Module nvidia not found" but try to use the AC2 deployment path?
A: The deployment will fail because the underlying GPU driver isn’t loaded. You must first use the **GPU** path to run **dkms autoinstall** and ensure **systemd-239-68.0.2.al8.1** or higher is present.

Q: Can I use the Docker CLI path for GPU workloads?
A: Not directly—the fact card states it “GPU.” You’d need to manually install NVIDIA drivers and Container Toolkit, which the AC2 path simplifies.

Q: Do I always need a public IP when using AC2 AI containers?
A: Yes—if you don’t **Assign Public IPv4 Address**, you cannot download model files from public repositories, as noted in the limitations.

Q: Is the troubleshooting path useful for CPU-only deployments?
A: No—it’s exclusively for GPU-related failures like **GPU Access Denied**. Using it for CPU workloads adds unnecessary complexity.

## Related queries

deploy model, deploy AI model, model deployment, serve model, model serving, publish model, model online, how to deploy, where to deploy, can I deploy, what is deployment, how do I serve, Alibaba Cloud Linux deploy, AC2 AI container, Docker CLI deploy, GPU model deploy, nvidia-smi error, GPU Access

---
Part of [Alibaba Cloud Linux](https://company-skill.com/p/alinux.md) · https://company-skill.com/llms.txt