AMD ROCm AI Infrastructure on Ubuntu 26.04

This is Part 4 of the Ubuntu 26.04 AI Infrastructure series, and it covers a different accelerator path: AMD GPUs driven by ROCm. If you followed Parts 1–3 you already know how to build a base node and stand up an NVIDIA CUDA stack. This lesson treats AMD as its own first-class path — a separate machine with its own driver, runtime, and diagnostics — not as a footnote to NVIDIA.

What You’ll Learn

What ROCm is — AMD’s open software platform for GPU compute and AI, and where it sits in your stack.
Identifying an AMD GPU — confirming the card exists on the PCI bus and reading what Linux reports.
The AMD GPU software stack — how an AI workload reaches the silicon, layer by layer.
Whether a GPU is supported — the single most important check for AMD, and how to do it correctly before you spend time or money.
Installing ROCm on Ubuntu — the AMD-documented package flow, once support is confirmed.
Using ROCm diagnostics — rocminfo and rocm-smi, and what their fields mean.
Verifying acceleration — proving the GPU actually does compute work.
Running a PyTorch workload on ROCm — including the one detail that surprises everyone the first time.
CUDA vs ROCm at an infrastructure level — a balanced, brand-neutral comparison to help you choose.
Troubleshooting ROCm — a repeatable, layer-by-layer diagnostic method.

We assume you finished Parts 1–3. If you have not built a base node yet, start with Building Your First Ubuntu 26.04 AI Server and come back.

The AMD GPU Software Stack

Before any commands, picture how an AI workload reaches an AMD GPU. Every layer here has a job, and when acceleration “does not work,” the failure is almost always one specific layer — not the whole stack.

  AI Application
        |
        v
     PyTorch
        |
        v
      ROCm
        |
        v
  AMD GPU Driver (amdgpu)
        |
        v
    Linux Kernel
        |
        v
     AMD GPU

A few terms you will meet as you work down this stack:

ROCm — AMD’s open GPU-compute platform. It is the AMD counterpart to the role CUDA plays on NVIDIA: the runtime, libraries, and tools that let software use the GPU for math.
HIP — AMD’s portable GPU programming model inside ROCm. You do not have to write HIP, but you will see it in package names, environment variables, and framework builds, so it helps to recognize it.
ROCm libraries — the math and communication libraries frameworks call under the hood (linear algebra, deep-learning primitives, and so on).
rocminfo — a diagnostic tool that enumerates the GPU compute agents ROCm can see, with model and memory details.
rocm-smi — the “system management interface” tool: live temperature, clocks, utilization, and memory, analogous in spirit to nvidia-smi.

Those four tool names — HIP, the ROCm libraries, rocminfo, and rocm-smi — are the current, verified surface you will use in this lesson.

What Is AMD ROCm?

ROCm (Radeon Open Compute) is AMD’s open software platform for GPU compute and AI/HPC workloads. Where Part 3’s CUDA is NVIDIA’s proprietary compute ecosystem, ROCm is AMD’s — and it is open source, which is part of its identity, not just a marketing line. For a DevOps engineer, ROCm is the set of software that turns an AMD GPU from “a card Linux can see” into “a card your AI framework can compute on.”

ROCm bundles several things you will interact with as infrastructure:

HIP, the portable programming model that lets GPU code target AMD hardware.
A runtime that schedules and executes work on the GPU.
Math libraries (linear algebra, deep-learning primitives, collective communication) that frameworks like PyTorch call into.
Framework support, most importantly a PyTorch build compiled against ROCm.
GPU management and diagnostic tools — rocminfo and rocm-smi — for inventory and live health.

You do not need to master ROCm’s internals to operate an AMD AI node. You need to know which layer each tool inspects, how to confirm each layer is healthy, and how the layers connect — which is exactly how we will work through it.

Detecting an AMD GPU

Same first principle as every hardware lesson in this series: confirm the OS sees the card on the PCI bus before you touch drivers. On the AMD path we grep for AMD’s vendor names:

lspci | grep -Ei 'amd|ati'

lspci lists every device on the PCI bus. Piping it through grep -Ei 'amd|ati' filters case-insensitively for AMD’s current and legacy vendor strings (AMD GPUs still commonly report under the historical “ATI” name). If you see a VGA compatible controller or Display controller line naming an AMD/Radeon device, Linux sees the hardware. That is the win for this step — it does not require or imply that any driver is installed yet.

One thing that trips people up: an AMD system can expose several AMD PCI devices, not just one GPU line. Depending on the platform you may see an integrated graphics device, audio functions tied to the GPU, and the discrete GPU itself — all matching your filter. Read the descriptions rather than assuming the first match is your compute GPU; the line you care about is the discrete display/3D controller.

To see which kernel driver is currently bound to the device, ask lspci for verbose output on that specific device:

lspci -k | grep -EA3 'VGA|3D|Display'

lspci -k adds kernel driver information; the Kernel driver in use: line in the output tells you what is currently driving the card. On a configured AMD compute node you want that to read amdgpu. Early on — before install — it may show a generic driver or none at all, which is expected.

ROCm Hardware Compatibility (Read This First)

This is the most important section in the lesson, and it is the step people skip and then regret. On the AMD path, compatibility is not automatic. Do not assume that because a Radeon or workstation card is powerful, or because it works fine for graphics, it is supported by the ROCm release you are about to install. ROCm officially supports a specific list of GPU models per release, and that list changes over time. A card outside the list may work partially, unpredictably, or not at all for compute — and diagnosing that after the fact wastes hours.

The same applies to the operating system. ROCm supports specific Ubuntu versions per release, and this is where an AMD node differs meaningfully from the NVIDIA path you built in Part 3.

❗ Important — Before you purchase or configure an AMD GPU for ROCm, verify the exact GPU model and your Ubuntu version against AMD’s current compatibility matrix at https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html. As of this writing, the current ROCm release officially supports Ubuntu 24.04 and 22.04 LTS, and Ubuntu 26.04 is not yet on ROCm’s supported-OS list. Do not assume 26.04 is supported just because it is the newest LTS. If it is not yet listed when you check, you have two clean options: run the AMD node on a supported Ubuntu LTS (24.04) until ROCm adds 26.04, or wait for that support to land. The compatibility matrix is the authority — check it, do not guess.

That is the honest state of things. The install flow below is the correct, AMD-documented method, and it is what you will run once you have confirmed your GPU and OS are supported. If your target is 26.04 specifically and ROCm has not added it yet, standing up the AMD node on 24.04 is the pragmatic call, and every concept in this lesson still applies unchanged.

Installing ROCm

Once the compatibility matrix says your GPU and Ubuntu version are supported, this is the AMD-documented installation flow. We use AMD’s packaging, not a random .run file, for the same reason we used Ubuntu’s driver tooling in Part 3: packaged installs are reproducible, upgradeable, and integrate with the system rather than fighting it.

1. Register AMD’s repository. AMD distributes an amdgpu-install package from repo.radeon.com that adds AMD’s apt repositories and provides a helper for selecting components. Get the current installer package and version from AMD’s documentation rather than copying a filename from a blog — the exact amdgpu-install_<version>.deb name goes stale with every release, and installing a mismatched one is a common self-inflicted failure. Install the one AMD’s current docs point to, then refresh apt:

sudo apt update

2. Install matching kernel headers and extra modules. The AMD kernel driver is built against your running kernel, so it needs the headers and extra modules for that exact kernel:

sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"

$(uname -r) expands to your running kernel version, so these always match the kernel you are actually on. This matters: if the kernel later updates, the driver may need to rebuild against the new one.

3. Install the AMD kernel driver. amdgpu-dkms is the kernel-mode driver packaged via DKMS (Dynamic Kernel Module Support), which automatically rebuilds the module when the kernel changes:

sudo apt install amdgpu-dkms

4. Add yourself to the GPU access groups. ROCm exposes the GPU through device files owned by the render and video groups. Your user needs membership to use the GPU without root:

sudo usermod -a -G render,video $LOGNAME

$LOGNAME is your current username. The -a -G flags append you to those groups without removing you from any others (leaving out -a would replace your group list — a classic footgun).

5. Install the ROCm stack. This pulls in the runtime, math libraries, and the diagnostic tools:

sudo apt install rocm

6. Reboot. A reboot cleanly loads the new amdgpu kernel driver and applies your new group membership in a fresh session:

sudo reboot

🛠️ DevOps Tip — On some setups ROCm’s binaries are not on your default PATH. If rocminfo or rocm-smi are “not found” after install, they are likely under a ROCm directory such as /opt/rocm/bin. Confirm the path AMD’s docs specify for your release and add it to PATH in your shell profile rather than calling the tools by absolute path forever. Do not cargo-cult environment variables you do not need — only add what your install actually requires.

The amdgpu Kernel Driver

amdgpu is the Linux kernel driver for AMD GPUs. It is the layer that turns the raw PCI device into something the rest of the stack can talk to: it manages the hardware, exposes the compute device nodes, and handles memory and scheduling at the kernel level. Everything above it — ROCm, then your framework — depends on amdgpu being loaded and healthy. If this layer is down, nothing above it can possibly work, so it is the first thing to check when a GPU “disappears.”

Confirm the module is loaded:

lsmod | grep amdgpu

lsmod lists loaded kernel modules; grepping for amdgpu should return one or more lines showing the module and its size and dependents. No output means the driver is not loaded — and that, not ROCm, is your problem to solve first.

The dependency direction is worth keeping in your head:

  Hardware
     |
     v
   amdgpu   (kernel driver)
     |
     v
    ROCm    (runtime + libraries)
     |
     v
  PyTorch   (framework)

Read failures top-down: hardware present, then driver loaded, then ROCm sees it, then the framework sees it. A break at any level makes every level above it look broken.

Validating ROCm

With amdgpu loaded and ROCm installed, two tools confirm the runtime can actually see and manage the GPU.

rocminfo enumerates the compute agents ROCm can address, including the GPU model and memory. Run it:

rocminfo

The output is long; the part you care about is the GPU agent block. An example excerpt looks like this:

*** Agent 2 ***
  Name:                    gfx____
  Marketing Name:          AMD Radeon ...
  Device Type:             GPU
  Compute Unit:            <count>
  ...
  Pool 1
    Segment:               GLOBAL; FLAGS: ...
    Size:                   <VRAM in KB>

The key fields: Device Type: GPU confirms ROCm found a compute-capable GPU (not just the CPU agent it also lists), the Name/Marketing Name identify the model, and the memory pool size tells you the VRAM ROCm sees. If your GPU appears here as a GPU agent, the runtime is talking to the hardware.

rocm-smi is the live health and management view:

rocm-smi

It prints a table of per-GPU status. An example:

GPU  Temp   Power   SCLK    MCLK   Fan   Perf   VRAM%   GPU%
0    45.0c  35.0W   1500Mhz 1000Mhz 20%  auto   10%     3%

The fields to read: Temp (die temperature — sustained AI work runs hot, and a throttling card just looks “slow”), Power draw, SCLK/MCLK (GPU and memory clocks), VRAM% (how full GPU memory is — your ceiling for model size), and GPU% (utilization; near zero while a job “runs” usually means the work never reached the GPU). This is the tool you will keep open in a second terminal while a workload runs.

✅ Validation — ROCm is working at the platform level when: lsmod shows amdgpu loaded, rocminfo lists your GPU as a Device Type: GPU agent, and rocm-smi reports live temperature and clocks. All three green means the stack is ready for a framework.

HIP, Briefly

HIP (Heterogeneous-compute Interface for Portability) is AMD’s portable GPU programming model inside ROCm. Its purpose is to let GPU code be written once and run on AMD hardware (and, by design, be portable from CUDA-style code). This is a DevOps lesson, not a programming course, so you will not write HIP here — but you will encounter it, and recognizing it saves confusion:

Package and library names in the ROCm stack reference HIP.
Environment variables that tune or target the GPU are sometimes HIP-prefixed.
Error messages from frameworks may mention HIP when GPU calls fail.
Application requirements will sometimes state a HIP or ROCm version.

When you see “HIP” in a log or a package, read it as “the AMD GPU compute layer,” and check the ROCm layer accordingly. That is all the mental model you need as an operator.

Running PyTorch on ROCm

Now the payoff: running a real AI framework on the AMD GPU. The shape of this mirrors the NVIDIA path from Part 3, with an AMD-specific install and one important twist at the end.

  Ubuntu
    |
  amdgpu
    |
   ROCm
    |
   venv  (/opt/ai-lab)
    |
  PyTorch ROCm build
    |
  AMD GPU

Create an isolated environment. Never install AI packages into the system Python — a bad upgrade there can break system tooling. Use a virtual environment in the lab directory:

sudo mkdir -p /opt/ai-lab
sudo chown $LOGNAME:$LOGNAME /opt/ai-lab
python3 -m venv /opt/ai-lab/venv
source /opt/ai-lab/venv/bin/activate

python3 -m venv creates a self-contained Python with its own packages; source .../activate switches your shell into it. Everything you pip install now stays in /opt/ai-lab/venv and touches nothing system-wide.

Install the ROCm build of PyTorch. PyTorch ships hardware-specific builds, and the AMD one is selected by a ROCm index URL. The install pattern is:

pip install torch --index-url https://download.pytorch.org/whl/<rocmX.Y>

Replace <rocmX.Y> with the current ROCm build tag (for example a rocm6.x-style tag). Get the exact current tag from the selector at https://pytorch.org/get-started/locally — the tag shown in any tutorial is an example and moves with releases. Choosing the wrong tag is the most common reason PyTorch installs “fine” but never sees the GPU.

Here is the twist that surprises everyone. PyTorch exposes AMD GPU acceleration through the same torch.cuda API you would use on NVIDIA. The ROCm build maps those calls onto AMD hardware via HIP under the hood, so you do not call a different “torch.rocm” interface — there isn’t one for day-to-day use. That means the readiness check is still:

torch.cuda.is_available()

Read cuda here as “the GPU accelerator API,” not “NVIDIA CUDA the product.” It returning True on a ROCm build means PyTorch is talking to your AMD GPU. This abstraction is deliberate: framework code stays portable across vendors, and the ROCm build quietly does the translation.

Validate it. Save this script and run it inside the venv:

import torch
print(torch.__version__)
print("GPU available:", torch.cuda.is_available())
if torch.cuda.is_available():
    print("Device:", torch.cuda.get_device_name(0))
    x = torch.rand(3, 3, device="cuda"); print((x @ x).sum().item())

torch.__version__ should show a ROCm build. torch.cuda.is_available() should print True. get_device_name(0) should name your AMD GPU. The final line allocates a tensor on the GPU and multiplies it — if it prints a number without error, real compute ran on the AMD card. While it runs, glance at rocm-smi in another terminal and watch GPU% move.

CUDA vs ROCm: A DevOps Comparison

You will be asked “should we use NVIDIA or AMD?” more than once. The honest answer is “it depends on compatibility and workload,” and this table frames the tradeoffs at an infrastructure level without cheerleading for either vendor.

Dimension	CUDA (NVIDIA)	ROCm (AMD)
GPU vendor	NVIDIA	AMD
Compute ecosystem	CUDA	ROCm
Linux AI support	Strong	Strong, but hardware-specific
Framework support	Broad	Growing, workload-dependent
Container support	Yes	Yes
Kubernetes support	Yes	Yes
Hardware validation	Required	Especially important
Primary driver ecosystem	NVIDIA driver	amdgpu + ROCm

A note on honesty: NVIDIA has a longer head start in the AI tooling ecosystem, and AMD’s ROCm support is real and growing but more sensitive to your exact GPU model and workload. That is a compatibility statement, not a market-share claim — verify the specifics for your case rather than trusting any blanket “X wins” assertion.

Choose CUDA When…

Your frameworks, models, or vendor tooling document CUDA as the primary or only supported path.
You want the broadest set of pre-built, well-trodden framework builds.
Your team’s existing runbooks, images, and expertise are already CUDA-based.

Choose ROCm When…

You have (or can get) AMD GPUs that appear on ROCm’s current compatibility matrix.
Your workload and frameworks are confirmed supported on ROCm.
An open compute stack, or AMD hardware availability and pricing in your environment, favors the AMD path.

Choose Based on Compatibility, Not Brand Loyalty

The right decision is driven by facts about your workload, not allegiance to a logo. Weigh:

Supported framework — is your framework officially built for the stack?
Model runtime — do the specific models you run have a supported path?
Workload — training vs inference, and at what scale?
Memory — does the GPU’s VRAM fit your models?
Budget — total cost, not sticker price.
Hardware availability — what can you actually procure?
Operational support — who supports it in production, and how?

Considering NVIDIA instead? See Part 3: NVIDIA GPUs and CUDA on Ubuntu 26.04 for the CUDA side of this same node build, then compare against your compatibility findings here.

User and Device Permissions

ROCm reaches the GPU through device files, and access to them is controlled by group membership, not by loosening file permissions. Two groups matter:

render — grants access to the GPU’s compute/render nodes.
video — grants access to the video/display device nodes.

You added yourself to both during install with usermod -a -G render,video. The catch: new group membership does not apply to your current session. Linux fixes your group list at login, so you must start a fresh session — log out and back in, or reboot — before the new groups take effect. This is the single most common “I did everything right and it still says permission denied” cause on AMD nodes.

Inspect the device ownership instead of guessing:

ls -l /dev/kfd /dev/dri

/dev/kfd is the ROCm compute device and /dev/dri holds the render nodes. The listing shows their owning group — typically render and video. If your user is in those groups (groups confirms it) and the devices are owned by them, permissions are correct. If groups does not yet list them, your session simply has not refreshed.

⚠️ Warning — Never “fix” GPU access with chmod 777 on /dev/kfd, /dev/dri, or anything under them. It does not correctly solve the problem, it silently opens the GPU to every user and process on the box, and it can break on reboot when the devices are recreated. Use group membership and a fresh session — that is the supported, secure fix.

Troubleshooting ROCm

Every ROCm problem is easier when you walk the same ladder, top to bottom, and change one thing at a time. Find the lowest layer that fails; that is your real problem.

  Linux sees GPU?      (lspci)
        |
        v
  amdgpu loaded?       (lsmod | grep amdgpu)
        |
        v
  ROCm sees GPU?       (rocminfo / rocm-smi)
        |
        v
  Framework sees GPU?  (torch.cuda.is_available)
        |
        v
     Application

AMD GPU in PCI but ROCm can’t see it

Likely Cause: amdgpu not loaded, or an unsupported GPU/OS combination.
Check: lspci | grep -Ei 'amd|ati' (present), then lsmod | grep amdgpu.
Fix: Ensure amdgpu-dkms installed and the driver loads; confirm the GPU and Ubuntu version on AMD’s compatibility matrix.
Validate: rocminfo lists the GPU as a Device Type: GPU agent.

rocminfo fails

Likely Cause: ROCm not fully installed, driver not loaded, or permissions.
Check: Re-run install steps; lsmod | grep amdgpu; groups for render and video.
Fix: Reinstall the rocm package, ensure the driver is loaded, refresh your session for group membership.
Validate: rocminfo returns a GPU agent block without error.

amdgpu not loaded

Likely Cause: Driver failed to build/install, or a kernel mismatch.
Check: lsmod | grep amdgpu; review dmesg for amdgpu messages.
Fix: Install matching linux-headers-$(uname -r) and linux-modules-extra-$(uname -r), reinstall amdgpu-dkms, reboot.
Validate: lsmod | grep amdgpu shows the module loaded.

Permission denied on GPU devices

Likely Cause: User not in render/video, or session not refreshed.
Check: groups; ls -l /dev/kfd /dev/dri.
Fix: sudo usermod -a -G render,video $LOGNAME, then log out and back in.
Validate: groups lists both, and rocminfo runs without root.

Unsupported GPU model

Likely Cause: The GPU is not on the current ROCm compatibility matrix.
Check: Compare the exact model against AMD’s matrix.
Fix: Use a supported GPU, or a ROCm release that lists yours; do not force an unsupported card into production.
Validate: GPU appears in rocminfo and runs a real workload.

PyTorch can’t use GPU

Likely Cause: Wrong PyTorch build (CPU-only or wrong ROCm tag), or ROCm not visible.
Check: In the venv, torch.__version__ (is it a ROCm build?) and torch.cuda.is_available().
Fix: Reinstall with the correct --index-url .../whl/<rocmX.Y> tag from pytorch.org after confirming rocminfo works.
Validate: torch.cuda.is_available() is True and a GPU tensor op runs.

Kernel compatibility problem

Likely Cause: Kernel updated and the DKMS module did not rebuild for it.
Check: uname -r vs the headers installed; dmesg for build errors.
Fix: Install headers/modules for the current kernel, reinstall amdgpu-dkms, reboot.
Validate: lsmod | grep amdgpu and rocminfo both succeed on the new kernel.

ROCm package mismatch

Likely Cause: Mixed component versions, or an outdated amdgpu-install.
Check: Confirm the installer and repo match a single current ROCm release.
Fix: Reregister AMD’s repo with the current installer from AMD’s docs and reinstall consistently.
Validate: rocminfo/rocm-smi run and report the expected version.

User added to group but session not refreshed

Likely Cause: Group changes only apply to new sessions.
Check: groups in the current shell omits render/video.
Fix: Log out and back in, or reboot.
Validate: groups now lists both and GPU tools run without root.

Container sees no AMD GPU

Likely Cause: The container has not been granted the GPU devices/runtime.
Check: Whether the container was started with GPU access at all.
Fix: This is exactly what Part 5 covers — running GPU workloads in Docker. We are not implementing container GPU access here.
Validate: Covered in the containers lesson (coming soon).

Hands-On Lab: Build an AMD ROCm AI Node on Ubuntu 26.04

🧪 Hands-On Lab — We build a dedicated AMD node, ai-amd-node01. Follow along on AMD hardware (or a cloud AMD-GPU instance). Read-only checks are safe; the install and reboot steps change the system — they are flagged. Remember the compatibility caveat: if ROCm does not yet list Ubuntu 26.04 when you check, run this lab on a supported Ubuntu LTS (24.04) instead — the steps are otherwise identical.

Verify Ubuntu 26.04. cat /etc/os-release and uname -r. Confirm the release and note the kernel — the AMD driver builds against it.
Verify the AMD GPU. lspci | grep -Ei 'amd|ati'. Identify the discrete GPU line (ignore integrated/audio functions).
Check current hardware + OS compatibility. Look up your exact GPU model and your Ubuntu version on AMD’s compatibility matrix. Do not proceed until both are confirmed supported by the ROCm release you will install.
Verify amdgpu will drive the card. lspci -k | grep -EA3 'VGA|3D|Display' to see the kernel driver situation before install.
Install supported ROCm packages. Register AMD’s repo with the current amdgpu-install from AMD’s docs, sudo apt update, install linux-headers-$(uname -r) and linux-modules-extra-$(uname -r), then amdgpu-dkms, then rocm.
Configure permissions/groups. sudo usermod -a -G render,video $LOGNAME.
Reboot / refresh session. sudo reboot to load amdgpu and apply group membership cleanly.
Validate ROCm. After reboot, rocminfo (confirm a Device Type: GPU agent) and rocm-smi (confirm live temp/clocks).
Create the venv. python3 -m venv /opt/ai-lab/venv and source /opt/ai-lab/venv/bin/activate.
Install the supported PyTorch ROCm build. pip install torch --index-url https://download.pytorch.org/whl/<rocmX.Y> using the current tag from pytorch.org.
Confirm GPU visibility. In Python, torch.cuda.is_available() returns True and torch.cuda.get_device_name(0) names your AMD GPU.
Run a minimal tensor op. Allocate torch.rand(3, 3, device="cuda") and multiply it; a numeric result means compute ran on the GPU.
Inspect GPU status. Watch rocm-smi while the op runs and confirm GPU% and power move.
Record the config. Append what you built to a system-info file:

{
  echo "host: ai-amd-node01"
  echo "os: $(. /etc/os-release; echo $PRETTY_NAME)"
  echo "kernel: $(uname -r)"
  echo "amdgpu: $(lsmod | grep -c amdgpu) module(s) loaded"
  echo "torch: $(python -c 'import torch; print(torch.__version__)')"
} | tee /opt/ai-lab/ai-amd-node01-info.txt

That block writes a small, greppable record of the node’s key facts to /opt/ai-lab/ai-amd-node01-info.txt — the kind of note that lets you rebuild or debug the box later without guessing.

  +----------------------------------+
  |  ai-amd-node01 status            |
  +----------------------------------+
  |  Ubuntu (supported LTS) .... OK  |
  |  AMD GPU detected .......... OK  |
  |  amdgpu loaded ............. OK  |
  |  ROCm validated ............ OK  |
  |  PyTorch (ROCm) ............ OK  |
  |  GPU compute confirmed ..... OK  |
  +----------------------------------+
        |
        v
   Next: Containers

What’s Next: Containers Above Either GPU Path

So far every workload has run directly on the host against the host’s GPU stack. The next step is containers — packaging AI workloads so they run the same way anywhere. Containers do not replace the driver and runtime; they sit above them.

  Today                    Next
  -----                    ----
  App                      Container
   |                         |
  Host ROCm                GPU runtime
   |                         |
  GPU                      Host driver
                             |
                            GPU

The important idea is that containers layer on top of either GPU ecosystem. Whether a host runs the NVIDIA path or the AMD path, the container story converges:

  Ubuntu 26.04
        |
   +----+----------------------+
   |                           |
  NVIDIA Path              AMD Path
  driver                   amdgpu
   |                          |
  CUDA                      ROCm
   |                          |
  framework                framework
   |                          |
  workload                 workload
   |                          |
   +-----------+--------------+
               |
               v
        converge -> Part 5: Docker

Both paths end at the same place: a GPU-accelerated workload that Part 5 will teach you to containerize. That lesson does not exist yet, so it is not linked here.

What You Learned

What ROCm is — AMD’s open GPU-compute platform — and how its layers (HIP, runtime, math libraries, framework support, rocminfo/rocm-smi) stack up.
How to detect an AMD GPU with lspci | grep -Ei 'amd|ati', why an AMD system shows several PCI devices, and how to read the bound kernel driver.
The compatibility-first rule: verify the exact GPU model and Ubuntu version against AMD’s current matrix — and that ROCm currently targets Ubuntu 24.04/22.04 LTS, with 26.04 not yet listed at time of writing.
The AMD-documented ROCm install flow: AMD’s repo, matching kernel headers, amdgpu-dkms, render/video groups, rocm, and a reboot.
How amdgpu fits as the kernel driver, and how to confirm it with lsmod.
How to validate ROCm with rocminfo and rocm-smi, and what their fields mean.
That PyTorch’s ROCm build exposes the GPU through the same torch.cuda API via HIP — so torch.cuda.is_available() is still the check.
A brand-neutral CUDA-vs-ROCm comparison, and how to choose on compatibility and workload rather than brand loyalty.
A repeatable, layer-by-layer troubleshooting method for AMD nodes.

ai-amd-node01 is the AMD-path mirror of ai-node01 from Part 2: same build-along discipline, same validation habits, a different accelerator ecosystem underneath. If you built both, you now have one node on each path, prepared identically and ready to converge on containers.

Next Lesson

Docker for AI Workloads on Ubuntu 26.04 — Coming Soon. We take a prepared node — NVIDIA or AMD — and learn to run GPU workloads inside containers, so an AI service ships and runs the same way regardless of which GPU ecosystem sits underneath. There is no page for it yet, so it is not linked here.

While you wait, revisit the Ubuntu 26.04 AI series hub for the full roadmap, or build container fundamentals now in the Docker Academy so the next lesson lands easily. If you want to strengthen the operational side, the Linux admins and Kubernetes & Helm guides pair well with the cluster work coming later, and the observability stack previews the monitoring we will add to these nodes. For performance, benchmarking, and observability preparation on either GPU path, the AI Systems Performance Engineering book below is a strong companion; the hands-on GPU book is useful as cross-platform, comparison-oriented background rather than a ROCm guide.