Skip to content
DevOps AI ToolKit
Newsletter
Learning path · 10 lessons · Ubuntu 26.04 LTS

Ubuntu 26.04 AI Infrastructure

Build, operate, monitor, and scale AI infrastructure with Ubuntu Linux.

Learn how to build modern AI infrastructure on Ubuntu 26.04 LTS — from your first GPU-enabled Linux server to Docker, Kubernetes, GPU monitoring, LLM inference, and production AI platforms.

Start the Series → Build Your First AI Server

For DevOps, platform, cloud, and SRE engineers who know basic Linux and want hands-on AI infrastructure skills.

The stack

How AI infrastructure fits together

              AI Applications
                    |
               AI / LLM API
                    |
          +---------+---------+
          |                   |
       Docker            Kubernetes
          |                   |
          +---------+---------+
                    |
              AI Frameworks
                    |
        +-----------+-----------+
        |                       |
       CUDA                   ROCm
        |                       |
    NVIDIA GPU               AMD GPU
        +-----------+-----------+
                    |
             Ubuntu 26.04 LTS

Applications sit on top. Everything beneath them — the runtime, the GPU compute layer, the drivers, the OS — is the infrastructure engineer's job.

Foundations

What is AI infrastructure?

A chat interface is only the visible top layer. Beneath it is a stack of systems that has to be installed, configured, secured, monitored, and kept running — and that stack is what a DevOps or platform engineer owns.

AI Application
      |
API / Model Server
      |
Inference Engine
      |
PyTorch / Runtime
      |
CUDA / ROCm
      |
Linux Kernel + Drivers
      |
GPU Hardware

Each layer can fail independently — a driver mismatch, an out-of-memory GPU, a mis-scheduled pod — and each is something you learn to build and troubleshoot in this series.

What you'll learn

Eight areas, one infrastructure

  • Linux

    Ubuntu administration specifically for AI systems.

  • GPUs

    NVIDIA and AMD accelerator infrastructure — drivers, CUDA, ROCm.

  • Containers

    Containerizing GPU workloads with Docker and the NVIDIA Container Toolkit.

  • Kubernetes

    Scheduling AI workloads across nodes with GPU-aware scheduling.

  • AI Inference

    Operating models as APIs — the practical DevOps side of AI.

  • Monitoring

    Observing GPU, server, container, and inference performance.

  • Automation

    Infrastructure as Code and configuration management for AI nodes.

  • Production Operations

    Security, scalability, availability, and troubleshooting.

The roadmap

The 10-part curriculum

  1. Part 01Available

    Ubuntu 26.04 for AI Infrastructure: Getting Started

    What AI infrastructure is, Ubuntu 26.04’s role, CPU vs GPU, VRAM, CUDA vs ROCm, and training vs inference — the foundation for the whole series.

    BeginnerStart lesson
  2. Part 02Available

    Building Your First Ubuntu 26.04 AI Server

    A hands-on lab: install and prepare an Ubuntu 26.04 server as an AI compute node — hostname, SSH, updates, storage, networking, baseline security, and GPU detection.

    BeginnerStart lesson
  3. Part 03Coming Soon

    NVIDIA GPUs and CUDA on Ubuntu 26.04

    Install NVIDIA drivers with ubuntu-drivers, the CUDA toolkit, and verify GPU compute on Ubuntu 26.04.

    IntermediateIn production
  4. Part 04Coming Soon

    AMD ROCm AI Infrastructure on Ubuntu 26.04

    Set up AMD ROCm on Ubuntu 26.04 and run AI frameworks on AMD accelerators.

    IntermediateIn production
  5. Part 05Coming Soon

    Docker for AI Workloads on Ubuntu 26.04

    Containerize GPU workloads with the NVIDIA Container Toolkit and run reproducible AI containers.

    IntermediateIn production
  6. Part 06Coming Soon

    Running Local LLMs on Ubuntu 26.04

    Serve local large language models on your Ubuntu GPU server and reason about VRAM, quantization, and tokens per second.

    IntermediateIn production
  7. Part 07Coming Soon

    Kubernetes for AI Workloads on Ubuntu 26.04

    Schedule GPU workloads across nodes with Kubernetes device plugins and GPU-aware scheduling.

    AdvancedIn production
  8. Part 08Coming Soon

    Monitoring Ubuntu AI Infrastructure

    Observe GPU, server, container, and inference performance with Prometheus and Grafana.

    IntermediateIn production
  9. Part 09Coming Soon

    Building an AI Inference Server on Ubuntu 26.04

    Operate models as APIs — an inference server with health checks, batching, and observability.

    AdvancedIn production
  10. Part 10Coming Soon

    Building a Production AI Platform on Ubuntu 26.04

    Bring it together: a multi-node, observable, automated, secure GPU-powered AI platform.

    AdvancedIn production
Progression

How the series builds

  1. 01. Getting Started
  2. 02. Build an AI Server
  3. 03. NVIDIA CUDA soon
  4. 04. AMD ROCm soon
  5. 05. Docker AI soon
  6. 06. Local LLMs soon
  7. 07. Kubernetes AI soon
  8. 08. Monitoring soon
  9. 09. Inference Server soon
  10. 10. Production Platform soon
Where this leads

What you'll eventually build

            Users
              |
            HTTPS
              |
         API Gateway
              |
      +-------+-------+
      |               |
  ai-node01       ai-node02
   Ubuntu          Ubuntu
      |               |
     GPU             GPU
      +-------+-------+
              |
          Kubernetes
              |
      +-------+-------+
      |               |
   Models         Storage
              |
          Monitoring
              |
     Prometheus + Grafana

You build toward this one node at a time — starting with a single Ubuntu server in Part 2.

Optional resources

Recommended reading & hardware

Optional references. The tutorials are the primary learning path.

Recommended Hardware

The right GPU depends on your model, VRAM needs, workload, power, cooling, budget, and software compatibility — there is no single “best.” Cloud GPU instances are a valid alternative to buying hardware.

Affiliate Disclosure: Some links on this page are affiliate links. If you purchase through one of these links, DevOps AI Toolkit may earn a commission at no additional cost to you. See our affiliate disclosure.

FAQ

Ubuntu AI infrastructure — common questions

What is AI infrastructure?

AI infrastructure is everything beneath an AI application — the model server, inference engine, runtime (PyTorch), GPU compute layer (CUDA or ROCm), Linux kernel and drivers, and the GPU hardware itself. AI infrastructure engineers build and operate those layers so models can run reliably.

Do I need an expensive GPU to follow this series?

No. Parts 1–2 need only an Ubuntu machine — no GPU required to learn hardware detection and server preparation. Later GPU lessons work on modest consumer cards or a cloud GPU instance. Hardware requirements depend entirely on the model and workload.

Is this an AI or machine-learning course?

No. This is an infrastructure course. We focus on Ubuntu administration, GPUs, drivers, Docker, Kubernetes, networking, storage, monitoring, automation, and running models as services — not model training, data science, or prompt engineering.

Which Ubuntu version does the series target?

Ubuntu 26.04 LTS ("Resolute Raccoon"), the April 2026 long-term-support release. GPU tooling changes between Ubuntu versions, so the series teaches the 26.04 method specifically rather than reusing older 22.04/24.04 instructions.

NVIDIA or AMD — which does the series cover?

Both. Part 3 covers NVIDIA drivers and CUDA; Part 4 covers AMD ROCm. Not every AI workload requires NVIDIA, and ROCm is not identical to CUDA — the series explains the trade-offs rather than assuming one vendor.

What background do I need?

Comfort with basic Linux: SSH, sudo, apt, systemd, IP addresses, filesystems, and basic Bash. You do not need prior GPU, CUDA, ROCm, or AI experience — those concepts are introduced and explained as they come up.

What will I have built by the end?

A progressively built lab — starting from a single prepared Ubuntu node (ai-node01) and growing toward a multi-node, GPU-powered, observable, automated, production-style AI inference platform.

Start with a single Ubuntu server

Begin with the concepts, then build and prepare your first Ubuntu 26.04 AI node — and grow it into production AI infrastructure across the series.

Start Part 1: Getting Started →