Skip to content
DevOps AI ToolKit
Complete 10-part series · Ubuntu 26.04 LTS

Ubuntu 26.04 AI Infrastructure

All 10 lessons available — the full series is complete

Build, operate, monitor, and scale AI infrastructure with Ubuntu Linux.

A complete, hands-on path: progress from a single Ubuntu 26.04 AI server to GPU acceleration, Docker, local LLMs, Kubernetes, monitoring, secure inference APIs, and a production-style AI platform.

Start with Part 1 → Build Your First AI Server

For DevOps, platform, cloud, and SRE engineers who know basic Linux and want hands-on AI infrastructure skills. Already comfortable? Jump to the production capstone →

The stack

How AI infrastructure fits together

              AI Applications
                    |
               AI / LLM API
                    |
          +---------+---------+
          |                   |
       Docker            Kubernetes
          |                   |
          +---------+---------+
                    |
              AI Frameworks
                    |
        +-----------+-----------+
        |                       |
       CUDA                   ROCm
        |                       |
    NVIDIA GPU               AMD GPU
        +-----------+-----------+
                    |
             Ubuntu 26.04 LTS

Applications sit on top. Everything beneath them — the runtime, the GPU compute layer, the drivers, the OS — is the infrastructure engineer's job.

Foundations

What is AI infrastructure?

A chat interface is only the visible top layer. Beneath it is a stack of systems that has to be installed, configured, secured, monitored, and kept running — and that stack is what a DevOps or platform engineer owns.

AI Application
      |
API / Model Server
      |
Inference Engine
      |
PyTorch / Runtime
      |
CUDA / ROCm
      |
Linux Kernel + Drivers
      |
GPU Hardware

Each layer can fail independently — a driver mismatch, an out-of-memory GPU, a mis-scheduled pod — and each is something you learn to build and troubleshoot in this series.

What you'll learn

Eight areas, one infrastructure

  • Linux

    Ubuntu administration specifically for AI systems.

  • GPUs

    NVIDIA and AMD accelerator infrastructure — drivers, CUDA, ROCm.

  • Containers

    Containerizing GPU workloads with Docker and the NVIDIA Container Toolkit.

  • Kubernetes

    Scheduling AI workloads across nodes with GPU-aware scheduling.

  • AI Inference

    Operating models as APIs — the practical DevOps side of AI.

  • Monitoring

    Observing GPU, server, container, and inference performance.

  • Automation

    Infrastructure as Code and configuration management for AI nodes.

  • Production Operations

    Security, scalability, availability, and troubleshooting.

The roadmap

The 10-part curriculum

  1. Part 01Available

    Ubuntu 26.04 for AI Infrastructure: Getting Started

    What AI infrastructure is, Ubuntu 26.04’s role, CPU vs GPU, VRAM, CUDA vs ROCm, and training vs inference — the foundation for the whole series.

    BeginnerStart lesson
  2. Part 02Available

    Building Your First Ubuntu 26.04 AI Server

    A hands-on lab: install and prepare an Ubuntu 26.04 server as an AI compute node — hostname, SSH, updates, storage, networking, baseline security, and GPU detection.

    BeginnerStart lesson
  3. Part 03Available

    NVIDIA GPUs and CUDA on Ubuntu 26.04

    Configure NVIDIA drivers and CUDA on Ubuntu 26.04, verify GPU acceleration, run your first GPU workload, and troubleshoot the NVIDIA software stack.

    IntermediateStart lesson
  4. Part 04Available

    AMD ROCm AI Infrastructure on Ubuntu 26.04

    Explore AMD’s ROCm stack on Ubuntu 26.04, validate supported GPUs, configure ROCm, and run accelerated AI workloads with PyTorch.

    IntermediateStart lesson
  5. Part 05Available

    Docker for AI Workloads on Ubuntu 26.04

    Containerize AI workloads with Docker on Ubuntu 26.04, give containers access to NVIDIA or AMD GPUs, persist model data, and troubleshoot GPU-enabled containers.

    IntermediateStart lesson
  6. Part 06Available

    Running Local LLMs on Ubuntu 26.04

    Deploy local LLM infrastructure on Ubuntu 26.04, manage models and VRAM, expose inference APIs, and build a GPU-powered Docker-based AI lab.

    IntermediateStart lesson
  7. Part 07Available

    Kubernetes for AI Workloads on Ubuntu 26.04

    Build GPU-aware Kubernetes infrastructure on Ubuntu 26.04, schedule NVIDIA or AMD AI workloads, persist models, configure services, and troubleshoot GPU Pods.

    AdvancedStart lesson
  8. Part 08Available

    Monitoring Ubuntu AI Infrastructure

    Monitor Ubuntu AI infrastructure with Prometheus and Grafana — Linux servers, Kubernetes, GPU utilization, VRAM, temperature, inference latency, and alerts.

    IntermediateStart lesson
  9. Part 09Available

    Building an AI Inference Server on Ubuntu 26.04

    Turn a GPU LLM workload into a secure inference API on Ubuntu 26.04 — vLLM on Kubernetes, TLS via the Gateway API, authentication, rate limiting, health probes, and benchmarking.

    AdvancedStart lesson
  10. Part 10Available

    Build a Production AI Platform on Ubuntu 26.04

    The capstone: automate, secure, monitor, and recover a production-style GPU AI platform on Ubuntu 26.04 — IaC, Ansible, CI/CD, GitOps, secrets, backups, disaster recovery, and capacity planning.

    AdvancedStart lesson
Progression

How the series builds

  1. 01. Getting Started
  2. 02. Build an AI Server
  3. 03. NVIDIA CUDA
  4. 04. AMD ROCm
  5. 05. Docker AI
  6. 06. Local LLMs
  7. 07. Kubernetes AI
  8. 08. Monitoring
  9. 09. Inference Server
  10. 10. Production Platform
Where this leads

What you'll eventually build

            Users
              |
            HTTPS
              |
         API Gateway
              |
      +-------+-------+
      |               |
  ai-node01       ai-node02
   Ubuntu          Ubuntu
      |               |
     GPU             GPU
      +-------+-------+
              |
          Kubernetes
              |
      +-------+-------+
      |               |
   Models         Storage
              |
          Monitoring
              |
     Prometheus + Grafana

You build toward this one node at a time — starting with a single Ubuntu server in Part 2.

Optional resources

Recommended reading & hardware

Optional references. The tutorials are the primary learning path.

Recommended Hardware

The right GPU depends on your model, VRAM needs, workload, power, cooling, budget, and software compatibility — there is no single “best.” Cloud GPU instances are a valid alternative to buying hardware.

Affiliate Disclosure: Some links on this page are affiliate links. If you purchase through one of these links, DevOps AI Toolkit may earn a commission at no additional cost to you. See our affiliate disclosure.

FAQ

Ubuntu AI infrastructure — common questions

What is AI infrastructure?

AI infrastructure is everything beneath an AI application — the model server, inference engine, runtime (PyTorch), GPU compute layer (CUDA or ROCm), Linux kernel and drivers, and the GPU hardware itself. AI infrastructure engineers build and operate those layers so models can run reliably.

Do I need an expensive GPU to follow this series?

No. Parts 1–2 need only an Ubuntu machine — no GPU required to learn hardware detection and server preparation. Later GPU lessons work on modest consumer cards or a cloud GPU instance. Hardware requirements depend entirely on the model and workload.

Is this an AI or machine-learning course?

No. This is an infrastructure course. We focus on Ubuntu administration, GPUs, drivers, Docker, Kubernetes, networking, storage, monitoring, automation, and running models as services — not model training, data science, or prompt engineering.

Which Ubuntu version does the series target?

Ubuntu 26.04 LTS ("Resolute Raccoon"), the April 2026 long-term-support release. GPU tooling changes between Ubuntu versions, so the series teaches the 26.04 method specifically rather than reusing older 22.04/24.04 instructions.

NVIDIA or AMD — which does the series cover?

Both. Part 3 covers NVIDIA drivers and CUDA; Part 4 covers AMD ROCm. Not every AI workload requires NVIDIA, and ROCm is not identical to CUDA — the series explains the trade-offs rather than assuming one vendor.

What background do I need?

Comfort with basic Linux: SSH, sudo, apt, systemd, IP addresses, filesystems, and basic Bash. You do not need prior GPU, CUDA, ROCm, or AI experience — those concepts are introduced and explained as they come up.

What will I have built by the end?

A progressively built lab — starting from a single prepared Ubuntu node (ai-node01) and growing toward a multi-node, GPU-powered, observable, automated, production-style AI inference platform.

Start with a single Ubuntu server

Begin with the concepts, then build and prepare your first Ubuntu 26.04 AI node — and grow it into production AI infrastructure across the series.

Start Part 1: Getting Started →