Building Your First Ubuntu 26.04 AI Server

This is Part 2 of the Ubuntu 26.04 AI Infrastructure series. In Part 1 we talked about why GPUs matter for AI and how the software stack layers together. Now we get our hands dirty: we build the physical (or virtual) machine that everything else in this series runs on. By the end you will have a clean, patched, network-connected Ubuntu 26.04 server that can see its GPU at the hardware level and is ready for drivers and CUDA in Part 3.

🧪 Hands-On Lab — This is a build lab. Follow along on real hardware, a spare workstation, a virtual machine, or a cloud GPU instance. Every command here is safe to run and only reads system state until we reach the package-install and firewall steps, which we flag clearly.

What You’ll Learn

• Hardware planning — how CPU, system RAM, GPU, VRAM (the GPU’s own onboard memory), storage, and networking each affect an AI server, and how to size them to a workload instead of to hype.
• Installing Ubuntu 26.04 Server — the LTS “Resolute Raccoon” release, disk layout, and filesystem considerations for a compute node.
• Baseline setup — setting a hostname, enabling SSH, correct time sync, applying updates, and installing the packages you will actually use.
• Storage and system checks — verifying disks, free space, CPU, and RAM with standard Linux tools, and reading their output.
• PCI/GPU detection — confirming Linux physically sees the GPU with lspci, before any driver is installed.
• System validation — a repeatable checklist that proves the node is healthy.
• Baseline security — a minimal firewall posture that will not lock you out of a remote box.
• Preparing for CUDA/ROCm — leaving the machine in exactly the state Parts 3 (NVIDIA CUDA) and 4 (AMD ROCm) expect.

We are deliberately not installing GPU drivers yet. Detecting hardware and installing drivers are two different jobs, and mixing them up is one of the most common ways beginners get stuck.

Hardware Architecture of a GPU Server

Before you buy or provision anything, it helps to picture how data actually moves through an AI server. Think of the GPU as a very fast worker that is useless unless the rest of the machine can keep feeding it.

   +-----------+        +-----------+
   |    CPU    |<------>|  System   |
   | (orchestr)|  RAM   |   RAM     |
   +-----+-----+        +-----------+
         |
         |  PCIe lanes (gen & width)
         v
   +-----------+        +-----------+
   |   GPU(s)  |<------>|   VRAM    |
   | (compute) | onboard| (models)  |
   +-----------+        +-----------+

   +-----------+   read/write   +-----------+
   |   NVMe    |<-------------->|  Model &  |
   |  storage  |                |  dataset  |
   +-----------+                |  storage  |
                                +-----------+

   +-----------+   inference / cluster traffic
   |  Network  |<----> API clients, other nodes
   +-----------+

Here is why each piece matters, in plain terms:

• PCIe generation & lanes. PCIe (PCI Express) is the bus that connects the GPU to the CPU. Its generation (Gen4, Gen5) sets per-lane speed, and the lane width (x8, x16) sets how many lanes the slot gives the card. A GPU in a slow or narrow slot still works, but moving models and batches in and out of the card is slower. For a single-GPU learning box this rarely bottlenecks you; for multi-GPU or heavy data movement it can.
• System RAM. This is the CPU’s working memory, separate from VRAM. You load and preprocess data here before it reaches the GPU. Too little RAM causes swapping and stalls. A common rule of thumb is to have at least as much system RAM as GPU VRAM, and more if you preprocess large datasets.
• VRAM (video RAM). The GPU’s dedicated onboard memory. This is usually the single most important number for AI work, because the model’s weights must fit in VRAM to run efficiently. A model that does not fit either spills to slower memory or simply will not load. VRAM sizing is workload-driven, which we return to in the tiers below.
• NVMe storage. NVMe is fast flash storage on the PCIe bus. AI model files and datasets are large, and you read them repeatedly, so fast local storage shortens load times noticeably versus older SATA drives.
• PSU (power supply). GPUs draw significant, spiky power. An undersized or low-quality PSU causes instability that looks like random crashes. Match the PSU headroom to the card(s) you install.
• Cooling. Sustained AI workloads keep the GPU busy for minutes or hours, not seconds. Without adequate airflow the card thermally throttles (slows itself to avoid overheating), so your “slow” server may just be hot.

🛠️ DevOps Tip — Treat an AI node as infrastructure, not as a pet gaming rig. From day one, plan for monitoring, patching, recovery, and reproducibility. If you cannot rebuild this machine from notes or a script, you do not have a server — you have a liability. We wire in observability later in the series via the observability stack.

Three AI Server Tiers

There is no single “AI server.” What you need depends entirely on the models you run and the workload you put on them. A 7-billion-parameter chat model quantized to 4-bit is a very different requirement than a large multi-model inference service. The tiers below are reference shapes, not shopping lists, and there are no prices here on purpose.

Tier 1 — Learning / Homelab

The goal is to learn the stack, not to break records. A modest desktop or workstation with a consumer GPU is plenty. You will run Linux, Docker, small local models, and general experimentation. This tier is where most people in this series should start. You can learn every concept in Parts 3–10 on hardware in this class, and a smaller GPU simply means you run smaller models or wait a little longer — nothing here is gated behind expensive gear.

Tier 2 — Serious AI Workstation

For heavier GPU inference (running a trained model to produce outputs), active development, larger models, and performance testing. Here you care more about VRAM capacity, cooling, and a PSU with headroom, because the GPU stays busy longer and the models you load are bigger. This is a productive single-machine setup for a developer or a small team.

Tier 3 — Professional AI Infrastructure

For advanced inference, multiple concurrent workloads, and larger GPU resources — the kind of node that sits in a rack and serves many clients or participates in a cluster. Networking, storage throughput, and multi-GPU PCIe topology start to matter as much as the GPU itself. You would design this tier around a specific service-level goal, not around a spec sheet.

The layout renders grouped GPU recommendations below this lesson — see the recommended GPUs below for concrete cards mapped to each of these shapes. Match the group to your workload, not to the biggest number.

🤖 AI Infrastructure Tip — You do not have to own any of this. Cloud GPU instances are a completely valid alternative to buying hardware, especially while learning or for bursty work. You rent a GPU by the hour, follow the exact same Ubuntu 26.04 steps in this lesson, and shut it down when you are done. Buy hardware when your steady-state usage makes renting more expensive than owning — not before.

Installing Ubuntu Server 26.04

We target Ubuntu 26.04 LTS “Resolute Raccoon” (released April 2026). LTS (“Long-Term Support”) releases get years of security updates, which is exactly what you want for a machine you will build a stack on.

The install itself is standard, so rather than screenshot every wizard screen, here is the workflow and the decisions that actually matter for an AI node:

1. Get the ISO and boot the installer. Download the Ubuntu Server 26.04 ISO, write it to a USB drive, and boot from it. Choose Server, not Desktop — an AI node does not need a graphical desktop, and skipping it frees RAM and reduces the attack surface.
2. Disk layout & filesystem. Give the OS a healthy root partition and, ideally, put large model and dataset storage on fast NVMe. ext4 is a safe, well-understood default filesystem. If you expect to grow storage later, LVM (Logical Volume Manager) makes resizing easier. Do not over-engineer this on a first build — a single ext4 root on NVMe is fine to learn on.
3. Enable the OpenSSH server when the installer offers it. This lets you manage the box remotely, which is how real servers are run. If you miss it, you can install it later with sudo apt install openssh-server.
4. Create your user and finish the install. Reboot into the new system.

Once you are logged in, confirm you are on the right release:

cat /etc/os-release
uname -r

cat /etc/os-release prints the distribution name and version — you want to see Ubuntu 26.04. uname -r prints the running kernel version. You do not need to memorize the kernel number, but note it: GPU drivers in Part 3 are built against the kernel, so if the kernel changes later, drivers may need a rebuild.

Set the hostname

A server should have a deliberate name, not the random one the installer picked. We use hostnamectl, systemd’s tool for host identity:

hostnamectl

Run with no arguments, hostnamectl shows the current hostname, machine ID, OS, and kernel. We will set the actual name in the next section, where we establish our naming convention.

Time sync

Correct time is not optional on a server. Certificates, package metadata, logs, and cluster coordination all assume clocks are close to correct:

timedatectl

This shows the local and UTC time and, crucially, whether NTP (Network Time Protocol, which keeps the clock synced automatically) is active. Look for System clock synchronized: yes and NTP service: active. If NTP is off, enable it with sudo timedatectl set-ntp true.

Networking and DNS

Confirm the machine has an address and a route to the internet:

ip addr
ip route

ip addr lists network interfaces and their IP addresses — find your primary interface (often something like enp3s0) and confirm it has an inet address. ip route shows the routing table; you want a default via line pointing at your gateway, which is what sends traffic to the wider network. Then check name resolution:

resolvectl status

resolvectl status shows which DNS servers each interface is using. DNS (Domain Name System) turns names like archive.ubuntu.com into IP addresses; if it is misconfigured, apt will hang on “connecting” even though ip route looks fine. Networking fundamentals are worth a deeper read — the Kali Linux networking lesson covers interfaces, routing, and DNS in more depth if any of this is unfamiliar.

See what is already listening on the network:

ss -tulpn

ss -tulpn lists open TCP and UDP sockets that are listening, with the numeric ports and owning processes. On a fresh server you should see little more than SSH on port 22. This is your baseline: anything new that appears here later, you should be able to explain.

Updates

Bring the system fully current before you build anything on it:

sudo apt update && sudo apt upgrade

sudo apt update refreshes the list of available packages (it downloads metadata, it does not change installed software). sudo apt upgrade then installs newer versions of packages you already have. Running them together with && means upgrade only runs if update succeeded. On a fresh install this may pull a lot of security fixes — let it finish, and reboot if it tells you a new kernel was installed.

Useful packages

Install a small, honest set of tools you will actually use on a compute node. These are all real Ubuntu packages:

sudo apt install build-essential git curl htop pciutils net-tools

• build-essential — compilers and build tools; some later steps and many AI tools compile components locally.
• git — version control, for pulling model tooling and your own scripts.
• curl — fetch files and probe HTTP endpoints from the shell.
• htop — an interactive process and resource viewer; far friendlier than raw top for watching CPU and RAM under load.
• pciutils — provides lspci, which we use below to detect the GPU.
• net-tools — classic networking utilities; handy for quick checks and compatibility with older runbooks.

Baseline firewall

Ubuntu ships UFW (Uncomplicated Firewall), a friendly front-end to the kernel’s packet filter. The order of operations here matters enormously.

sudo ufw allow OpenSSH
sudo ufw enable

The first line creates a rule permitting SSH before the firewall turns on. The second line activates the firewall. Doing them in this order keeps your management connection alive.

⚠️ Warning — Never run sudo ufw enable on a remote machine without first running sudo ufw allow OpenSSH. UFW’s default policy denies incoming traffic, so enabling it without an SSH allow rule will drop your current SSH session and lock you out of the box — with no console access, that can mean a trip to the data center or a full reprovision. Allow SSH first, every time.

Naming Your Node: ai-node01

Random hostnames age badly. We adopt a simple, boring, scalable convention: AI compute nodes are named ai-node01, ai-node02, and so on. Boring is good — it is predictable, it sorts correctly, and it reads clearly in logs, dashboards, and ssh commands.

Set it now:

sudo hostnamectl set-hostname ai-node01

hostnamectl set-hostname writes the new name to /etc/hostname and applies it to the running system. Open a fresh shell (or log out and back in) to see your prompt update. Run hostnamectl again to confirm the “Static hostname” line now reads ai-node01.

Here is the machine we are building:

  ai-node01
  |
  +-- OS ...... Ubuntu 26.04 LTS
  +-- CPU ..... orchestration / preprocessing
  +-- RAM ..... system working memory
  +-- NVMe .... models & datasets
  +-- GPU ..... AI compute (detected next)

We start at 01 for a reason. When Parts later in the series introduce clustering, having ai-node02 and ai-node03 join ai-node01 makes the Kubernetes lessons far easier to follow — the names already tell you these machines belong together and play the same role.

GPU Detection at the PCI Level

This is the moment of truth for the hardware: does the operating system even see the GPU on the PCI bus? This is a pure hardware detection check. It does not require, and does not care about, any GPU driver. A card can be perfectly visible here and still have no driver — that is normal at this stage.

lspci | grep -i -E 'vga|3d|nvidia|amd'

lspci lists every device on the PCI bus. Piping it through grep -i -E 'vga|3d|nvidia|amd' filters, case-insensitively, for lines that look like a display or 3D controller or that mention NVIDIA or AMD. On a working GPU server you should see a line naming your card, for example a VGA compatible controller or 3D controller entry with the vendor and model. If you see your GPU listed, Linux sees the hardware — that is the win for this lesson.

If nothing GPU-like appears, the card is not being detected, and no driver will fix that until the hardware is visible first.

🔍 Troubleshooting — GPU not visible in lspci

• Problem: lspci | grep -i -E 'vga|3d|nvidia|amd' returns nothing, or omits your discrete GPU.
• Likely Cause: The card is not seated/powered, is disabled in firmware, is in a dead slot, or (in a VM) has not been passed through to the guest.
• Check: Work down the chain below, top to bottom.
• Fix: Reseat and power the card; enable it in BIOS/UEFI; try a known- good PCIe slot; for a VM, attach the GPU via passthrough/vGPU on the host.
• Validate: Re-run the lspci command and confirm the card now appears.

 GPU not visible in lspci?
   |
   v
 [1] Physical install ... seated? power connected?
   |
   v
 [2] BIOS/UEFI ......... PCIe/GPU enabled in firmware?
   |
   v
 [3] PCIe slot ......... try another known-good slot
   |
   v
 [4] Virtualization .... passthrough/vGPU to the VM?
   |
   v
 [5] lspci ............. re-run; card should appear

Keep two ideas firmly separated:

• Hardware detection (this lesson): lspci proves the GPU exists on the bus. That is all we are doing today.
• Driver installation (Part 3): Making the GPU usable for compute needs a driver. On Ubuntu that is handled with the ubuntu-drivers tool, and we will walk through it carefully in Part 3. We are deliberately not writing that procedure here — installing drivers before you have confirmed detection just adds a second variable when something goes wrong.

Physical Server vs Virtual Machine

You can run ai-node01 on bare metal or as a virtual machine, and the choice mostly comes down to how the GPU is attached.

• Bare metal — Ubuntu runs directly on the hardware and has direct access to the GPU. This is the simplest path and the fewest moving parts, which is why it is a great choice for a first build. What lspci shows is exactly the physical card.
• Hypervisor (virtualized) — Ubuntu runs as a guest on top of a virtualization layer. For the guest to use a GPU, the host must hand it over, either by GPU passthrough (giving the whole physical card to one VM) or vGPU (slicing one physical GPU into virtual pieces shared across VMs). Only then will the card appear in the guest’s lspci.

Conceptually you will meet several of these environments: KVM, the Linux kernel’s built-in hypervisor; OpenStack, which orchestrates VMs (including GPU flavors) at cloud scale; VMware, common in enterprises; and public-cloud GPU instances, which are simply someone else’s hypervisor renting you a GPU-attached VM by the hour.

Full PCI passthrough — binding the GPU away from the host, configuring IOMMU, and attaching it to a guest — is a genuinely advanced topic with real footguns. We are flagging it as future advanced content rather than walking through it here. For this series, bare metal or a cloud GPU instance is the smoother road, and both leave you in the same place: a GPU visible in lspci.

Validation Checklist

Before we call ai-node01 “ready,” prove it. Run the checks from this lesson and tick each box only when you have seen the expected output — not when you assume it is fine.

✅ Validation — A node passes when every item below is confirmed on the actual machine. If any item fails, fix it now; later lessons assume this baseline.

• Ubuntu 26.04 installed (cat /etc/os-release shows 26.04)
• System fully patched (sudo apt update && sudo apt upgrade clean)
• Networking operational (ip addr has an address, ip route has a default route)
• SSH operational (you can reach the box over SSH; port 22 shows in ss -tulpn)
• DNS operational (resolvectl status lists working DNS servers)
• Hostname configured (hostnamectl shows ai-node01)
• Storage verified (lsblk shows expected disks and partitions)
• CPU identified (lscpu shows the expected cores/model)
• RAM verified (free -h shows the expected total memory)
• GPU visible at PCI level (lspci | grep -i -E 'vga|3d|nvidia|amd' lists the card)
• Time sync working (timedatectl shows NTP active and clock synchronized)
• Firewall reviewed (sudo ufw status shows SSH allowed)
• Sufficient free disk (df -h shows healthy free space on / and model storage)
• Ready for CUDA or ROCm (all of the above green)

Two commands appear here that we should read carefully. lsblk lists block devices in a tree — disks and their partitions — so you can confirm your NVMe drive is present and mounted where you expect. df -h shows filesystem usage in human-readable units; watch the “Use%” column, because a full root filesystem breaks package installs and driver builds in confusing ways. lscpu summarizes the CPU (cores, model, virtualization support) and free -h shows total and available memory — together they confirm the machine is what you think it is.

Your Ubuntu AI Server Is Ready

If every box above is ticked, the foundation is done:

  +-------------------------------+
  |  ai-node01 status             |
  +-------------------------------+
  |  Ubuntu 26.04 ............ OK  |
  |  Networking .............. OK  |
  |  Storage ................. OK  |
  |  GPU Detected ............ OK  |
  |  System Updated .......... OK  |
  +-------------------------------+
        |
        v
   Next: NVIDIA CUDA

You have a clean, patched, named, network- and GPU-aware Ubuntu 26.04 server. That is a real milestone: everything else in this series builds on top of exactly this state.

Your AI Lab So Far

Here is the bigger picture we are assembling across the series. Part 2 completes the first block; the rest are coming in later lessons.

  ai-node01
  ------------------------------------
  Ubuntu 26.04 ............... [done]
  Networking ................. [done]
  Storage .................... [done]
  GPU detected (PCI) ......... [done]
  ------------------------------------
  NVIDIA CUDA ................ [next]
  Docker ..................... [upcoming]
  Kubernetes ................. [upcoming]
  Monitoring ................. [upcoming]

Each upcoming block turns this single prepared machine into a real AI platform: GPU compute with CUDA, containerized workloads with Docker, orchestration with Kubernetes, and visibility with monitoring.

What You Learned

• How the parts of a GPU server fit together — CPU, RAM, PCIe, GPU, VRAM, NVMe storage, PSU, and cooling — and why each one can bottleneck AI work.
• That AI server requirements follow the workload, not a spec sheet, and that a modest Tier 1 box (or a rented cloud GPU) is enough to learn everything in this series.
• How to install and baseline Ubuntu 26.04 Server: hostname, SSH, time sync, updates, useful packages, and a firewall that will not lock you out.
• How to read the output of hostnamectl, timedatectl, ip addr, ip route, resolvectl status, ss -tulpn, lsblk, df -h, lscpu, and free -h — not just run them.
• The naming convention ai-node01, and why it pays off at cluster time.
• How to confirm the GPU is visible at the PCI level with lspci, and how to keep hardware detection mentally separate from driver install (Part 3, via ubuntu-drivers).
• Bare metal versus virtualized GPU access, and where passthrough fits as future advanced content.
• A repeatable validation checklist that proves the node is ready for CUDA or ROCm.

Next Lesson

NVIDIA GPUs and CUDA on Ubuntu 26.04 — Coming Soon. We take the detected GPU from this lesson and make it usable for compute: installing drivers the Ubuntu way, understanding what CUDA actually is, and verifying the GPU is doing real work. There is no page for it yet, so it is not linked here.

While you wait, revisit the Ubuntu 26.04 AI series hub to see the full roadmap, or keep building adjacent skills: the Docker Academy prepares you for the containerization lessons ahead, and the observability stack previews the monitoring we will add to ai-node01. If you want to shore up Linux fundamentals first, the Linux admins guides are a solid companion to this build.