📁 Python pathlib for Filesystem Automation the Modern Way

For years my Python file-handling code was a swamp of os.path.join, os.path.dirname, string concatenation with /, and the occasional Windows-versus-Linux separator bug that only showed up in CI. Then I committed to pathlib, and a whole category of bugs just disappeared. Paths became objects with methods instead of strings I had to be careful with.

If you write automation that touches the filesystem — log rotation, backup staging, config templating, cleanup jobs — this is the standard-library module worth adopting fully. Let me walk through the patterns I use every day.

Path objects instead of strings

The core idea: Path represents a filesystem path as an object, and the / operator joins paths the way it reads in your head.

from pathlib import Path

base = Path("/var/log/myapp")
today = base / "2026" / "06" / "app.log"
print(today)            # /var/log/myapp/2026/06/app.log
print(today.parent)     # /var/log/myapp/2026/06
print(today.name)       # app.log
print(today.suffix)     # .log
print(today.stem)       # app

No more os.path.join(base, "2026", "06", "app.log"). The / operator handles separators correctly on every platform, and the attributes (parent, name, suffix, stem) give you the parts without string slicing.

Reading and writing without open()

For small files, pathlib gives you one-liners that handle opening and closing for you:

config = Path("/etc/myapp/config.yaml")
text = config.read_text(encoding="utf-8")

out = Path("/tmp/report.txt")
out.write_text("done\n", encoding="utf-8")

# Binary works too
blob = Path("artifact.bin").read_bytes()

For large files you still want a streaming with open(...), but for config files and small reports these one-liners remove a lot of boilerplate.

Pro Tip: Always pass encoding="utf-8" explicitly. The platform default encoding has bitten me on servers with an unexpected locale, where the same script that worked on my laptop mangled UTF-8 in production.

Finding files with glob and rglob

This is where pathlib shines for ops work. Need every log file older than a pattern? Every YAML under a tree?

base = Path("/var/log/myapp")

# Non-recursive: just this directory
for log in base.glob("*.log"):
    print(log)

# Recursive: the whole tree
for cfg in base.rglob("*.yaml"):
    print(cfg)

rglob("*.yaml") walks the entire subtree. Combine it with stat() to build a real cleanup job:

import time

cutoff = time.time() - 30 * 86400  # 30 days ago
for log in base.rglob("*.log"):
    if log.stat().st_mtime < cutoff:
        print(f"Would delete {log}")
        # log.unlink()

I always run cleanup jobs in dry-run mode first, printing what would be deleted before I uncomment the unlink().

Creating directories safely

mkdir with the right flags is idempotent, which is exactly what automation needs:

staging = Path("/tmp/backup-staging")
staging.mkdir(parents=True, exist_ok=True)

parents=True creates intermediate directories, and exist_ok=True makes a second run a no-op instead of an error. No more wrapping os.makedirs in a try/except.

Checking existence and type

The query methods read like plain English and avoid the classic race conditions of checking then acting:

p = Path("/etc/myapp/config.yaml")

if p.exists():
    ...
if p.is_file():
    ...
if p.is_dir():
    ...
if p.is_symlink():
    ...

When you need an absolute, resolved path (following symlinks and collapsing ..), use resolve():

real = Path("./config/../config.yaml").resolve()

This is also a useful security check: resolve a user-supplied path and verify it stays inside an expected directory before you read or write it.

Copying, moving, and removing

pathlib covers most filesystem mutations directly, and falls back to shutil for recursive operations:

from pathlib import Path
import shutil

src = Path("/etc/myapp/config.yaml")

# Rename / move (atomic within a filesystem)
src.rename("/etc/myapp/config.yaml.bak")

# Copy a single file (shutil for the bytes)
shutil.copy2("/etc/myapp/config.yaml", "/backup/config.yaml")

# Remove a single file
Path("/tmp/scratch.txt").unlink(missing_ok=True)

# Remove an empty directory
Path("/tmp/empty").rmdir()

# Remove a whole tree (shutil, no pathlib equivalent)
shutil.rmtree("/tmp/staging", ignore_errors=True)

unlink(missing_ok=True) is the idempotent delete I want in automation: it does not raise if the file is already gone, so re-running the script is safe. rename is atomic within a single filesystem, which makes it the right tool for the write-temp-then-swap pattern that avoids leaving readers with a half-written file.

A word of caution on shutil.rmtree: it deletes recursively with no confirmation. I guard every call with an explicit, resolved path check so a bad variable cannot point it somewhere catastrophic:

target = Path("/tmp/staging").resolve()
if not str(target).startswith("/tmp/"):
    raise SystemExit(f"refusing to rmtree outside /tmp: {target}")
shutil.rmtree(target, ignore_errors=True)

Walking metadata with stat

Beyond modification time, stat() exposes size, permissions, and ownership for reporting and policy checks:

from pathlib import Path

for f in Path("/var/log/myapp").rglob("*.log"):
    st = f.stat()
    size_mb = st.st_size / (1024 * 1024)
    print(f"{f.name}: {size_mb:.1f} MB, mode {oct(st.st_mode)[-3:]}")

This is the backbone of disk-usage reports and permission audits. Pair it with oct(st.st_mode & 0o777) to extract the familiar three-digit mode, or use f.chmod(0o600) to tighten a file that should not be world-readable — useful right after writing anything that might hold sensitive config.

Building config files from templates

A common automation task is rendering a config from a template and writing it out. pathlib keeps it tidy:

from pathlib import Path

template = Path("templates/nginx.conf.tmpl").read_text(encoding="utf-8")
rendered = template.replace("{{PORT}}", "8080")

target = Path("/etc/nginx/conf.d/app.conf")
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(rendered, encoding="utf-8")

For anything beyond trivial substitution, reach for Jinja2, but the path handling stays the same.

Letting AI modernize legacy os.path code

If you have an old script full of os.path, converting it to pathlib is mechanical and a great task to hand an AI assistant. I will paste the file into Claude or Cursor and ask: “Rewrite this using pathlib, keep behavior identical, add explicit UTF-8 encodings.”

The conversion is usually clean, but I treat the assistant as a fast junior engineer and review every change, because file code can delete or overwrite real data. I look specifically for:

• Any unlink(), rmdir(), or shutil.rmtree() the AI introduced, and whether the target path is guarded.
• Glob patterns that got broadened (* becoming **) and now match more than intended.
• Whether a dry-run path still exists before destructive operations run.

I also never paste real config containing secrets into the prompt to “give context.” The AI does not need your credentials, and a path-handling refactor certainly does not. I keep these review notes in the prompt workspace and send destructive scripts through the code review dashboard first.

Conclusion

pathlib turns paths from fragile strings into objects with safe, readable operations: / for joining, glob/rglob for finding, mkdir(parents=True, exist_ok=True) for idempotent directories, and read_text/write_text for the small files that dominate ops work. Adopt it fully, pass explicit encodings, dry-run your deletes, and let an AI handle the tedious conversions while you review every destructive line.

More patterns live in the Bash and Python automation category. Reusable starters are in the prompt library, and curated sets are in the prompt packs.