🔀 Bash File-Descriptor Redirection: exec, tee, and Custom FDs for Script Logging

Most Bash scripts handle logging by sprinkling echo "..." >> /var/log/myscript.log everywhere and hoping for the best. It works until you need to also see the output on the terminal, or you need stderr in a separate file, or you discover that half your diagnostic chatter is polluting the data your script is supposed to emit on stdout. At that point you are reaching for file-descriptor redirection, and Bash gives you everything you need: exec to rewire descriptors for the whole script, tee to fork a stream to a file, and custom numbered FDs to carve out a private channel.

This is one of those areas where the syntax is terse and unforgiving, and a single misplaced & changes the meaning entirely. It is also where I find AI assistance genuinely useful for drafting the redirection line, because the operators are easy to fat-finger. As always, the draft is a starting point. AI drafts, human verifies, and with redirection you verify by watching where bytes actually land.

A Refresher on the Standard Descriptors

Every process starts with three open file descriptors: 0 is stdin, 1 is stdout, and 2 is stderr. Redirection operators rewire these. The two that trip people up:

• > file is shorthand for 1> file, redirecting stdout.
• 2> file redirects stderr.
• 2>&1 makes stderr point at wherever stdout currently points. Order matters: >file 2>&1 sends both to the file, but 2>&1 >file sends stderr to the original terminal and only stdout to the file.

That ordering gotcha is the single most common redirection bug, and it is worth internalizing before you go further.

Redirecting the Whole Script with exec

Inside a script, exec without a command applies redirections to the current shell process. That means every subsequent command inherits them. This is how you log an entire script without touching any individual line:

#!/usr/bin/env bash
set -euo pipefail

LOGFILE="/var/log/provision.log"

# Send everything from here on to the log file
exec >>"$LOGFILE" 2>&1

echo "Starting provisioning run at $(date -Is)"
do_some_work
echo "Done"

After that exec line, every echo, every command’s stdout, and every error all land in provision.log. Nothing reaches the terminal anymore, which is fine for a cron job but unhelpful when you want to watch it run.

Logging to a File and the Terminal at Once with tee

To keep terminal output while also capturing to a file, combine exec with process substitution and tee:

#!/usr/bin/env bash
set -euo pipefail

LOGFILE="/var/log/provision.log"

# stdout and stderr both go to the log AND to the terminal
exec > >(tee -a "$LOGFILE") 2>&1

echo "This shows on screen and is appended to the log"

The > >(tee -a "$LOGFILE") is process substitution: >(...) spawns tee and hands exec a file descriptor connected to its stdin. tee -a appends to the log and also passes everything through to its own stdout, which is still the terminal. Then 2>&1 folds stderr into the same path.

One caveat worth knowing: because tee runs as a background process, the script can exit before tee has flushed its final lines. For most scripts this is invisible, but if you see truncated logs on fast-exiting scripts, add a brief drain. A common pattern is to capture tee’s PID and wait on it, or simply trap exit and let the descriptors close cleanly.

Splitting stdout and stderr into Separate Files

When you want stdout in one file and stderr in another, do not collapse them. Keep two tee processes:

#!/usr/bin/env bash
set -euo pipefail

exec > >(tee -a /var/log/run.out) 2> >(tee -a /var/log/run.err >&2)

echo "this goes to run.out"
echo "this goes to run.err" >&2

The stderr substitution ends with >&2 so that tee’s passthrough copy still reaches the real terminal’s stderr rather than looping back into stdout. This pattern gives you a clean data channel and a clean diagnostics channel, which is exactly what you want when stdout is meant to be machine-parseable.

Custom File Descriptors for a Private Log Channel

Here is the pattern that solves the “my logging is polluting my data” problem. Open a dedicated FD, say 3, point it at your log, and write structured diagnostics there while stdout stays pristine for actual output:

#!/usr/bin/env bash
set -euo pipefail

# Open FD 3 for the log; stdout stays clean for data
exec 3>>/var/log/etl.log

log() { printf '[%s] %s\n' "$(date -Is)" "$*" >&3; }

log "ETL run begins"
generate_report_csv          # writes pure CSV to stdout
log "ETL run complete"

exec 3>&-   # close FD 3

Now myscript.sh > report.csv produces a clean CSV file, while all the timestamped log lines flow to /var/log/etl.log through FD 3. The >&3 on the printf directs output to descriptor 3. Always close custom descriptors with exec 3>&- when you are done, or let the process exit close them.

You can save the original stdout before redirecting, too, which lets you “escape” back to the terminal even after an exec redirect:

exec 4>&1            # stash the real stdout in FD 4
exec >>/var/log/job.log 2>&1
echo "this is logged"
echo "this prints to the real terminal" >&4

Letting AI Draft the Redirection, Then Verifying

The operators above are exactly the kind of thing that is faster to draft than to recall. When I am wiring up a new logging setup I will often start from a generated skeleton, using a prompt like the leveled logging library starter, and cross-check the redirection mechanics against the process substitution patterns prompt.

A typical exchange:

Prompt: Give me a Bash header that tees all output to a log with timestamps but keeps it on screen, and uses a separate FD for debug-only lines.

Response:

exec > >(tee -a "$LOG") 2>&1     # combined output, screen + file
exec 3>>"$DEBUG_LOG"             # FD 3 = debug-only channel
dbg() { printf 'DBG %s\n' "$*" >&3; }

Remember 2>&1 must come after the stdout redirect, and close FD 3 with exec 3>&- before exit.

That draft is correct and the caveat it volunteered is the real one. But I still verify, because redirection bugs are invisible until you look. My verification is mechanical: run the script, then prove each stream went where I claimed.

# Verify the split actually happened
./run.sh >/tmp/out.txt 2>/tmp/err.txt
echo "--- stdout ---"; cat /tmp/out.txt
echo "--- stderr ---"; cat /tmp/err.txt
echo "--- debug log ---"; cat "$DEBUG_LOG"

If a debug line shows up in out.txt, the FD wiring is wrong, no matter how confident the assistant sounded. AI drafts, human verifies.

Takeaways

exec rewires descriptors for the whole script, tee via process substitution lets you log and watch simultaneously, and custom FDs like 3 keep your diagnostics out of your data. Mind the ordering of 2>&1, remember that pipelines and process substitutions run tee in the background so fast scripts may need a drain, and close any descriptors you open. Use AI to draft the cryptic redirection lines, then run the script and actually look at where the bytes landed before you ship it.

More shell-plumbing guides live under Bash and Python automation.