Shell Redirection 101
Shell redirection can be confusing.
I’ve always wondered what
2>&1 means and why it’s needed, but as many others, I just copy/pasted things from the internet into my terminal and hoped for the best.
What could possibly go wrong?
But no more! It’s finally time to look behind the curtain and decipher and understand what’s actually happening.
Table of Contents
To understand how redirection works, you first have to look at what the numbers actually represent: file descriptors (FD).
A file descriptor is a “process-unique identifier” or “handle” to a file or another I/O resource. They’re part of the POSIX API and each process “should” have three standard POSIX FDs:
You can actually check where these file descriptors are pointing at:
Don’t worry about any warning that the command might output, as
lsof checks all file systems and may lack the required permissions to do so.
However, as you can see in the example output, the 3 FDs,
2, are all mapped to a
Depending on your terminal, it might be something different.
On my Mac, the standard FDs point to
For simplicity reasons, I’ll use
/dev/tty0as it usually represents the current virtual console and is found in many guides.
Writing to the Standard File Descriptors
Most programming languages have means to write to either output FDs, or read from the input, like in Java:
… or Go:
Redirecting File Descriptors
The main reason you might want to redirect the standard FDs is how they’re used by their owning process, which might not match our requirements.
Maybe you want to save a command’s output into a file on disk, or read a file for input instead from
Or you want to “connect” two commands, passing the first output to the second command’s input.
These are all use cases for redirects.
By default, the standard FDs are mapped according to their direction to the same thing:
There are 3 operators for redirecting FDs:
- Output redirection with
- Input redirection with
- Combining output and input with
Let’s take a look at all of them and their different use cases!
Output Redirection to a File
The simplest and maybe most common redirection is to point the
stdout FD towards a file:
To achieve this, you just need to combine the correct source, operator, and target:
> (output redirection)
A file on disk
If you want to save the output from Gradle’s dependencies task into a file, this would look like this:
stdout is, well, the “standard output”, this commonly used type of redirection is the default FD for the
> operators, so you don’t need to add the FD at all, which makes it easier on the eyes and more straight-forward:
To redirect only error output, you need to redirect
stderr) in the same way as before, so the redirection looks like this:
The space between the operator and target is optional and often omitted, so the redirection builds a single block.
A great use case for redirecting
stderr is to suppress the output completely by redirecting the magical black hole of any system:
For example, the
find command might output “Permission denied” errors, which you want to ignore.
By redirecting the
stderr FD to
/dev/null, only the successful output will be displayed, or redirected to a file:
Or, if you’re interested in which directories have the wrong permissions, you can easily create an error log file:
Input Redirection from a File
Redirecting a file as input is equivalent to redirecting the output to one, expect using
The used operator actually matches the arrowhead on the direction:
Let’s say you want to extract the database port from a JSON config file with
To do so, you can redirect the config file to
Most commands support a file as one of their arguments, so redirecting a file to
stdin might seem superfluous.
However, the possibility of redirection enables the next type of redirection: pipes.
Connection Commands with Pipes
| (pipe) operator simply combines a command’s
stdout with the
stdin following the operator.
For example, looking up all markdown files and sorting the output can be done by using the output of
find as input for
Thinking in FDs, the following is happening:
This works because any redirections are connected before the commands are actually executed.
Even More Shenanigans with File Descriptors
We looked at the three fundamental operators and use cases, but there’s a lot more to FD redirection. And even more important, the common pitfalls and errors you may encounter.
Duplicating File Descriptors
As seen in the section about
| (pipe), it’s an easy way to connect
stdout of one process with
stdin of another.
But what about redirecting more than one FD?
This article opened with the redirection
2>&1 so let’s dissect it and learn what it does.
& (ampersand) modifier indicates that the left side of the operator “duplicates” the target of the right side.
In this case, that means that
stderr (2) redirects wherever
stdout (1) is going:
Be aware that FDs are always duplicated, not aliased! The arrows in the graphs represent distinct FDs from the source to the target.
To simplify the call, you can use the shortcut
|& to redirect both
stderr into the next command and, therefore, removing the additional
As your requirements might be more complex than “redirect x to y”, you can use multiple redirection commands to set up the FDs as you need.
For example, you want to output both
stderr to the same file.
Using what we’ve learned so far, this would require two steps:
stdoutto a file (
The resulting call would most likely look like this:
Even though you’d think that’s the obvious solution, it’s a common error when dealing with multiple redirections. It’s time for another redirection dissection!
Redirections are set up from left to right. We start as usual:
2>&1 gets set up and changes the FDs to this:
Well, nothing happened, but why?
stderr already has its distinct FD to the target of
stdout, there’s no need to duplicate it.
The second redirect,
>file, will lead to this:
There’s still a
file with the contents of
stdout, and you might think there are just no errors.
So the correct order for the two redirections is:
command >file 2>&1
stdout is redirected to
stderr duplicates the target of
As with the other common use cases, there’s a shortcut to simplify the redirections and don’t mix up the order:
Which one to prefer is up to you, as there are multiple options available:
Appending Vs. Overwrite files
The output redirection
>file always overrides
file if it exists.
If you want to append to it instead, you can use
In the case of a non-existing file, it gets created either way.
Preventing Accidental Overwrites
< share the same key, mixing them up is easy, and accidentally destroying a file by redirecting from
file instead of redirecting
Or you typed a single
> instead of the appending variant
You can use the shell option
set -o noclobber to prevent
> from overwriting a pre-existing file.
If the need to override a file arises, you don’t have to disable and re-enable the option each time, you can simply force it with
Reading and Writing the Same File
Another common error is trying to modify a file with a command that writes to
As redirections are set up before executing the command,
stdout gets redirected to
file which incidentally truncates
> is used.
In the case of
sed, you can use the
-i option for in-place editing.
A more general solution is using
| and the
tee command, as it reads from
stdin and writes to the provided file:
Shell redirections are powerful tools for building quite intricate command pipelines and bending the I/O to your will. That said, they’re also quite confusing if you’re not used to them. However, knowing the basics, like the different operators and FDs, will improve your shell productivity and help you to decipher all those weird one-liners you might copy into your shell.
If you want to know more about shell redirection, you can always consult the
man page of
bash directly in your terminal:
- Bash Reference Manual (gnu.org)
- Zsh Redirection (zsh.sourceforge.io)
- How to manipulate files with shell redirection and pipelines in Linux (redhat.com)