Shell scripting is a powerful tool available on all platforms, even Windows, thanks to WSL. But it can be easy to make mistakes. Here are some tips to improve our scripts and avoid many problems.
#1: Better Shell Options
All shells have configurable options, which can be used to enable behavior. Many of them are considered safer than the shell defaults.
Fail on Errors
An absolute no-brainer is the option
With this option, any command returning a non-zero status will exit the whole script, and not execute any further commands.
By default, a shell script is run entirely, regardless of any errors:
line 4: non-existent: command not found Will still be echoed
-e we can prevent the
line 5: non-existent: command not found
There are certain exceptions to make it actually more usable:
|| commands lists: The full list will be evaluated, and will fail afterward, if necessary:
Test conditions are allowed to fail, e.g.,
failure after the failure
Fail on Unset Variables
Never trip over unset variables ever again!
set -u the script will exit on using an unset variable:
line 5: NOT_SET: unbound variable
Of course, there are some exceptions to the rules again:
- The special parameters
*are still allowed.
- Default values can be set:
One caveat exists, though: an array is assumed unset if empty.
By default, the exit code of a pipeline is determined by the last command regardless of any previous non-zero status codes:
last command bash: non-existent: command not found 0
set -o pipefail, a pipeline only returns a zero status if ALL the parts exited successfully.
This won’t affect that all parts of the pipeline will be executed.
But now we can use
set -e with pipelines.
Better Empty Globbing
Filename expansion, also known as globbing, can be the root of many bugs.
One particular questionable default is the handling of empty expansions.
If no files are found by expanding the glob, by default, it’s passed “as-is” (passglob), instead of an empty variable:
Output, if no log files are found:
This behavior can be disabled by
shopt -s nullglob, which is already the default in some non-bash shells:
NO OUTPUT if no log files are found.
set -f disables filename expansion altogether.
It’s a good idea if you don’t need globing for our script and prevent any accidental expansion.
set -x enables trace-mode, which will print each command to
stdout before actually executing it.
Remember that all options can be set from the outside, too:
#2: It’s a TRAP
Our scripts are often not stateless, creating the need for some kind of cleanup. Especially in case of a premature exit on errors, we need a way to get notified.
By using the built-in
trap function we can register a command to be executed for a specific signal:
We can use functions or commands directly:
Error-handling can be improved by using
$LINENO to actually know where it all went wrong.
- The Bash Trap Command (Linux Journal)
#3: Check Requirements Early
If our script relies on an external program not usually found in a default installation, we should check their existence first:
This little snippet helps to enforce that all requirements are available, or exits on the first unavailable command.
#4: Temporary Files & Directories
There are many reasons for needing temporary files: downloads, atomic operations, etc.
A random name for a temporary file or directory is mandatory, or we might overwrite something by accident.
The shell helps us with
mktemp, creating files and directories in
The command creates the file or directory, not just return a random name/path.
It has a
--dry-run / -u option, but it’s considered unsafe.
There are more options to customize the generated filename/path, check out its manpage.
#5: Quoting (almost) everything
Always use quotes. It’s better to quote too much than not enough.
As shown before, automatic parameter expansion can be the source of many bugs. To preserve the literal meaning of a string, we need to quote it.
If a string contains whitespace or an asterisk, it’s a ticking timebomb:
By quoting variables when expanding them, we can make sure that the result will be passed as a single argument:
Line 3: for f in $(ls *.m3u); do ^-- SC2045: Iterating over ls output is fragile. Use globs. ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options. Line 4: grep -qi hq.*mp3 $f \ ^-- SC2062: Quote the grep pattern so the shell won't interpret it. ^-- SC2086: Double quote to prevent globbing and word splitting. Did you mean: (apply this, apply all SC2086) grep -qi hq.*mp3 "$f" \ Line 5: && echo -e 'Playlist $f contains a HQ file in mp3 format' ^-- SC2039: In POSIX sh, echo flags are undefined. ^-- SC2016: Expressions don't expand in single quotes, use double quotes for that.
ShellCheck highlights the detected problems, showing us exactly what went wrong, and where, so it can easily be fixed.
It’s available in the repositories of most Linux distributions, and can also be integrated into many editors, like Visual Studio Code.
#7: Doin’ it in Style
I’ve argued before about using style guides for our preferred language, to establish a good baseline.
The most widely mentioned style guide for shell scripts is the “Shell Style Guide” by Google.
It’s an extensive read, but worth it.
As usual, don’t force what won’t fit. For example, I’m a strong proponent of using 4 spaces and a line length of ~110 spaces. No style guide will make me change my mind.
But I’m willing to adapt if a project demands it.
#8: Targeting the Right Shell
There are multiple shells available to use, with different options.
Especially in the age of containers, it’s no longer a given that we encounter a full-fledged bash shell.
That’s why we should try to target the right kind of shell, and not depend too much on shell-specific behavior. At least if we can’t be absolutely sure about which environment we’re running in.
If unsure, a shebang tells the executing shell how the script should be run:
bash isn’t available in
/usr/bin instead? We can utilize
env get the correct location by returning the first occurrence in
#9: Don’t Use Shell Script
An essential aspect of being a developer is knowing the limitations of languages and tools. Not everything should be a shell script. Many higher-level languages provide a safer and more concise environment to begin with.
And thanks to shebangs we can use many different languages in scripts, given a suitable interpreter is available:
Shell scripts are great. From automating a tiny task, up to full-fledged TUI apps like bashtop, almost anything is possible.
But the syntax is a little unusual compared to a high-level language. And errors can easily be introduced, even by just a typo.
Setting the right options and linting our scripts will help avoid common sources of problems in the first place. They are easy to use and integrate into our workflow, and shouldn’t be dismissed due to convenience, especially if others have to use our scripts.
What are your favorite tips and tricks for better shell scripts?