Personal Programming Notes

To err is human; to debug, divine.

Bash Trap

In this post, we discuss common usage of bash trap to ensure proper cleanup operations in Bash scripts. It also discusses a common idiom trap cleanup INT TERM EXIT where other signals such as INT and TERM is also trapped in addition to EXIT. While such idiom could be valid in some Unix system, it is usually redundant and can be simply wrong (duplicate executions) in most cases, as shown on Mac. A simple test is provided to verify if such idiom is applicable in your current system.

Standard usage

There is a simple idiom to ensure your bash scripts to always do proper cleanup operations before exiting, even when something goes wrong during execution. In the context of Java or Python, this is similar to a finally clause that will execute after any exception is caught during execution.

DO THIS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

# Setup trap to cleanup before exiting script
function cleanup {
    echo "Removing temp files..."
    if [[ -f $CMD_TMPFILE ]] ; then
        rm $CMD_TMPFILE
    fi
    if [[ -f $LOG_TMPFILE ]] ; then
        rm $LOG_TMPFILE
    fi
}
trap cleanup EXIT

# Setup

# Thousand lines of code here

Putting the cleanup operations at the end of the bash script might not work in cases of error. Since the bash script already stops executing due to some fatal error, those clean up commands might never run.

DON'T DO THIS
1
2
3
4
5
6
7
8
9
10
11
12

# Setup

# Thousand lines of code here

# This might not run when there is error
echo "Removing temp files..."
if [[ -f $CMD_TMPFILE ]] ; then
    rm $CMD_TMPFILE
fi
if [[ -f $LOG_TMPFILE ]] ; then
    rm $LOG_TMPFILE
fi

For example, in Vertica, you should always run SELECT START_REFRESH() at the end of a deployment script, regardless of any error encountered during script execution. It is a good candidate for using trap statement. Adding those commands at the end of the script will not work in cases there is an error during deployment, and you might end up with “AHM Does Not Advance”-related errors (see this post).

Trap multiple signals

Note that many online examples for trap use a list of signals for cleanup tasks like this trap cleanup INT TERM EXIT, i.e., trapping not only EXIT signal but also INT and TERM signals. I believe that if EXIT signal is used, other signals such as INT or TERM are redundant for cleanup purposes. EXIT or 0 signal is invoked when the shell exits, an event that also happens when an INT or TERM signal is received. It is easy to confirm that with the following short bash script:

Trap tests in Mac OSX
1
2
3
4
5
6
7
8
9
10
11
12
13
14

MTVL1288aeea2-82:code tdongsi$ cat test_trap.sh
#!/bin/bash
trap 'echo SIGNAL CAPTURED' EXIT
sleep 3

MTVL1288aeea2-82:code tdongsi$ ./test_trap.sh & sleep 1; kill -INT %1
[1] 6613
SIGNAL CAPTURED
[1]+  Interrupt: 2            ./test_trap.sh

MTVL1288aeea2-82:code tdongsi$ ./test_trap.sh & sleep 1; kill -TERM %1
[1] 6624
SIGNAL CAPTURED
[1]+  Terminated: 15          ./test_trap.sh

As shown above, a lone EXIT is enough to capture INT and TERM signals. Having said that, I understand that my tests can only verify bash on Mac OSX. There are probably different shell variants on different operating systems which do not always work that way.

The problem of those trap examples lies in when someone copies and uses the code directly from the web, without understanding how it works. Listing multiple signals can make the cleanup steps executed twice, once for the signal such as TERM and once for EXIT, as shown in the modified experiment below. Not all cleanup steps could be and should be executed twice. For example, it is almost always true that removing some temporary file/folder should not be executed twice during a cleanup.

Problem of trapping multiple signals
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

MTVL1288aeea2-82:code tdongsi$ cat test_trap.sh
#!/bin/bash
trap 'echo SIGNAL CAPTURED' INT TERM EXIT
sleep 3

MTVL1288aeea2-82:code tdongsi$ ./test_trap.sh & sleep 1; kill -INT %1
[1] 7258
SIGNAL CAPTURED
SIGNAL CAPTURED
[1]+  Exit 130                ./test_trap.sh

MTVL1288aeea2-82:code tdongsi$ ./test_trap.sh & sleep 1; kill -TERM %1
[1] 7278
Terminated: 15
SIGNAL CAPTURED
SIGNAL CAPTURED
[1]+  Exit 143                ./test_trap.sh

In short, you should know how trap works on your production system before listing multiple signals as its parameters, especially when coupled with EXIT signal.

Other usage notes

The signal names might be specified with or without prefix SIG or even with numeric values for signal numbers, e.g., 2 for INT (see list below).

List of signals
1
2
3
4
5
6
7
8
9
10
11
12
13

MTVL1288aeea2-82:octopress tdongsi$ kill -l
 1) SIGHUP     2) SIGINT   3) SIGQUIT  4) SIGILL
 5) SIGTRAP    6) SIGABRT  7) SIGEMT   8) SIGFPE
 9) SIGKILL   10) SIGBUS  11) SIGSEGV 12) SIGSYS
13) SIGPIPE   14) SIGALRM 15) SIGTERM 16) SIGURG
17) SIGSTOP   18) SIGTSTP 19) SIGCONT 20) SIGCHLD
21) SIGTTIN   22) SIGTTOU 23) SIGIO   24) SIGXCPU
25) SIGXFSZ   26) SIGVTALRM   27) SIGPROF 28) SIGWINCH
29) SIGINFO   30) SIGUSR1 31) SIGUSR2

OR 

MTVL1288aeea2-82:octopress tdongsi$ man signal

If one of the signals specified in trap statement is DEBUG, the list of COMMANDS specified in trap statement will be executed after every simple command. This is useful for debugging purpose. The following example is taken from here:

Tracing when a variable is used
1
2
3
4

declare -t VARIABLE=value
trap "echo VARIABLE is being used here." DEBUG

# rest of the script

References

  1. Signals and Traps
  2. Other usages
  3. declare