Shell

Table of Contents

1 Concepts

1.1 Command Interaction

The first type of interaction is pipeline. <cmd1> | <cmd2>, Each command execute in its own sub-shell. The exit status of a pipeline is the exist status of the last command. <cmd1> |& <cmd2> is short hand for <cmd1> 2>&1 | <cmd2>.

Commands separated by ; are executed sequentially. The shell waits for each to terminate in turn. The return status of last one is returned.

&& and || have a higher precedence than ;. The later command is executed based on the return result of previous one. The return status is the last executed command.

Grouping of command is different. The grouping can be done by () and {}. () will create a sub-shell and execute the body commands, while {} will not create sub-shell. The last command in the body MUST terminate with ;.

1.2 redirection

  • ls > dirlist 2>&1
  • &>word is the same as >word 2>&1
  • &>>word is the same as >>word 2>&1

Here document is of the form:

     cmd << AUniqueString
     any thing
     interesting
     AUniqueString

The start and end unique string must be same and must not appear in the body. The block will be used as the input to the command. Similarly, the here string is used by:

     cmd <<< $word

And the expansion of word will be used as the input.

2 expansion

There are seven kinds of expansions, in order of expansion order. After these expansion are performed, the quote removal are performed.

brace
/usr/local/{old,new,dist} will expands to three strings.
tilde
home
parameter and variable
in separate section
arithmetic
command substitution
using $() or `cmd`. The value is the output of the command, with any trailing newlines deleted.
word splitting
The shell treats each character of '$IFS' as a delimiter. If 'IFS' is unset, or its value is exactly <space><tab><newline>.
filename
The file must exists to be expanded. * matches any string, including null string. ? matches a single character. [..] matches any one of those.

2.1 parameter and variable expansion

  • ${var:-word}: if var is unset or null, the value is expansion of word
  • ${var:=word}: if var is unset or null, the expansion of word is assigned to var

Meta data:

  • ${#var}: return length in character of the expansion of var

Sub-string:

  • ${str:offset}: substr from offset to end
  • ${str:offset:length}: substr from offset for count characters

String trimming pattern is first expanded, then remove the shortest or longest match from the beginning or tailing of str. Return the value, leave str unchanged.

${str#word}
shortest, beginning
${str##word}
longest, beginning
${str%word}
shortest, tailing
${str%%word}
longest, tailing

Replacing

  • ${str/pattern/string}: the first longest match is replaced with string if pattern begins with:
    • /: all matched is replaced
    • #: match must happen in the begin
    • %: match must happen in the tail

Case changing match of pattern will change case

  • ${str^pattern}: one match, lowercase to uppercase
  • ${str^^pattern}: all match, lower to upper
  • ${str,pattern}: one match, upper to lower
  • ${str,,pattern}: all match, upper to lower

2.1.1 special parameters

  • $*: "$1c$2c$3c…", c is the first character of $IFS (defaults to space)
  • $@: "$1" "$2" "$3" …
  • $#: the number of positional parameters, for a.out -h it is 1
  • $?: exit status
  • the nth parameter
  • $-: current option flags
  • $$: process ID of the shell
  • $!: process ID of the job most recently placed into the background

3 IO

The loop can accept the redirection, thus can be used to read a file:

  while read -r line; do
      # some job
  done < papers.txt

The ordinary reading from command line:

read -p "please input: " a b c

When using echo, use -e option can print out control characters such as \n.

4 Control Flow

4.1 Condition commands

[ and ] are used to evaluate a conditional expression. Expressions can be combined by: !, (), -a, -o.

((...)) will cause the expression to be evaluated by shell arithmetic. If the value of the expression is non-0, the return status is 0, which is wired.

[[]] will return 0 or 1, depending on the evaluation of the conditional expression inside. The bash conditional expressions are the table done below.

The expression will performs some transformation. Not-performed:

  • Word splitting
  • filename expansion

Performed:

  • tilde expansion
  • parameter and variable expansion
  • arithmetic expansion
  • command substitution
  • process substitution
  • quote removal

When using == to compare string, the right hand side string is considered as a pattern, and the operation performed is pattern matching, as described in filename expansion.

The expression can be used with some operators:

(exp)
only add the precedence
!exp
negate
exp && exp
and
exp || exp
or

4.1.1 bash conditional expression (only work with double brackets)

expr meaning
-f file file exists and is regular file
-d file file exists and is directory
-a file file exists
-s file file exists and size > 0
-L <file> symbolic link
-r <file> readable
-w <file> writable
-x <file> executable
<file1> -nt <file2> newer than?
<file1> -ot <file2> older than?
-z string string is empty
-n string string is not empty
string1 == string2 equal
string1 != string2  

4.2 Conditional

The if clause signature is if ; then ; elif ; then ; else ; fi. The test command is not a condition, but a command. The return code of the command is used as the condition. If the command returns 0, the body is executed. Non-zero will perform the else clause.

case signature is: case word in p1) cmd;; p2) cmd;; esac The word undergoes

  • tilde expansion
  • parameter expansion
  • command substitution
  • arithmetic expansion
    • quote removal

Use * as the default pattern. Patterns can be combined with |.

Each case must be terminated by ;;, ;& or ;;&.

;;
no other matches are attempted
;&
continue with the next clause, without even evaluate the match
;;&
test next pattern

4.3 loop

As for other languages, break and continue can be used.

until cond; do cmd; done
while cond; do cmd; done
for name in words; do cmd; done
the words can be {1..10}, $(seq 1 10)
for ((i=0;i<10;i++)); do cmd; done

5 function

The function has two methods to declare, use the name and a pair of parenthesis, and the function keyword with optional parenthesis. I found using function keyword and NO parenthesis is the best because:

  • the function keyword tells me it is function, very explicitly
  • the parenthesis is not going to accept parameter, thus it causes confusion.

So in a word, use: function foo {}

A function declaration can be removed by unset -f. The argument of the function uses the same way for the argument to the script, the propositional special variables. And the call to the function is used as if it is a command.

Want the return value? The return statement will only set the return code of the command. Echo the result and assign the result to a variable may be a choice, but I found it not good because the body might echo something as well.

6 Tips

  • source will be in effect in current shell session, but not the sub-shells
  • export declare global variable, and will be in effect in sub-shells

6.1 Check whether a command exists

Do not use which, it is expensive, and the return value is not well defined. Use hash for commands and type when considering built-in and keywords. 1

  type foo >/dev/null 2>&1 || 
      { echo >&2 "I require foo but it's not installed.  Aborting."; exit 1; }
  hash foo 2>/dev/null ||
      { echo >&2 "I require foo but it's not installed.  Aborting."; exit 1; }

Footnotes: