Shell
Table of Contents
1 Concepts
1.1 Command Interaction
The first type of interaction is pipeline.
<cmd1> | <cmd2>
, Each command execute in its own sub-shell.
The exit status of a pipeline is the exist status of the last command.
<cmd1> |& <cmd2>
is short hand for <cmd1> 2>&1 | <cmd2>
.
Commands separated by ;
are executed sequentially.
The shell waits for each to terminate in turn.
The return status of last one is returned.
&&
and ||
have a higher precedence than ;
.
The later command is executed based on the return result of previous one.
The return status is the last executed command.
Grouping of command is different.
The grouping can be done by ()
and {}
.
()
will create a sub-shell and execute the body commands,
while {}
will not create sub-shell. The last command in the body MUST terminate with ;
.
1.2 redirection
ls > dirlist 2>&1
&>word
is the same as>word 2>&1
&>>word
is the same as>>word 2>&1
Here document is of the form:
cmd << AUniqueString any thing interesting AUniqueString
The start and end unique string must be same and must not appear in the body. The block will be used as the input to the command. Similarly, the here string is used by:
cmd <<< $word
And the expansion of word will be used as the input.
2 expansion
There are seven kinds of expansions, in order of expansion order.
After these expansion are performed, the quote removal
are performed.
- brace
/usr/local/{old,new,dist}
will expands to three strings.- tilde
- home
- parameter and variable
- in separate section
- arithmetic
- command substitution
- using
$()
or`cmd`
. The value is the output of the command, with any trailing newlines deleted. - word splitting
- The shell treats each character of '$IFS' as a delimiter.
If 'IFS' is unset, or its value is exactly
<space><tab><newline>
. - filename
- The file must exists to be expanded.
*
matches any string, including null string.?
matches a single character.[..]
matches any one of those.
2.1 parameter and variable expansion
${var:-word}
: if var is unset or null, the value is expansion of word${var:=word}
: if var is unset or null, the expansion of word is assigned to var
Meta data:
${#var}
: return length in character of the expansion of var
Sub-string:
${str:offset}
: substr from offset to end${str:offset:length}
: substr from offset for count characters
String trimming pattern is first expanded, then remove the shortest or longest match from the beginning or tailing of str. Return the value, leave str unchanged.
${str#word}
- shortest, beginning
${str##word}
- longest, beginning
${str%word}
- shortest, tailing
${str%%word}
- longest, tailing
Replacing
${str/pattern/string}
: the first longest match is replaced with string if pattern begins with:/
: all matched is replaced#
: match must happen in the begin%
: match must happen in the tail
Case changing match of pattern will change case
${str^pattern}
: one match, lowercase to uppercase${str^^pattern}
: all match, lower to upper${str,pattern}
: one match, upper to lower${str,,pattern}
: all match, upper to lower
2.1.1 special parameters
$*
: "$1c$2c$3c…", c is the first character of $IFS (defaults to space)$@
: "$1" "$2" "$3" …$#
: the number of positional parameters, fora.out -h
it is 1$?
: exit status- the nth parameter
$-
: current option flags$$
: process ID of the shell$!
: process ID of the job most recently placed into the background
3 IO
The loop can accept the redirection, thus can be used to read a file:
while read -r line; do # some job done < papers.txt
The ordinary reading from command line:
read -p "please input: " a b c
When using echo
, use -e
option can print out control characters such as \n
.
4 Control Flow
4.1 Condition commands
[
and ]
are used to evaluate a conditional expression.
Expressions can be combined by: !
, ()
, -a
, -o
.
((...))
will cause the expression to be evaluated by shell arithmetic.
If the value of the expression is non-0, the return status is 0, which is wired.
[[]]
will return 0 or 1, depending on the evaluation of the conditional expression inside.
The bash conditional expressions are the table done below.
The expression will performs some transformation. Not-performed:
- Word splitting
- filename expansion
Performed:
- tilde expansion
- parameter and variable expansion
- arithmetic expansion
- command substitution
- process substitution
- quote removal
When using ==
to compare string, the right hand side string is considered as a pattern,
and the operation performed is pattern matching, as described in filename expansion.
The expression can be used with some operators:
(exp)
- only add the precedence
!exp
- negate
exp && exp
- and
exp || exp
- or
4.1.1 bash conditional expression (only work with double brackets)
expr | meaning |
---|---|
-f file |
file exists and is regular file |
-d file |
file exists and is directory |
-a file |
file exists |
-s file |
file exists and size > 0 |
-L <file> |
symbolic link |
-r <file> |
readable |
-w <file> |
writable |
-x <file> |
executable |
<file1> -nt <file2> |
newer than? |
<file1> -ot <file2> |
older than? |
-z string |
string is empty |
-n string |
string is not empty |
string1 == string2 |
equal |
string1 != string2 |
4.2 Conditional
The if
clause signature is if ; then ; elif ; then ; else ; fi
.
The test command is not a condition, but a command.
The return code of the command is used as the condition.
If the command returns 0, the body is executed.
Non-zero will perform the else
clause.
case
signature is: case word in p1) cmd;; p2) cmd;; esac
The word undergoes
- tilde expansion
- parameter expansion
- command substitution
- arithmetic expansion
- quote removal
Use *
as the default pattern.
Patterns can be combined with |
.
Each case must be terminated by ;;
, ;&
or ;;&
.
;;
- no other matches are attempted
;&
- continue with the next clause, without even evaluate the match
;;&
- test next pattern
4.3 loop
As for other languages, break
and continue
can be used.
until cond; do cmd; done
while cond; do cmd; done
for name in words; do cmd; done
- the words can be
{1..10}
,$(seq 1 10)
for ((i=0;i<10;i++)); do cmd; done
5 function
The function has two methods to declare, use the name and a pair of parenthesis, and the function keyword with optional parenthesis. I found using function keyword and NO parenthesis is the best because:
- the function keyword tells me it is function, very explicitly
- the parenthesis is not going to accept parameter, thus it causes confusion.
So in a word, use: function foo {}
A function declaration can be removed by unset -f
.
The argument of the function uses the same way for the argument to the script, the propositional special variables.
And the call to the function is used as if it is a command.
Want the return value? The return
statement will only set the return code of the command.
Echo the result and assign the result to a variable may be a choice, but I found it not good because the body might echo something as well.
6 Tips
source
will be in effect in current shell session, but not the sub-shellsexport
declare global variable, and will be in effect in sub-shells
6.1 Check whether a command exists
Do not use which
, it is expensive, and the return value is not well defined.
Use hash
for commands and type
when considering built-in and keywords. 1
type foo >/dev/null 2>&1 || { echo >&2 "I require foo but it's not installed. Aborting."; exit 1; } hash foo 2>/dev/null || { echo >&2 "I require foo but it's not installed. Aborting."; exit 1; }