Teaching:
FeedbackThis is an old revision of the document!
The login shell in a terminal emulator runs in interactive mode.
When the shell executable is invoked with a file as its argument, it runs in
non-interactive mode, and reads the commands from the file rather than the
standard input.
Some diagnostics are not printed in the non-interactive mode.
Exercise 1 Edit a file filename so that in contains the text:
date echo hello world date
Then, run /bin/bash filename.
Shebang is the name for the
first line of a file whenever the line contains #! followed by a full path
to an executable (and optionally arguments).
A file that has execute permission and starts with a shebang is called a script. When a script is executed, the executable indicated in the shebang is used to interpreted the file.
For instance, when an executable file called myFile that starts with
the line #!/bin/prog_name myArg is executed by the standard POSIX
execlp("./myFile", "./myFile", NULL); function, then the kernel,
after verifying the execute permissions and noticing the shebang, executes in
return execl("/bin/prog_name", "/bin/prog_name", "myArg", "./myFile", NULL);.
An executable file that starts with #!/bin/sh is a shell script.
(just like an executable file that starts with #!/usr/bin/python3 is
a python script).
Exercise 2
Modify the file filename created for the last exercise so that it starts with
#!/bin/sh shebang.
Then, grant execute permission to the file (e.g., with chmod +x filename).
Finally, run the file with ./filename.
The shell has a . command (the command is literally a single dot).
The . filename executes the commands from the file filename in the
current shell (as if the contents of the file were typed to the standard input).
The file does not need to be executable.
Notice that ./file (and sh file) interprets the file in a new shell and
. ./file (and . file if PATH contains a .) executes the commands
from the file in the current shell.
The shell, in that order:
| 1. reads a line of input | echo ${Y} "$Z 2+3=$((2+3))" > file; l 'baz bar' |
|
| 2. splits the line into tokens | echo, ${Y}, "$Z 2+3=$((2+3))", >, file, ;, l , 'baz bar' |
|
| 3. parses tokens into commands | echo, ${Y}, "$Z 2+3=$((2+3))", >, file | l, 'baz bar' |
| 4. resolves aliases | echo, ${Y}, "$Z 2+3=$((2+3))", >, file | ls, -alF, --color=auto, 'baz bar' |
| 5. expands certain tokens | echo, a, b, c d 2+3=5, >, file | ls, -alF, --color=auto, baz bar |
| 5. sets up redirections | echo, a, b, c d 2+3=5 | … |
| 6. executes the command | ||
(cf. POSIX documentation)
Commands can be separated by:
; – cmd ; … fully executes cmd and only then goes to the next command,& – cmd & … starts cmd in background and immediately goes to the next command.
Each process returns an exit value. Value of 0 indicates success, non-zero value
indicates abnormal termination1).
Commands can be joined with logical operators that depending on the return values:
&& – a && b runs a and if a succeeds, b is run; a && b returns success iff both a and b succeed,|| – a || b runs a and if a fails, b is run; a || b returns failure iff both a and b fail.Commands can be grouped by:
{ … } – groups commands to be executed in this shell, { … } must be terminated by a ; or a & { and precede the }.(…) – runs a new shell (subshell) and executes the commands in the subshell
To understand the difference, consider the following commands executed from the /home/student directory:
{ cd /tmp; pwd; }; pwd – enters /tmp and outputs /tmp twice,(cd /tmp; pwd); pwd – runs a subshell, enters /tmp and outputs /tmp in the subshell, the subshell terminates, the shell outputs /home/student.
Exercise 3
Run the commands pwd and ls from the same line.
Exercise 4
Write a line that will run the command cd Desktop and if it succeeds,
will run a ls command.
When a & terminates a command, then the process is run in background.
At a time, at most one process can run in foreground, but any number of
processes may run in background or be suspended.
One may suspend a foreground process by pressing Ctrl+z.
The list of processes controlled by the shell, that includes the background
processes and suspended processes, can be printed by the jobs command.
To move a background or suspended process to the foreground, one has to issue
the fg command. To continue a suspended process in the background, one
can issue the bg command.
With no arguments, the fg and bg target the most recently suspended
process, and if there are no suspended process, then the most recently
backgrounded process.
The fg, bg and kill shell commands accept a job specifiers
to select a particular process by its number, name, ….
Exercise 5
Run a command ping put.poznan.pl. Suspend it with Ctrl+z. Inspect the
jobs table with jobs. Put the command back in foreground and terminate it
with Ctrl+c.
Exercise 6
Run a command ping put.poznan.pl &. Inspect the jobs table with jobs.
Put the command back in foreground and suspend it. Then, issue a command that
will continue it in background.
The shell expands the following tokens:
~, ~/… (and …:~ / …:~/… in variable assignment) is expanded to home directory of the current user, ~user, is expanded to home directory of the user,$NAME and ${…} are substituted by a variable/parameter value $((…)) is substituted by the result of arithmetic calculations2),$(…) and `…` are substituted by the standard output of given commands*, ? and […]) are expanded to filenames wherever possible
Inside double quotes only expansions 2-4 are performed.
Inside single quotes no expansions are performed.
Importantly, the results of expansions 2-4 are split again into tokens whenever the expansions are not double-quoted.
Environment variables
are key-value pairs associated with a process.
Their names must not contain the equals sign (=).
The usual naming convention is that the names start with a capital letter and
contain only capital letters, numbers and underscore.
In the shell, the characters a-zA-Z0-9_ are valid in variable names, and the
variables must not start with a number.
Each process has a separate set of environment variables.
Upon fork, the environment variables are copied. Upon exec…,
the variables are either retained or replaced with a given set (cf. execv vs execve).
The C language functions getenv
and setenv
provide access to the environment variables.
For the shell, the name
parameter
denotes either script/function arguments or some special variables, while the name
variable
denotes an ordinary shell variable (that is closely related to an environment variable).
The shell variables marked as exported are passed to child processes as
their environment variables. One can export a variable called NAME by
the means of export NAME command.
To set a variable, one has to write:
NAME=value
Warning: NAME =value attempts to run the program NAME with argument =value.
Warning: NAME = value attempts to run the program NAME with arguments = and value.
Warning: NAME= value attempts to run the program value with environment variable NAME set to empty value.
One may unset the variable with unset NAME.
The list of all variables can be obtained by the set command (with no arguments),
and the list of all exported variables can be obtained by the env command
(with no arguments).
The shell substitutes $X and ${X} with the value of the
variable X.
Moreover the following expansions are performed:
${#X} → the length of the value (in characters)${X:-expr} → value of X if X is set and nonempty, expr otherwise${X:+expr} → expr if X is set and nonempty, empty value otherwise${X:number} → value of X with first number of characters skipped${X::number} → the first number of characters of X's value ${X:skip:length} → the first length characters starting from skip+1 of the X's value${X%pattern} → X's value with the shortest match of the pattern removed from the beginning ${X%%pattern} → X's value with the longest match of the pattern removed from the beginning${X#pattern} → X's value with the shortest match of the pattern removed from the end ${X##pattern} → X's value with the longest match of the pattern removed from the end
Exercise 7
Print the value of a variable VAR using the echo command.
Then, set the variable VAR to text.
Finally, print the value of the variable VAR again.
Exercise 8
Assign the value of VAR to a variable OTHER and run echo
that outputs the value of the OTHER variable.
Exercise 9
Assign the value 5 to a variable named SIZE.
Then, construct
a command that uses the variable SIZE to print The file size is 5MB
Exercise 10
Set the variable PROG to ls and the variable ARG to /tmp.
Then run in the shell $PROG $ARG.
Arguments of a script or a function are referred to by ${1}, ${2},
${3}, ….
The first nine arguments can also be referred to by $1, $2, $3, …, $9.
Moreover, the following substitutions are performed:
$# | the number of arguments |
$* $@ | all arguments split into single words |
"$*" | all arguments as a single token |
"$@" | all arguments, each being a separate token |
~ $ bash -s "the_first_argument" "the second argument" user@host ~ $ echo $# 2 user@host ~ $ ls $* ls: cannot access 'the_first_argument': No such file or directory ls: cannot access 'the': No such file or directory ls: cannot access 'second': No such file or directory ls: cannot access 'argument': No such file or directory user@host ~ $ ls $@ ls: cannot access 'the_first_argument': No such file or directory ls: cannot access 'the': No such file or directory ls: cannot access 'second': No such file or directory ls: cannot access 'argument': No such file or directory user@host ~ $ ls "$*" ls: cannot access 'the_first_argument the second argument': No such file or directory user@host ~ $ ls "$@" ls: cannot access 'the_first_argument': No such file or directory ls: cannot access 'the second argument': No such file or directory
Other special parameters include:
$0 | within script, $0 holds the name of the script |
$$ | process identifier of the shell |
$? | exit status of the last command |
$! | process identifier of the last backgrounded process |
user@host ~ $ sleep 1m & [1] 21738 user@host ~ $ sleep 1s user@host ~ $ echo $! 21738 user@host ~ $ echo $? 0 user@host ~ $ false user@host ~ $ echo $? 1 user@host ~ $ python <<< 'exit(123)' user@host ~ $ echo $? 123 user@host ~ $ echo $$ 21731 user@host ~ $ ps PID TTY TIME CMD 21731 pts/9 00:00:00 bash 21738 pts/9 00:00:00 sleep 21742 pts/9 00:00:00 ps
Exercise 11 Write a shell script that prints its first and third argument.
Exercise 12 Write a shell script that prints its name and the number of its arguments.
PATH | Colon-separated list of directories where binaries are looked up Notice: it's the exec…p system call that looks up an executable files with a matching name in these paths |
HOME | Katalog domowy bieżącego użytkownika |
PS1 PS2 | The main prompt The prompt for next line of multi-line commands |
PWD | Current working directory |
EDITOR | Default text editor |
LOGNAME / USER UID | Name of the current user (Numeric) user identifier of the current user |
LANG | Language, region and character encoding; for instance de_CH.UTF-8 means german, Switzerland and UTF-8 encoding |
RANDOM | A random number (generated upon each access) |
Exercise 13
Set the variable LANG to the value ja_JP.UTF-8 and run the program date.
Then, change the value of LANG to de_DE.UTF-8 and run the command rm -rf /root/.ssh/nope.
Exercise 14
Display the value of the PS1 variable. Then, change it.
Exercise 15
Try to run the lspci command. Then modify the PATH variable so that it
additionally contains the paths /sbin and /usr/sbin. Retry the lspci
command.
The shell can do simple integer maths – it substitutes $((…))
with the result of an expression contained in the parentheses.
The shell understands assignments and operators similar to those in C (cf.
table of operators).
Inside $((…)) one can leave out the $ preceding variable names, e.g., X=$((X+2)).
Exercise 16 Set the value of X and Y to some two digit numbers, and then:
• increment X by 1,
• increment Y by 2,
• set Z to the result of X multiplied by Y,
• calculate the reminded of dividing Z by 128.
The shell substitutes for $(command) (and `command`) the
standard output of the command.
Notice that first the commands contained in $(…) execute, then the shell
replaces $(…) with the standard output (and if the $(…) was unquoted,
the result are split into tokens), and only afterwards the line containing
$(…) is run.
Exercise 17
Write a command that will print the text In the current directory there are N files,
with N being replaced by the real count of the files in the current directory.
Exercise 18
The command date +%H_%M_%S outputs the current time.
Write a command that redirects the output of pstree -au to a file
processes_TIME.log with the TIME substituted by the current time.
Exercise 19
Write one line that will:
• assign the current time in nanoseconds (the output of date +%s%N) to a variable START,
• run the command sleep 1s,
• assign the current time in nanoseconds to a variable END,
• output the time elapsed between START and END
Aliases
are intended to provide user-defined alternative names for a command.
Once a line is split into commands by the shell (but before other expansions are
done), the shell checks whether a word that is present where a command name is
expected matches any alias, and if so, the word is replaced by the value of the
alias.
For instance: some Linux distribution in default configuration files set up the
alias la to ls -la and the alias ls to ls --color=auto.
Typing la first turns that into ls -la, and then ls is turned
into ls --color=auto, yielding total of ls --color=auto -la
To create a new alias, one should issue:
alias word=value
For instance: alias la="ls -la"
An alias can be removed by unalias word.
The alias command with no arguments lists defined aliases.
Aliases, just like variables, are set only for the current shell process.
Exercise 20 Create an alias called year that will translate to cal -my.
Use the alias. Use the alias to display the calendar for 2025.
The read
command reads a line from the standard input.
read requires as arguments a list of variable names, e.g., read LINE or
read A B C
Once a line is read, it is split into words, and subsequent words are assigned
to subsequent variables in the list.
If there are less words in the input line than the variables in the list, then
the remaining variables are set to an empty value.
If there are more words in the input line than the variables in the list, then
the last variable in the list gets the extra text.
user@host ~ $ read X Y Z one "two three" user@host ~ $ echo "X='$X' Y='$Y' Z='$Z'" X='one' Y='"two' Z='three"' user@host ~ $ read X Y Z one user@host ~ $ echo "X='$X' Y='$Y' Z='$Z'" X='one' Y='' Z='' user@host ~ $ read X Y Z one two three four five user@host ~ $ echo "X='$X' Y='$Y' Z='$Z'" X='one' Y='two' Z='three four five'
Exercise 21
Print the text What is your name: , then read the name, and print the text
Hello name!.
Exercise 22 Read two numbers at a time and then output their sum.
The test command allows evaluating logical tests that involve strings,
numbers and files.
Apart from the name test, the command can also be invoked by the name [.
There is a slight difference between test and [: the latter needs
an extra ] argument as the last argument.
E.g., test "foo" != "foo" is identical to [ "foo" != "foo" ].
| Expression | True when |
|---|---|
[ "$X" ] [ -n "$X" ] | "$X" nas nonzero length |
[ -z "$X" ] | "$X" has zero length |
[ "$X" = "$Y" ] | The strings are the same |
[ "$X" != "$Y" ] | The strings differ |
Warning: the expression is going to work if [ $X = $Y ]$X
and $Y expand to single words, but if either of the variables is empty or
contains a space, then a syntax error will be raised.
| Expression | True when |
|---|---|
[ "$X" -eq "$Y" ] | The number "$X" is equal to "$Y" |
[ "$X" -ne "$Y" ] | The number "$X" is not equal to "$Y" |
[ "$X" -lt "$Y" ] | The number "$X" is lesser than "$Y" |
[ "$X" -le "$Y" ] | The number "$X" is lesser than or equal to "$Y" |
[ "$X" -gt "$Y" ] | The number "$X" is greter than "$Y" |
[ "$X" -ge "$Y" ] | The number "$X" is greter than or equal to "$Y" |
| Expression | True when the file "$X" exists and… |
|---|---|
[ -e "$X" ] | (just exists) |
[ -s "$X" ] | is nonempty |
[ -f "$X" ] [ -d "$X" ] … | is an ordinary file is a directory … |
[ -r "$X" ] [ -w "$X" ] [ -x "$X" ] | the current user can read the file the current user can write to the file the current user can execute the file |
! arg negates arg.
arg1 -a arg2 stands for arg1 and arg2
arg1 -o arg2 stands for arg1 or arg2
Parentheses ( … ) group the expressions, but one has to escape them (else
the shell sees them as shell special characters).
Exercise 23
Using the && and || syntax to group commands:
• test whether a directory dirname exists, and if it does exist, enter it,
• test whether the variable X is larger than the variable Y, and if the test fails, print Y≥X.
The if statement has the following syntax:
if command1
then
statements executed when command1 returned the exit value indicating success
elif command2
then
statements executed when command2 returned the exit value indicating success
else
statements executed when neither command1 nor command2 returned the exit value indicating success
fi
The elif and else branches are optional, and elif can appear multiple
times.
It is worth underlining that the if and elif require a command and check
the exit value returned by the command, and disregard the data written to the
standard output/error streams by the command.
A one-liner example:
if grep -q ^user: /etc/passwd; then echo "'user' present"; elif grep -q ^student: /etc/passwd; then echo "'user' absent, 'student' present"; else echo "neither present"; fi
Exercise 24
Write an if statement that tries to remove a file filename and upon
success outputs Removed the file while upon failure it outputs
Failed to remove the file.
Exercise 25 Create a script that verifies if it was run with exactly two arguments and if the first argument is lesser then or equal to the second argument. If any of the conditions does not hold, print a corresponding text and exit the script.
The case statement (in many other languages called the switch statement) has the following syntax:
case value in pattern1) statements1 ;; pattern2) statements2 ;; *) statements3 ;; esacThe value is consecutively matched against pattern1, pattern2, … and upon first match the corresponding statements are executed. Subsequent patterns are not taken under account.
;; terminating statements corresponding to a pattern.
* is used.
A one-liner example:
case $(LANG= date "+%A") in Monday) echo "Ugh...";; S*day) tput bel; echo "Weekend!";; *) echo "a day.";; esac
Exercise 26
Write a script that accepts a filename as an argument, and if the file has an extension:
• pdf, then run the pdftotext filename - command,
• zip, then run the unzip -l filename command,
• any other extension, then run the cat filename command.
In the loops one can use the break and continue keywords to,
respectively, leave the loop and start the next loop iteration.
The for loop iterates over a set of values. It has the following syntax:
for VAR in value1 value2 … do statements doneIn subsequent iterations, subsequent values from the list are assigned to the VAR variable. Once the loop finishes, the VAR contains the last value assigned in the loop.
in value1 value2 … part is omitted, the for
iterates over the positional parameters.
Usually the list of values is not hand-written, but is a result of an
expansion, for instance:
for NUM in $(seq 7) → for NUM in 1 2 3 4 5 6 7
for IMG in *.jpg → for IMG in alice.jpg bob.jpg eve.jpg
for USERNAME in $(getent passwd | cut -f1 -d:) → for USERNAME in root student ...
for FILE in $(grep -il 'ldap' /etc/*.conf 2>/dev/null) → for FILE in /etc/autofs.conf /etc/ldap.conf ...3)
A one-liner example:
for X in `seq 3`; do echo -n "Iteration $X: "; date +%N; done
Exercise 27 Write a script that accepts a list of files as the arguments, and for each file prints its name, its first line and its last line.
Exercise 28
Write a loop that outputs all powers of two up to a given exponent.
(To calculate the power, you can use the expression $((2**exponent)).)
The syntax of the while loop is as follows:
while command1 do statements2 doneand the syntax of the
until loop is as follows:
until command1 do statements2 doneThe
while executes statements2 as long as the return value of command1 indicates success. until executes statements2 as long as the return value of command1 indicates failure.
A one-liner example:
X=7; until [ $X -eq 1 ]; do [ $((X%2)) -eq 1 ] && X=$((3*X+1)) || X=$((X/2)); echo $X; done
Exercise 29 Write a loop that outputs all powers of two up to a given number.
Exercise 30
Write a script that will create a file myProg_1.log if it does not exist,
or if such file exists, a file myProg_2.log, or if such file also exists, a
file myProg_3.log, and so on. Put in the newly created file the current date.
To define a function
in the shell, one has to use the following syntax: name() body,
where the body shall be a compound command such as a list of statements
contained within a curly braces (e.g., mkcd() { mkdir -p $1 && cd $1; }).
To call a function, one has just to issue name [argument]... (e.g.,
mkcd /tmp/dir).
Inside a function the variables $# and $1, $2, … refer to the
arguments.
The command return [n] issued inside a function terminates the function
returning n (when specified) or the return value of the last executed command.
Consider the following example:
user@host ~ $ colorEcho() { > case "$1" in > red) CCODE=31;; > green) CCODE=32;; > blue) CCODE=34;; > *) CCODE=5;; > esac > echo -ne "\033[${CCODE}m"; > echo -n "$2"; > echo -e "\033[0m"; > } user@host ~ $ colorEcho red riding hood riding user@host ~ $ colorEcho green "anne of green gables" anne of green gables
Exercise 31
Write a function that displays Hello world!
Exercise 32 Write a function that takes two arguments: a number and a text. The function shall output the text the number of times.
Exercise 33
Write a function that resembles the tree program. It should traverse the
filesystem depth first and output for each file its name with appropriate
indentation.
The command trap "statements" CONDITION
is used to set up either handing a signal (when the CONDITION is a name of a
signal) or set the statements to be executed upon terminating the shell (when the
CONDITION is EXIT).
Shell runs the commands associated with a signal once the current foreground
command terminates or suspends.
Notice that the commands to be executed upon a signal / upon exit must be
provided as text.
The trap - CONDITION syntax restores the default handler.
Exercise 34
Handle the INT signal in the current shell by running the fortune command.
Trigger a SIGINT and see what happens.
Exercise 35 Create a script with the following contents:
TEMPFILE=$(mktemp) echo "Created a temporary file $TEMPFILE" date > "$TEMPFILE" sleep 12s date >> "$TEMPFILE" echo "Cleaning up the temporary file $TEMPFILE" rm "$TEMPFILE"
Ctrl+c, or Ctrl+\, or pkill -f scriptname.Most of the shells read certain configuration files upon start. There is no standard name for a user shell configuration file.
For all interactive shells, the Bash shell executes the commands from ~/.bashrc file
upon start.
Therefore, a Bash user that wants to permanently define aliases, functions,
set environment variables etc., shall add appropriate commands to the
~/.bashrc file.
The commands in .bashrc must output no text to standard streams when the
shell starts in non-interactive mode.