Jan Kończak

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Introduction =====

<html><small></html>

====  A bit of history ====

Terminal and console were a piece of hardware that let you interface with a computer.

  * [[https://www.merriam-webster.com/dictionary/terminal|terminal]]: a combination of a keyboard and output device (such as a video display unit) by which data can be entered into or output from a computer or electronic communications system
  * [[https://www.merriam-webster.com/dictionary/console|console]]: a combination of readouts or displays and an input device (such as a keyboard or switches) by which an operator can monitor and interact with a system (such as a computer or dubber)

https://thevaluable.dev/guide-terminal-shell-console/

'Terminal' and 'console' are nowadays used to name any (way of accessing a) text interface (of the operating system).

<html></small></html>

==== Unix-like operating systems ====

https://en.wikipedia.org/wiki/Unix-like

POSIX - a standard (published by IEEE and The Open Group) for operating system interface and environment, predominantly built to unify Unix-like systems

<html><small></html>

GNU/Linux
  * [[https://www.gnu.org/gnu/gnu.html|GNU]] is a collection of free software that altogether forms an operating system supplied with a broad choice of user applications [[https://www.gnu.org/manual/blurbs.html]]
  * the kernel of GNU, [[https://www.gnu.org/software/hurd/|Hurd]], has never been fit enough for the task
  * therefore GNU software is usually run on top of Linux kernel 
  * most Linux distributions, apart form GNU software, has also a multitude of other software (not necessarely free or open source)

<html></small></html>

==== Shell and vital utilities ====

Shell is a program that provides a (text) interface to an operating system.

There is a bunch of such programs https://en.wikipedia.org/wiki/Comparison_of_command_shells <html><small></html>[[https://wiki.archlinux.org/title/Command-line_shell|[1]]]<html></small></html>

<html><small></html>

Shells for Unix-like systems include at least ''sh'', ''csh'', ''tcsh'', ''ksh'', ''bash'', ''zsh'', ''dash'', ''busybox'', ''fish'' \\


Windows provides ''cmd.exe'' and ''PowerShell'' shells.

<html></small></html>

A shell reads commands from input and executes them. An example of a command is a program name followed by a list of arguments (separated by space).
\\
Apart from a shell, to control the operating system, one needs also a set of programs that realize basic tasks. Such programs are sometimes called utilities.

As well as there are multiple implementations of a shell, so there are multiple implementations of the basic utilities.

<html><small></html>

  * GNU/Linux systems usually use [[https://en.wikipedia.org/wiki/GNU_Core_Utilities|GNU core utilities]] and [[https://en.wikipedia.org/wiki/Util-linux|Linux utilities]]
  * .*BSD systems ship with their own implementations [[https://cvsweb.openbsd.org/src/bin/|OpenBSD]] [[https://svnweb.freebsd.org/base/head/bin/|FreeBSD]]
  * Apple operating systems (e.g. MacOS) ship with modified BSD tools [[https://opensource.apple.com/releases/|[1]]] [[https://github.com/apple-oss-distributions/file_cmds|[2]]] [[https://github.com/apple-oss-distributions/text_cmds|[3]]]
  * Embedded / resource restrained systems often use [[https://www.busybox.net/downloads/BusyBox.html|BusyBox]]
  * Android uses [[https://www.landley.net/toybox/|toybox]] [[https://cs.android.com/android/platform/superproject/+/master:external/toybox/toys/posix/|[4]]]
  * For more, see [[https://wiki.archlinux.org/title/core_utilities]]
<html></small></html>

POSIX standarizes both the [[https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html|shell]]
and its vital [[https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html|utilities]].

==== Accessing the shell ====

=== Locally ===

When one uses a graphical user interface, to access a shell one can run a [[https://en.wikipedia.org/wiki/Terminal_emulator|terminal emulator]].

<html><small></html>

There are numerous terminal emulators (consult a [[https://en.wikipedia.org/wiki/List_of_terminal_emulators|list]] on Wikipedia). A choice of terminal emulators:
  * a widespread legacy [[https://en.wikipedia.org/wiki/Xterm|xterm]]
  * terminals bundled with some Linux desktop environment: [[https://docs.xfce.org/apps/terminal/start|xfce4-terminal]], [[https://en.wikipedia.org/wiki/GNOME_Terminal|GNOME Terminal]], [[https://konsole.kde.org/|konsole]]
  * default MacOS [[https://en.wikipedia.org/wiki/Terminal_(macOS)|Terminal]]
  * drop-down terminals: [[https://apps.kde.org/yakuake/|yakuake]] / [[http://guake-project.org/|guake]] / [[https://github.com/lanoxx/tilda|tilda]]

<html></small></html>

Linux typically starts a [[https://en.wikipedia.org/wiki/Getty_(Unix)|program]] that enables logging into shell in several [[https://en.wikipedia.org/wiki/Virtual_console|virtual consoles]] alongside a graphical interface.
\\
To switch to a virtual console one has to use //Ctrl + Alt + Fn// shortcut, where //Fn// is a function key (//F1//÷//F12//).
\\
Typically numbers 1÷6 are text terminals, 7 stands for first graphical console, and some systems are configured to output system log messages to virtual console 12

=== Remotely ===

A standard way to work with remote UNIX-like systems is to log into a shell of the system using a SSH (Secure SHell) client.
\\
The basic command to log into a shell on a remote system is:
\\
''ssh user@host''
\\
where ''user'' is the user name and ''host'' is the name or address of the target device.
\\
For instance, if used ''john'' would like to log into a computer named ''polluks.put.poznan.pl'', he would have to input \\ ''ssh john@polluks.put.poznan.pl''.

Currently all major operating systems (including Windows) provide ''ssh'' command by default.

<html><small></html>

Secure SHell encrypts all traffic. Its predecessor – telnet – sends all data (including passwords) in plaintext.

<html></small></html>

~~Exercise.#~~ Run a terminal emulator. Execute command ''date'' and ''echo $SHELL'' within. 

~~Exercise.#~~ Switch to second virtual console and log in. Execute ''pwgen''. Return to graphical console. 

~~Exercise.#~~ Use SSH to access ''polluks.cs.put.poznan.pl''.

===== Getting help =====

==== System manual and 'help' switch ====

Most commands will display help when run with ''--help'' argument. Some commands also use ''-h'' for this.

~~Exercise.#~~ Display help for ''cat'' and ''bat'' commands by running them with help option. <html><small></html>(NB: ''bat'' is installed by default only in some Linux distros.)<html></small></html>

Usually together with a program comes with a set of system manual pages. The pages can be displayed using the ''man //pagename//'' command.

~~Exercise.#~~ Display manual page for ''timeout'' command.


Manual pages are organized into sections. Use ''man man'' to learn about the standard sections.
\\
Page names are unique within a section, but there may be pages with the same name in different sections.
\\
You may use the ''whatis //pagename//'' command to learn which sections contain given page name and display the full page title.

~~Exercise.#~~ List all pages that are named ''time''.

To select a page from a given section, one has to use ''man //section// //pagename//'' syntax. 

~~Exercise.#~~ The ''man'' itself is described on two manual pages. Display both. What information is contained in each one?

''man'' uses another tool, called ''less'', to actually display the manual pages. ''less'' will be discussed in details later.\\
To search for text in ''less'', one can type ''/'' followed by the search term and //enter//. Next / previous  occurrence of the search term can be found by typing ''n'' / ''N''. To jump to the beginning of the document press ''g''. ''h'' displays a summary of ''less'' commands.

~~Exercise.#~~ Display manual page for ''bash'' and search for ''printf'' there. Then, search for explanation of ''dirs'' command.

<html><small></html>

For searching within manual there is also the ''apropos //what//'' command that searches within page names and titles.

Apart from ''man'', there is an alternative (and less popular) documentation system called ''info''. To see an Info page, type ''info //name//''. Some programs ship with a more detailed Info document than manual page.

<html></small></html>

==== Command syntax notation ====

Manuals, documentation and examples for commands traditionally use a set of square and curly brackets to indicate (respectively) optional parts and choice of one of alternatives.
<html><small></html>Sometimes arbitrary brackets are used to list alternatives: this is the case if they contain tokens separated by a ''|'' character. Angle brackets are sometimes used to name elements.<html></small></html> \\

  * ''wait [pid]'' means that both ''wait'' and ''wait 1234'' is a correct command (provided 1234 is a pid)
  * ''grep --color[=WHEN]'' means that both ''grep --color'' and ''grep --color=always'' are correct
  * ''tar {c|x}'' means that one must write either ''tar c'' or ''tar x''

Ellipsis is used to indicate that a part of command may repeat, e.g. ''cat [FILE]...'' says that ''cat'' accepts multiple filenames.

Many programs accept //options// (called also //switches//) in the argument list.
\\
The options are often written as short options in form ''-o'' where ''o'' is a single character and long options ''--option'' where ''option'' is the option name.
\\
<html><small></html>Some programs use only long options preceded with single dash (e.g., ''find'', ''convert'', ''ffmpeg'').<html></small></html>
\\
Short options can usually be combined in the following way: ''ls -color'' is identical to ''ls -c -o -l -o -r'' (and different than ''ls --color'')

Options may require an argument or may accept an optional argument.

Apart form command name, options (and keywords, if any) all other text is a placeholder for concrete values.
\\
For instance, in ''find -amin N'' and ''find -anewer FILE'' the ''N'' and ''FILE'' are such placeholders: ''N'' must be replaced by number of minutes, and ''FILE'' must be replaced by a name of a file.

~~Exercise.#~~ Run: \\ ''man zip'' and review the SYNOPSIS, \\ ''more --help'' and explain its syntax \\ ''taskset -h'' and explain its syntax 

==== Autocompletion ====

Almost any shell supports autocompletion. Autocompletion is invoked by pressing //TAB// key.\\
Autocompletion is supposed to type the rest of a word for you, and does so if only one possibility is found.\\
If autocompletion recognizes multiple possibilities, it does nothing. If one presses //TAB// key again, most autocompletion procedures display all possibilities.

Autocompletion implementation differs across shells. Many shells support context-aware helpers.

~~Exercise.#~~ Type ''bad'' and press //TAB//. Type ''ba'' and press //TAB// twice.

~~Exercise.#~~ Type ''man --'' and press //TAB// twice. Type ''man fs'' and press //TAB// twice.

===== Browsing the filesystem =====

==== File tree ====

In Unix-like systems all files and directories are organized in a tree, i.e., any two files share the same top-level directory called [[https://en.wikipedia.org/wiki/Root_directory|root directory]]. The path to root directory is ''/''. \\
Filesystems other than the root one (e.g., other partitions, compact disks, flash drives), are //mounted// inside a selected directory located within ''/'' (rather than being a separate file hierarchy).

A path is a chain of names separated by ''/''. \\
''.'' stands for current directory and ''..'' stands for parent directory. \\

A path that starts with ''/'' is called absolute. A path that is not absolute is called relative.

Each user has a [[https://en.wikipedia.org/wiki/Home_directory|home directory]]. Shell (and some other programs) replaces the word ''~'' with home directory of the current user. <html><small></html>''~user'' stands for home directory of ''user'', so ''~/x/a.txt'' is within home directory of current user, and ''~x/b.txt'' is within home directory of user ''x''.<html></small></html>


==== Listing directory contents ====

The command ''ls [//dir//]'' lists file in directory ''dir''. When run with no arguments, it lists the current working directory.
\\
By default ''ls'' does not list hidden files. The ''-a'' and ''--all'' switch changes this behaviour.
\\
Switch ''-l'' / ''--list'' prints list of files with details such as file owner, size, modification date, etc.

~~Exercise.#~~ List files in ''/usr/share/zoneinfo/'' and ''../../../../../bin''. 

~~Exercise.#~~ List all files in your home directory.

<html><small></html>

~~Exercise.#~~ Test the following switches with ''ls'' command:  ''-A''   ''-d''   ''-s''   ''-h''   ''-t''   ''-1''

Apart from ''ls'' command, there are standard ''dir'' and ''vdir'' commands that list files.
\\
Shell is often configured to use colors in ''ls'' by default and understand ''ll'' as ''ls -l'', and ''la'' as ''ls -l -a''.

The ''tree'' command outputs recursively contents of the directory in a tree form.

<html></small></html>

==== Changing working directory ====

To print the working directory one can execute the ''pwd'' command.

~~Exercise.#~~ Check in which directory you end up upon logging into the system.

To change directory the command ''cd [//target//]'' is used. If no target is provided, ''cd'' changes current directory to the home directory.
\\
<html><small></html>If ''-'' is uses as the target, the directory is changed to the previously visited. <html></small></html>

~~Exercise.#~~ Enter ''.config'' directory (located in your home directory) and list files within.

~~Exercise.#~~ Navigate to ''/var/log''. What is stored there?

<html><small></html>

~~Exercise.#~~ Go back to where you've been before with ''cd -''

To change directory and save the previous one can use ''pushd //dir//'' command. \\
''pushd //dir//'' pushes the previous directory on a directory stack. The directory stack can be displayed with ''dirs'' command. \\
To change back one can either execute ''popd'' (that removes a directory from the stack and enters it) or ''pushd'' (that swaps current directory with the directory on the stack). ''pushd'' and ''popd'' accept arguments ''+n''/''-n'' that select directory on the stack.

<html></small></html>

==== Glob names and wildcards ====

Upon parsing the user input, the shell attempts to replace text containing wildcard characters ''*'', ''?'' and expressions in square brackets ''[…]''. Any text that contains such wildcards is called [[https://en.wikipedia.org/wiki/Glob_(programming)|glob]].

''*'' expands to any text, ''?'' expand to single character, ''[ace]'' expands to either ''a'', or ''c'', or ''e'', and ''[f-h]'' expands to either ''f'', or ''g'', or ''h''.

If there is at least one file that matches the glob, the shell replaces the glob with all filenames that match. If no file matches, then the glob is left unaltered.



~~Exercise.#~~ Change directory to ''/usr/bin/'' directory. List all files that end with ''cat''.

~~Exercise.#~~ List all files that end with ''cat'' in ''/usr/bin/'' while being in your home directory.

~~Exercise.#~~ List all files that are in the ''/usr/bin/'' directory that have names consisting of two chatacters.

~~Exercise.#~~ List contents of ''/usr/share/zoneinfo/'' and its subdirectories using a single command.

~~Exercise.#~~ List all files named ''UTC'' that are in any subdirectory of ''/usr/share/zoneinfo/''.

~~Exercise.#~~ List all files starting with ''L'' that are in any subdirectory of ''/usr/share/zoneinfo/''.

<html><small></html>

~~Exercise.#~~ List all files starting with ''F'' or ''H''  that are in any subdirectory of ''/usr/share/zoneinfo/''. Use a single glob.

~~Exercise.#~~ Navigate to ''/usr/bin'' and attempt to execute ''ls*''. What has happened?

<html></small></html>


==== Copying, moving and erasing files ====

<html><small></html>

To create an empty file, you can use ''touch //file//''. To create file with some contents, you can use ''echo //contents// > //file//''. The ''touch'' and ''echo'' commands as well as meaning of ''>'' will be explained later on.

To display a file, use ''cat //filename//''.

<html></small></html>

=== Directories ===

''mkdir //dir//'' creates a new directory.
\\
<html><small></html>
''mkdir -p //dir1/dir2/dir3//'' creates directory ''dir1'', and directory ''dir2'' within ''dir1'', and ''dir3'' within ''dir1/dir2''.
With the ''-p'' switch ''mkdir'' does not print an error if a directory already exists.
<html></small></html>

To remove an empty directory one can use ''rmdir //dir//''.

~~Exercise.#~~ Create the following directory tree:
  .
  ├── foo
  │   ├── baz
  │   │   └── bar
  │   └── bar
  └── foobaz
      └── bar
 
~~Exercise.#~~ Remove ''foo/baz'' directory

=== Hard and soft links ===

Multiple directory entries can point to the same file. This is called a [[https://en.wikipedia.org/wiki/Hard_link|hard link]]. \\
A directory entry can also point to another directory entry.  This is called a [[https://en.wikipedia.org/wiki/Symbolic_link|soft link or symbolic link]] (usually abbreviated as symlink).

All Linux filesystems support hard links for ordinary files, and soft links to arbitrary path (e.g., a file or a directory).

To create a hard link, one can use ''ln //source// //destination//''.\\
To create a symlink, one can use ''ln -s //source// //destination//''.

~~Exercise.#~~ Create file ''file1'' with some contents. Create a hard link called ''file2'' of the file ''file1''. Modify ''file2''. Display ''file1''.

<html><small></html>

~~Exercise.#~~ The ''ls'' command can display for each file its link count. Discover how to do it.

The disk space used by a file is reclamied once link count drops to 0 (all directory entries that link to the file are erased) and the file is no longer open in any process.

<html></small></html>

~~Exercise.#~~ Create in your home directory a symlink called ''TMP'' pointing to ''/tmp''. Change directory to ''TMP''. What does ''pwd'' output?

~~Exercise.#~~ Create in your home directory a symlink called ''loop'' that points to your home directory. Enter it. And enter it again.

~~Exercise.#~~ Create a symlink to a non-existent path. List the directory containing it.

<html><small></html>

The ''readlink [-f] //target//'' command resolves all symlinks and prints a [[https://en.wikipedia.org/wiki/Canonicalization|canonical]] path.

The Windows NTFS filesystem supports links as well. Creating the links is possible e.g., with [[https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/mklink|mklink]] command.

<html></small></html>

=== Copying and moving files ===

To copy files, one can use the ''cp'' command. To move (or rename) file, one can use the ''mv'' command.

The basic syntax is ''cp/mv //source//… //destination//''. \\
Multiple source files can be provided if the destination is a directory. \\
If the destination is a file, it will be overwritten without warning <html><small></html>(unless ''-i'' or ''-n'' switch is used)<html></small></html>.

<html><small></html>

By default ''cp'' will deny copying a directory. Use ''-r'' to copy a directory recursively.

<html></small></html>

When ''cp'' copies a file, it creates a new file with current date, default permissions, etc.
\\
To copy recursively, and preserve dates, permissions and more, one can use the ''-a'' switch (that stands for ''--archive'').

<html><small></html>

With the ''-l'' switch ''cp'' creates a hard link instead of copying a file. Notice that this can be combined with ''--recursive''.

<html></small></html>

~~Exercise.#~~ Run ''mkdir someDir && for F in file1 file2 file3; do echo $RANDOM$RANDOM > someDir/$F; done'' to create ''someDir'' directory with three files inside.

~~Exercise.#~~ Copy the ''someDir'' directory recursively under a new name.

~~Exercise.#~~ Move the newly copied directory into the ''someDir'' directory.

~~Exercise.#~~ Rename ''someDir'' to a name of your choice.

<html><small></html>

~~Exercise.#~~ Copy the renamed directory with ''-al'' switches. Modify a selected file with any of the directories. Which files changed contents? <html><small></html> (You can ''cat //filename//'' or display modification dates with second accuracy using ''ls -l --time-style=+%H:%M:%S …'' or ''tree -D --timefmt=%H:%M:%S …'')<html></small></html>

<html></small></html>

=== Copying files to/from a remote machine ===

It is possible to copy files via SSH. Whenever one has SSH access to a remote machine, one can copy files with ''scp'' command. \\
''scp'' accepts ''//file//'' as file path on local machine and ''//user//@//host//://file//'' as a file path on a remote machine. Remote relative paths are relative to home directory.\\
''scp'' accepts the ''-r'' switch for copying recursively.

<html><small></html>

Microsoft Windows now ships with ''scp'' command, but usually [[https://en.wikipedia.org/wiki/Comparison_of_FTP_client_software#Protocol_support|various file commanders]] are more convenient.

<html></small></html>

Most SSH servers enable also the SFTP protocol that allows copying files more conveniently.\\
''sftp //user//@//host//'' launches sftp command line. You can use ''ls'' and ''cd'' to navigate the remote filesystem, and ''get'' and ''put'' to copy files. Type ''help'' to see all supported commands.

~~Exercise.#~~ Create a file in ''/tmp'' directory in your computer. Copy the file to home directory of user ''student'' on another computer.

~~Exercise.#~~ Copy the file to ''/tmp'' directory on another computer.

~~Exercise.#~~ Copy the file to ''/tmp'' directory on another computer using ''sftp''.

<html><small></html>
=== Rsync ===

The ''[[https://en.wikipedia.org/wiki/Rsync|rsync]]'' program is widely used to copy files and directories. It efficiently compares source files with destination files and copies only the differences. It can copy data to/from remote machines, and can compress the data sent via network to increase throughput. ''rsync'' is also commonly used to make backups.

<html></small></html>

=== Removing files ===

The program that removes files is called ''rm''.

''rm'' by default won't remove directories (regardless if empty) and write-protected files.

To remove a directory with ''rm'' (recursively with their contents) one has to add a ''-r'' switch.

To remove write-protected files (and stop printing warnings whenever a file to be removed already does not exist) the switch ''-f'' (''--force'') can be used.

A misused ''rm -rf …'' command is a notorious source of data loss. Beware especially of asterisk and what it expands to.

''rm'' accepts ''-I'' and ''-i'' switches that ask for confirmation. ''-I'' asks once upon attempt to remove multiple files, and ''-i'' asks upon each file.

~~Exercise.#~~ Create multiple files with ''touch file_{a..z}''. \\ Remove all files with a single ''rm'' command with 1) no switches 2) ''-f'' switch 3) ''-I'' switch 4) ''-i'' switch.

~~Exercise.#~~ Repeat the remove command from the previous exercise when the files are gone. \\ Then repeat the command again with ''-f'' switch.

~~Exercise.#~~ Create a directory and remove it with ''rm''.



~~META:
language = en
~~
Jan Kończak

User Tools

Site Tools

Page Tools
