Jan Kończak

User Tools

Site Tools

You are here: start » os_cp » intro
os_cp:intro

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
os_cp:intro [2023/03/01 12:33] – [Command syntax notation] jkonczakos_cp:intro [2026/03/11 23:57] (current) – [Glob names and wildcards] jkonczak
Line 22: Line 22:
 POSIX - a standard (published by IEEE and The Open Group) for operating system interface and environment, predominantly built to unify Unix-like systems POSIX - a standard (published by IEEE and The Open Group) for operating system interface and environment, predominantly built to unify Unix-like systems
  
-<html><small></html>+<small>
  
-GNU/Linux+Linux, GNU/Linux, Linux distribution…
   * [[https://www.gnu.org/gnu/gnu.html|GNU]] is a collection of free software that altogether forms an operating system supplied with a broad choice of user applications [[https://www.gnu.org/manual/blurbs.html]]   * [[https://www.gnu.org/gnu/gnu.html|GNU]] is a collection of free software that altogether forms an operating system supplied with a broad choice of user applications [[https://www.gnu.org/manual/blurbs.html]]
   * the kernel of GNU, [[https://www.gnu.org/software/hurd/|Hurd]], has never been fit enough for the task   * the kernel of GNU, [[https://www.gnu.org/software/hurd/|Hurd]], has never been fit enough for the task
   * therefore GNU software is usually run on top of Linux kernel    * therefore GNU software is usually run on top of Linux kernel 
-  * most Linux distributions, apart form GNU software, has also a multitude of other software (not necessarely free or open source)+  * most Linux distributions, apart form GNU software, has also a multitude of other software (not necessarily free or open source) 
 +  * some Linux distributions base on free and/or open source software that is neither GNU software nor uses GNU licenses
  
-<html></small></html>+</small>
  
 ==== Shell and vital utilities ==== ==== Shell and vital utilities ====
Line 59: Line 60:
   * Apple operating systems (e.g. MacOS) ship with modified BSD tools [[https://opensource.apple.com/releases/|[1]]] [[https://github.com/apple-oss-distributions/file_cmds|[2]]] [[https://github.com/apple-oss-distributions/text_cmds|[3]]]   * Apple operating systems (e.g. MacOS) ship with modified BSD tools [[https://opensource.apple.com/releases/|[1]]] [[https://github.com/apple-oss-distributions/file_cmds|[2]]] [[https://github.com/apple-oss-distributions/text_cmds|[3]]]
   * Embedded / resource restrained systems often use [[https://www.busybox.net/downloads/BusyBox.html|BusyBox]]   * Embedded / resource restrained systems often use [[https://www.busybox.net/downloads/BusyBox.html|BusyBox]]
-  * Android uses [[https://www.landley.net/toybox/|toybox]] [[https://cs.android.com/android/platform/superproject/+/master:external/toybox/toys/posix/|[4]]]+  * Android uses [[https://www.landley.net/toybox/|toybox]] [[https://cs.android.com/android/platform/superproject/main/+/main:external/toybox/toys/posix/|[4]]]
   * For more, see [[https://wiki.archlinux.org/title/core_utilities]]   * For more, see [[https://wiki.archlinux.org/title/core_utilities]]
 <html></small></html> <html></small></html>
  
-POSIX standarizes both the [[https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html|shell]] +POSIX standarizes both the [[https://pubs.opengroup.org/onlinepubs/9799919799/utilities/V3_chap02.html|shell]] 
-and its vital [[https://pubs.opengroup.org/onlinepubs/9699919799/utilities/contents.html|utilities]].+and its vital [[https://pubs.opengroup.org/onlinepubs/9799919799/utilities/contents.html|utilities]].
  
 ==== Accessing the shell ==== ==== Accessing the shell ====
Line 78: Line 79:
   * terminals bundled with some Linux desktop environment: [[https://docs.xfce.org/apps/terminal/start|xfce4-terminal]], [[https://en.wikipedia.org/wiki/GNOME_Terminal|GNOME Terminal]], [[https://konsole.kde.org/|konsole]]   * terminals bundled with some Linux desktop environment: [[https://docs.xfce.org/apps/terminal/start|xfce4-terminal]], [[https://en.wikipedia.org/wiki/GNOME_Terminal|GNOME Terminal]], [[https://konsole.kde.org/|konsole]]
   * default MacOS [[https://en.wikipedia.org/wiki/Terminal_(macOS)|Terminal]]   * default MacOS [[https://en.wikipedia.org/wiki/Terminal_(macOS)|Terminal]]
-  * drop-down terminals: [[https://apps.kde.org/yakuake/|yakuake]] / [[http://guake-project.org/|guake]] / [[https://github.com/lanoxx/tilda|tilda]]+  * drop-down terminals: [[https://apps.kde.org/yakuake/|yakuake]] / [[https://guake.github.io/|guake]] / [[https://github.com/lanoxx/tilda|tilda]]
  
 <html></small></html> <html></small></html>
Line 98: Line 99:
 where ''user'' is the user name and ''host'' is the name or address of the target device. where ''user'' is the user name and ''host'' is the name or address of the target device.
 \\ \\
-For instance, if used ''john'' would like to log into a computer named ''polluks.cs.put.poznan.pl'', he would have to input \\ ''ssh john@polluks.cs.put.poznan.pl''.+For instance, if the user ''john'' would like to log into a computer named ''polluks.cs.put.poznan.pl'', he would have to input \\ ''ssh john@polluks.cs.put.poznan.pl''. 
 +\\ 
 +To terminate a ssh session, one can use either ''logout'' or ''exit'' command, or use //Ctrl// + //d// key combination on empty prompt line((//Ctrl// + //d// actually indicates //end-of-file// condition.)).
  
 Currently all major operating systems (including Windows) provide ''ssh'' command by default. Currently all major operating systems (including Windows) provide ''ssh'' command by default.
  
-<html><small></html>+<small>
  
 Secure SHell encrypts all traffic. Its predecessor – telnet – sends all data (including passwords) in plaintext. Secure SHell encrypts all traffic. Its predecessor – telnet – sends all data (including passwords) in plaintext.
  
-<html></small></html>+Upon password authentication, the client must send the password to the server. 
 +\\ 
 +It is vital for security to confirm the authenticity of the server – the client 
 +must verify that it does not contact a rogue server that can learn the password 
 +([[https://docs.ssh-mitm.at/user_guide/authentication.html#id2|ready-to-go tool]]) 
 +and/or hijack the session. 
 +\\ 
 +[[https://en.wikipedia.org/wiki/Public-key_cryptography|Asymmetric cryptography]] 
 +is used to this end: the server has a (secret) private key, and upon each 
 +connection it sends the public key to the client, and the client verifies whether 
 +the public key pairs with the private. 
 +\\  
 +Hence, upon the first connection to a new server the SSH client program requests 
 +the  user to verify the authenticity of the public key and stores the key (in 
 +OpenSSH in ''~/.ssh/known_hosts'') to verify it automatically upon subsequent 
 +connections. 
 + 
 +</small>
  
 ~~Exercise.#~~ Run a terminal emulator. Execute command ''date'' and ''echo $SHELL'' within.  ~~Exercise.#~~ Run a terminal emulator. Execute command ''date'' and ''echo $SHELL'' within. 
  
-~~Exercise.#~~ Execute command ''​sleep 1h''​ and interrupt it by pressing //Ctrl + c//.+~~Exercise.#~~ Execute command ''sleep 1h'' and interrupt it by pressing //Ctrl + c//.
  
 ~~Exercise.#~~ Switch to second virtual console and log in. Execute ''pwgen''. Return to graphical console.  ~~Exercise.#~~ Switch to second virtual console and log in. Execute ''pwgen''. Return to graphical console. 
  
-~~Exercise.#~~ Use SSH to access ''polluks.cs.put.poznan.pl''.+~~Exercise.#4~~a Use SSH to access your neighbour's computer. 
 +\\ 
 +<small>~~Exercise.#4~~b Use SSH to access ''polluks.cs.put.poznan.pl''.</small>
  
 ===== Getting help ===== ===== Getting help =====
Line 124: Line 146:
 ~~Exercise.#~~ Display help for ''cat'' and ''bat'' commands by running them with help option. <html><small></html>(NB: ''bat'' is installed by default only in some Linux distros.)<html></small></html> ~~Exercise.#~~ Display help for ''cat'' and ''bat'' commands by running them with help option. <html><small></html>(NB: ''bat'' is installed by default only in some Linux distros.)<html></small></html>
  
-Usually together with a program comes with a set of system manual pages. The pages can be displayed using the **''man //pagename//''** command. To leave the manual, type ''q''.+Usually a program comes together with a set of system manual pages. The pages can be displayed using the **''man //pagename//''** command. To leave the manual, type ''q''.
  
 ~~Exercise.#~~ Display manual page for ''timeout'' command. ~~Exercise.#~~ Display manual page for ''timeout'' command.
Line 132: Line 154:
 Page names are unique within a section, but there may be pages with the same name in different sections. Page names are unique within a section, but there may be pages with the same name in different sections.
 \\ \\
-You may use the **''whatis //pagename//''** command to learn which sections contain given page name and display the full page title.+You may type **''whatis //pagename//''** or <small>''man -f //pagename//''</small> to learn which sections contain given page name and display the full page title.
  
 ~~Exercise.#~~ List all pages that are named ''time''. ~~Exercise.#~~ List all pages that are named ''time''.
  
-To select a page from a given section, one has to use ''man //section// //pagename//'' syntax. +To select a page from a given section, one has to use ''man //section// //pagename//'' syntax, e.g., ''man 2 write''.
  
 ~~Exercise.#~~ The ''man'' itself is described on two manual pages. Display both. What information is contained in each one? ~~Exercise.#~~ The ''man'' itself is described on two manual pages. Display both. What information is contained in each one?
Line 147: Line 169:
 <html><small></html> <html><small></html>
  
-For searching within manual there is also the ''apropos //what//'' command that searches within page names and titles.+For searching within manual there are also the ''apropos //what//'' and ''man -k //what//'' commands that search within page names and titles.
  
 Apart from ''man'', there is an alternative (and less popular) documentation system called ''info''. To see an Info page, type ''info //name//''. Some programs ship with a more detailed Info document than manual page. Apart from ''man'', there is an alternative (and less popular) documentation system called ''info''. To see an Info page, type ''info //name//''. Some programs ship with a more detailed Info document than manual page.
Line 213: Line 235:
 The command **''ls [//dir//]''** lists file in directory ''dir''. When run with no arguments, it lists the current working directory. The command **''ls [//dir//]''** lists file in directory ''dir''. When run with no arguments, it lists the current working directory.
 \\ \\
-By default ''ls'' does not list hidden files. The **''-a''** and ''--all'' switch changes this behaviour.+By default ''ls'' does not list hidden files. The **''-a''** switch changes this behaviour.
 \\ \\
-Switch **''-l''** / ''--list'' prints list of files with details such as file owner, size, modification date, etc.+Switch **''-l''** prints list of files with details such as file owner, size, modification date, etc.
  
 ~~Exercise.#~~ List files in ''/usr/share/zoneinfo/'' and ''../../../../../bin''.  ~~Exercise.#~~ List files in ''/usr/share/zoneinfo/'' and ''../../../../../bin''. 
Line 241: Line 263:
 To change directory the command **''cd [//target//]''** is used. If no target is provided, ''cd'' changes current directory to the home directory. To change directory the command **''cd [//target//]''** is used. If no target is provided, ''cd'' changes current directory to the home directory.
 \\ \\
-<html><small></html>If ''-'' is uses as the target, the directory is changed to the previously visited. <html></small></html>+<html><small></html>If ''-'' is used as the target, the directory is changed to the previously visited. <html></small></html>
  
 ~~Exercise.#~~ Enter ''.config'' directory (located in your home directory) and list files within. ~~Exercise.#~~ Enter ''.config'' directory (located in your home directory) and list files within.
Line 257: Line 279:
 <html></small></html> <html></small></html>
  
-==== Glob names and wildcards ==== +==== Wildcard, regex, … ====
- +
-Upon parsing the user input, the shell attempts to replace any word containing wildcard characters **''*''**, **''?''** and expressions in square brackets **''[…]''**. A word that contains such wildcards is called [[https://en.wikipedia.org/wiki/Glob_(programming)|glob]]. +
- +
-''*'' expands to any text, ''?'' expand to single character, ''[ace]'' expands to either ''a'', or ''c'', or ''e'', and ''[f-h]'' expands to either ''f'', or ''g'', or ''h''. +
- +
-If there is at least one file that matches the glob, the shell replaces the glob with all filenames that match. If no file matches, then the glob is left unaltered. +
- +
- +
- +
-~~Exercise.#~~ Change directory to ''/usr/bin/'' directory. List all files that end with ''cat''. +
- +
-~~Exercise.#~~ List all files that end with ''cat'' in ''/usr/bin/'' while being in your home directory. +
- +
-~~Exercise.#~~ List all files that are in the ''/usr/bin/'' directory that have names consisting of two chatacters. +
- +
-~~Exercise.#~~ List contents of ''/usr/share/zoneinfo/'' and its subdirectories using a single command. +
- +
-~~Exercise.#~~ List all files named ''UTC'' that are in any subdirectory of ''/usr/share/zoneinfo/''. +
- +
-~~Exercise.#~~ List all files starting with ''L'' that are in any subdirectory of ''/usr/share/zoneinfo/''. +
- +
-<html><small></html> +
- +
-~~Exercise.#~~ List all files starting with ''F'' or ''H''  that are in any subdirectory of ''/usr/share/zoneinfo/''. Use a single glob. +
- +
-~~Exercise.#~~ Navigate to ''/usr/bin'' and attempt to execute ''ls*''. What has happened? +
- +
-<html></small></html> +
- +
- +
-==== Copying, moving and erasing files ==== +
- +
-<html><small></html> +
- +
-To create an empty file, you can use ''touch //file//''. To create file with some contents, you can use ''echo //contents// > //file//''. The ''touch'' and ''echo'' commands as well as meaning of ''>'' will be explained later on. +
- +
-To display a file, use ''cat //filename//''. +
- +
-<html></small></html>+
  
-=== Directories ===+<small>
  
-**''mkdir //dir//''** creates a new directory.+In computer science patterns/expressions that describe the expected data can be 
 +expressed and used to match against some data, usually to find fitting data or 
 +check if the data matches the description.
 \\ \\
-<html><small></html> +For instance, //lines beginning with a word followed by an equals sign// might 
-''mkdir -p //dir1/dir2/dir3//'' creates directory ''dir1'', and directory ''dir2'' within ''dir1'', and ''dir3'' within ''dir1/dir2''. +be written as a regular expression ''^\w+=''.
-With the ''-p'' switch ''mkdir'' does not print an error if a directory already exists. +
-<html></small></html>+
  
-To remove an empty directory one can use ''rmdir //dir//''.+There are countless syntaxes and implementations of such patterns/expressions.
  
-~~Exercise.#~~ Create the following directory tree: +In the shell, [[https://en.wikipedia.org/wiki/Wildcard_character|wildcards]] 
-  . +and [[https://en.wikipedia.org/wiki/Regular_expression|regular expressions]] 
-  ├── foo +are commonplace.
-  │   ├── baz +
-  │   │   └── bar +
-  │   └── bar +
-  └── foobaz +
-      └── bar +
-  +
-~~Exercise.#~~ Remove ''foo/baz'' directory +
- +
-=== Hard and soft links === +
- +
-Multiple directory entries can point to the same file. This is called a [[https://en.wikipedia.org/wiki/Hard_link|hard link]]. \\ +
-A directory entry can also point to another directory entry.  This is called a [[https://en.wikipedia.org/wiki/Symbolic_link|soft link or symbolic link]] (usually abbreviated as symlink). +
- +
-All Linux filesystems [[https://​en.wikipedia.org/​wiki/​Comparison_of_file_systems#​File_capabilities|support]] hard links for ordinary files, and soft links to arbitrary path (e.g., a file or a directory). +
- +
-To create a hard link, one can use **''ln //source// //destination//''**.\\ +
-To create a symlink, one can use **''ln -s //source// //destination//''**. +
- +
-~~Exercise.#~~ Create file ''file1'' with some contents. Create a hard link called ''file2'' of the file ''file1''. Modify ''file2''. Display ''file1''. +
- +
-<html><small></html> +
- +
-~~Exercise.#~~ The ''ls'' command can display for each file its link count. Discover how to do it. +
- +
-The disk space used by a file is reclamied once link count drops to 0 (all directory entries that link to the file are erased) and the file is no longer open in any process. +
- +
-<html></small></html> +
- +
-~~Exercise.#~~ Create in your home directory a symlink called ''TMP'' pointing to ''/tmp''. Change directory to ''TMP''. What does ''pwd'' output? +
- +
-~~Exercise.#~~ Create in your home directory a symlink called ''loop'' that points to your home directory. Enter it. And enter it again. +
- +
-~~Exercise.#~~ Create a symlink to a non-existent path. List the directory containing it. +
- +
-<html><small></html> +
- +
-The ''readlink [-f] //target//'' command resolves all symlinks and prints a [[https://en.wikipedia.org/wiki/Canonicalization|canonical]] path. +
- +
-The Windows NTFS filesystem supports links as well. Creating the links is possible e.g., with [[https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/mklink|mklink]] command. +
- +
-<html></small></html> +
- +
-=== Copying and moving files === +
- +
-To copy files, one can use the **''cp''** command. To move (or rename) file, one can use the **''mv''** command. +
- +
-The basic syntax is ''cp/mv //source//… //destination//''. \\ +
-Multiple source files can be provided if the destination is a directory. \\ +
-If the destination is a file, it will be overwritten without warning <html><small></html>(unless ''-i'' or ''-n'' switch is used)<html></small></html>. +
- +
-<html><small></html> +
- +
-By default ''cp'' will deny copying a directory. Use ''-r'' to copy a directory recursively. +
- +
-<html></small></html>+
  
-When ''cp'' copies a file, it creates a new file with current date, default permissions, etc.+Syntax and implementation of both wildcards and regexes is language/application 
 +specific.
 \\ \\
-To copy recursively, and preserve dates, permissions and more, one can use the **''-a''** switch (that stands for ''--archive'').+The shell uses syntax summarized e.g., in 
 +''[[https://man7.org/linux/man-pages/man7/glob.7.html|man 7 glob]]'' and  
 +''[[https://man7.org/linux/man-pages/man7/regex.7.html|man 7 regex]]''
  
-<html><small></html>+</small>
  
-With the ''-l'' switch ''cp'' creates a hard link instead of copying a file. Notice that this can be combined with ''--recursive''.+==== Glob names and wildcards ====
  
-<html></small></html>+Upon parsing the user input, the shell attempts to replace with filenames any 
 +word containing wildcard characters **''*''**, **''?''** and expressions in 
 +square brackets **''[…]''**. A word that contains such wildcards is called 
 +[[https://en.wikipedia.org/wiki/Glob_(programming)|glob]].
  
-~~Exercise.#~~ Copy, using a single command, the files ''/​etc/​os-release''​ and ''/​etc/​SUSE-brand''​ to the current working directory.+''*'' expands to any text\\ 
 +''?'' expand to single character\\ 
 +''[ace]'' expands to either ''a'', or ''c'', or ''e'' \\ 
 +''[f-h]'' expands to either ''f'', or ''g'', or ''h''
  
-~~Exercise.#~~ Run ''mkdir someDir && for F in file{1..3}; do echo $RANDOM$RANDOM > someDir/$F; done'' to create ''someDir'' directory with three files inside.+If there is at least one file that matches the glob, the shell replaces the glob with all filenames that match. If no file matches, then the glob is left unaltered.
  
-~~Exercise.#~~ Copy the ''someDir'' directory recursively under a new name. 
  
-~~Exercise.#~~ Move the newly copied directory into the ''someDir'' directory. 
  
-~~Exercise.#~~ Rename ''someDir'' to a name of your choice.+~~Exercise.#~~ Change directory to ''/usr/bin/'' directory. List all files that end with ''cat''.
  
-<html><small></html>+~~Exercise.#~~ Change directory to your home directory. From there, list all files that end with ''cat'' in ''/usr/bin/''.
  
-~~Exercise.#~~ Copy the renamed directory with ''-al'' switches. Modify a selected file with any of the directories. Which files changed contents? <html><small></html> (You can ''cat //filename//'' or display modification dates with second accuracy using ''ls -l --time-style=+%H:%M:%S …'' or ''tree -D --timefmt=%H:%M:%S …'')<html></small></html>+~~Exercise.#~~ List all files that are in the ''/usr/bin/'' directory that have names consisting of two characters.
  
-<html></small></html>+~~Exercise.#~~ List contents of ''/usr/share/zoneinfo/'' and its subdirectories using a single command.
  
-=== Copying files to/from a remote machine ===+~~Exercise.#~~ List all files named ''UTC'' that are in any subdirectory of ''/usr/share/zoneinfo/''.
  
-It is possible to copy files via SSH. Whenever one has SSH access to a remote machine, one can copy files with **''scp''** command. \\ +~~Exercise.#~~ List all files starting with ''L'' that are in any subdirectory of ''/usr/share/zoneinfo/''.
-''scp'' accepts ''//file//'' as file path on local machine and ''//user//@//host//://file//'' as a file path on a remote machine. Remote relative paths are relative to home directory.\\ +
-''scp'' accepts the ''-r'' switch for copying recursively.+
  
 <html><small></html> <html><small></html>
  
-Microsoft Windows now ships with ''scp'' command, but usually [[https://en.wikipedia.org/wiki/Comparison_of_FTP_client_software#Protocol_support|various file commanders]] are more convenient.+~~Exercise.#~~ List all files starting with ''F'' or ''H''  that are in any subdirectory of ''/usr/share/zoneinfo/''. Use a single glob.
  
-<html></small></html> +~~Exercise.#~~ Navigate to ''/usr/bin'' and attempt to execute ''ls*''. What has happened?
- +
-Most SSH servers enable also the SFTP protocol that allows copying files more conveniently.\\ +
-**''sftp //user//@//host//''** launches sftp command line. You can use ''ls'' and ''cd'' to navigate the remote filesystem, and ''get'' and ''put'' to copy files. Type ''help'' to see all supported commands. +
- +
-~~Exercise.#~~ Create a file in ''/tmp'' directory in your computer. Copy the file to home directory of user ''student'' on another computer. +
- +
-~~Exercise.#~~ Copy the file to ''/tmp'' directory on another computer. +
- +
-~~Exercise.#~~ Copy the file to ''/tmp'' directory on another computer using ''sftp''. +
- +
-<html><small></html> +
-=== Rsync === +
- +
-The ''[[https://en.wikipedia.org/wiki/Rsync|rsync]]'' program is widely used to copy files and directories. It efficiently compares source files with destination files and copies only the differences. It can copy data to/from remote machines, and can compress the data sent via network to increase throughput. ''rsync'' is also commonly used to make backups.+
  
 <html></small></html> <html></small></html>
- 
-=== Removing files === 
- 
-The program that removes files is called **''rm''**. 
- 
-''rm'' by default won't remove directories (regardless if empty) and write-protected files. 
- 
-To remove a directory with ''rm'' (recursively with their contents) one has to add a ''-r'' switch. 
- 
-To remove write-protected files (and stop printing warnings whenever a file to be removed already does not exist) the switch ''-f'' (''--force'') can be used. 
- 
-A misused ''rm -rf …'' command is a notorious source of data loss. Beware especially of asterisk and what it expands to. 
- 
-''rm'' accepts **''-I''** and ''-i'' switches that ask for confirmation. ''-I'' asks once upon attempt to remove multiple files, and ''-i'' asks upon each file. 
- 
-~~Exercise.#~~ Create multiple files with ''touch file_{a..z}''. \\ Remove all files with a single ''rm'' command with 1) no switches 2) ''-f'' switch 3) ''-I'' switch 4) ''-i'' switch. 
- 
-~~Exercise.#~~ Repeat the remove command from the previous exercise when the files are gone. \\ Then repeat the command again with ''-f'' switch. 
- 
-~~Exercise.#~~ Create a directory and remove it with ''rm''. 
  
  
os_cp/intro.1677673999.txt.gz · Last modified: (external edit)