This chapter takes look at the most common directories in the Linux file tree. It also shows that on Unix everything is a file.
Many Linux distributions partially follow the Filesystem Hierarchy Standard. The FHS may help make more Unix/Linux file system trees conform better in the future. The FHS is available online at http://www.pathname.com/fhs/ where we read: “The filesystem hierarchy standard has been designed to be used by Unix distribution developers, package developers, and system implementers. However, it is primarily intended to be a reference and is not a tutorial on how to manage a Unix filesystem or directory hierarchy.”
All Linux systems have a directory structure that starts at the root directory. The root directory is represented by a forward slash, like this: /. Everything that exists on your Linux system can be found below this root directory. Let’s take a brief look at the contents of the root directory.
[user@Predator-pc]$ ls /
bin dev home media mnt proc sbin srv tftpboot usr boot etc lib misc opt root selinux sys tmp var
Binaries are files that contain compiled source code (or machine code). Binaries can be executed on the computer. Sometimes binaries are called executables.
The /bin directory contains binaries for use by all users. According to the FHS the /bin directory should contain /bin/cat and /bin/date (among others). you see common Unix/Linux commands like cat, cp, cpio, date, dd, echo, grep, and so on. Many of these will be covered in this book.
user@predator-pc~$ ls /bin
archdetect egrep mt setupcon autopartition false mt-gnu sh bash fgconsole mv sh.distrib bunzip2 fgrep nano sleep bzcat fuser nc stralign bzcmp fusermount nc.traditional stty bzdiff get_mountoptions netcat su bzegrep grep netstat sync bzexe gunzip ntfs-3g sysfs bzfgrep gzexe ntfs-3g.probe tailf bzgrep gzip parted_devices tar bzip2 hostname parted_server tempfile bzip2recover hw-detect partman touch bzless ip partman-commit true bzmore kbd_mode perform_recipe ulockmgr cat kill pidof umount ...
You can find a /bin subdirectory in many other directories. A user named serena could put her own programs in /home/serena/bin.
Some applications, often when installed directly from source will put themselves in /opt. A samba server installation can use /opt/samba/bin to store its binaries.
/sbin contains binaries to configure the operating system. Many of the system binaries require root privilege to perform certain tasks.
Below a screenshot containing system binaries to change the ip address, partition a disk and create an ext4 file system.
user@ predator-pc:~$ ls -l /sbin/ifconfig /sbin/fdisk /sbin/mkfs.ext4
-rwxr-xr-x 1 root root 97172 2011-02-02 09:56 /sbin/fdisk
-rwxr-xr-x 1 root root 65708 2010-07-02 09:27 /sbin/ifconfig -rwxr-xr-x 5 root root 55140 2010-08-18 18:01 /sbin/mkfs.ext4
Binaries found in /bin and /sbin often use shared libraries located in /lib. Below is a screenshot of the partial contents of /lib.
user@predator-pc~$ ls /lib/libc*
/lib/libc-2.5.so /lib/libcfont.so.0.0.0 /lib/libcom_err.so.2.1
/lib/libcap.so.1 /lib/libcidn-2.5.so /lib/libconsole.so.0
/lib/libcap.so.1.10 /lib/libcidn.so.1 /lib/libconsole.so.0.0.0 /lib/libcfont.so.0 /lib/libcom_err.so.2 /lib/libcrypt-2.5.so
Typically, the Linux kernel loads kernel modules from /lib/modules/$kernelversion/. This directory is discussed in detail in the Linux kernel chapter.
We currently are in a transition between 32-bit and 64-bit systems. Therefore, you may encounter directories named /lib32 and /lib64 which clarify the register size used during compilation time of the libraries. A 64-bit computer may have some 32-bit binaries and libraries for compatibility with legacy applications. This screenshot uses the file utility to demonstrate the difference.
user@predator-pc~$ file /lib32/libc-2.5.so
/lib32/libc-2.5.so: ELF 32-bit LSB shared object, Intel 80386, \ version 1 (SYSV), for GNU/Linux 2.6.0, stripped user@predator-pc~$ file /lib64/libcap.so.1.10
/lib64/libcap.so.1.10: ELF 64-bit LSB shared object, AMD x86-64, \ version 1 (SYSV), stripped
The ELF (Executable and Linkable Format) is used in almost every Unix-like operating system since System V.
The purpose of /opt is to store optional software. In many cases this is software from outside the distribution repository. You may find an empty /opt directory on many systems.
A large package can install all its files in /bin, /lib, /etc subdirectories within /opt/ $packagename/. If for example the package is called wp, then it installs in /opt/wp, putting binaries in /opt/wp/bin and manpages in /opt/wp/man.
The /boot directory contains all files needed to boot the computer. These files don’t change very often. On Linux systems you typically find the /boot/grub directory here. /boot/grub contains /boot/grub/grub.cfg (older systems may still have /boot/grub/grub.conf) which defines the boot menu that is displayed before the kernel starts.
All of the machine-specific configuration files should be located in /etc. Historically /etc stood for etcetera, today people often use the Editable Text Configuration backronym.
Many times the name of a configuration files is the same as the application, daemon, or protocol with .conf added as the extension.
user@predator-pc~$ ls /etc/*.conf
/etc/adduser.conf /etc/ld.so.conf /etc/scrollkeeper.conf
/etc/brltty.conf /etc/lftp.conf /etc/sysctl.conf
/etc/ccertificates.conf /etc/libao.conf /etc/syslog.conf
/etc/cvs-cron.conf /etc/logrotate.conf /etc/ucf.conf
/etc/ddclient.conf /etc/ltrace.conf /etc/uniconf.conf
/etc/debconf.conf /etc/mke2fs.conf /etc/updatedb.conf
/etc/deluser.conf /etc/netscsid.conf /etc/usplash.conf
/etc/fdmount.conf /etc/nsswitch.conf /etc/uswsusp.conf
/etc/hdparm.conf /etc/pam.conf /etc/vnc.conf
/etc/host.conf /etc/pnm2ppa.conf /etc/wodim.conf
/etc/inetd.conf /etc/povray.conf /etc/wvdial.conf
/etc/kernel-img.conf /etc/resolv.conf user@predator-pc~$
There is much more to be found in /etc.
A lot of Unix/Linux distributions have an /etc/init.d directory that contains scripts to start and stop daemons. This directory could disappear as Linux migrates to systems that replace the old init way of starting all daemons.
The graphical display (aka X Window System or just X) is driven by software from the X.org foundation. The configuration file for your graphical display is /etc/X11/xorg.conf. /etc/skel/
The skeleton directory /etc/skel is copied to the home directory of a newly created user. It usually contains hidden files like a .bashrc script.
This directory; which is not mentioned in the FHS, contains a lot of Red Hat Enterprise Linux configuration files. We will discuss some of them in greater detail.
user@Predator-pc:~$ ls /etc/sysconfig/
apmd firstboot irda network saslauthd apmscripts grub irqbalance networking selinux authconfig hidd keyboard ntpd spamassassin autofs httpd kudzu openib.conf squid bluetooth hwconf lm_sensors pand syslog clock i18n mouse pcmcia sys-config-sec console init mouse.B pgsql sysconfig-users crond installinfo named prelink syslogviewer
desktop ipmi netdump rawdevices tux diskdump iptables netdump_id_dsa rhn vncservers dund iptables-cfg netdump_id_dsa.p samba xinetd user@Predator-pc:~$
The file /etc/sysconfig/firstboot tells the Red Hat Setup Agent not to run at boot time. If you want to run the Red Hat Setup Agent at the next reboot, then simply remove this file, and run chkconfig --level 5 firstboot on. The Red Hat Setup Agent allows you to install the latest updates, create a user account, join the Red Hat Network and more. It will then create the /etc/sysconfig/firstboot file again.
user@Predator-pc:~$ cat /etc/sysconfig/firstboot RUN_FIRSTBOOT=NO
The /etc/sysconfig/harddisks file contains some parameters to tune the hard disks. The file explains itself.
You can see hardware detected by kudzu in /etc/sysconfig/hwconf. Kudzu is software from Red Hat for automatic discovery and configuration of hardware. The keyboard type and keymap table are set in the /etc/sysconfig/keyboard file. For more console keyboard information, check the manual pages of keymaps(5), dumpkeys(1), loadkeys(1) and the directory /lib/kbd/keymaps/.
root@Predator-pc:/etc/sysconfig# cat keyboard
KEYBOARDTYPE=“pc”
KEYTABLE=“us”
We will discuss networking files in this directory in the networking chapter. data directories
Users can store personal or project data under /home. It is common (but not mandatory by the fhs) practice to name the users home directory after the user name in the format /home/ $USERNAME. For example:
Besides giving every user (or every project or group) a location to store personal files, the home directory of a user also serves as a location to store the user profile. A typical Unix user profile contains many hidden files (files whose file name starts with a dot). The hidden files of the Unix user profiles contain settings specific for that user.
user@Predator-pc::~$ ls -d /home/user/.*
/home/user/. /home/user/.bash_profile /home/user/.ssh
/home/user/.. /home/user/.bashrc /home/user/.viminfo
/home/user/.bash_history /home/user/.lesshst
On many systems /root is the default location for personal data and profile of the root user. If it does not exist by default, then some administrators create it.
You may use /srv for data that is served by your system. The FHS allows locating cvs, rsync, ftp and www data in this location. The FHS also approves administrative naming in / srv, like /srv/project55/ftp and /srv/sales/www.
On Sun Solaris (or Oracle Solaris) /export is used for this purpose.
The /media directory serves as a mount point for removable media devices such as CDROM’s, digital cameras, and various usb-attached devices. Since /media is rather new in the Unix world, you could very well encounter systems running without this directory. Solaris 9 does not have it, Solaris 10 does. Most Linux distributions today mount all removable media in /media.
user@predator:~$ ls /media/ cdrom cdrom0 usbdisk
The /mnt directory should be empty and should only be used for temporary mount points (according to the FHS).
Unix and Linux administrators used to create many directories here, like /mnt/something/. You likely will encounter many systems with more than one directory created and/or mounted inside /mnt to be used for various local and remote filesystems.
Applications and users should use /tmp to store temporary data when needed. Data stored in /tmp may use either disk space or RAM. Both of which are managed by the operating system. Never use /tmp to store data that is important or which you wish to archive.
Device files in /dev appear to be ordinary files but are not actually located on the hard disk. The /dev directory is populated with files as the kernel is recognising hardware.
common physical devices
Common hardware such as hard disk devices are represented by device files in /dev. Below a screenshot of SATA device files on a laptop and then IDE attached drives on a desktop. (The detailed meaning of these devices will be discussed later.)
#
# SATA or SCSI or USB
#
user@predator-pc~$ ls /dev/sd* /dev/sda /dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb /dev/sdb1 /dev/sdb2
#
# IDE or ATAPI
#
user@barry:~$ ls /dev/hd*
/dev/hda /dev/hda1 /dev/hda2 /dev/hdb /dev/hdb1 /dev/hdb2 /dev/hdc
Besides representing physical hardware, some device files are special. These special devices can be very useful.
For example, /dev/tty1 represents a terminal or console attached to the system. (Don’t break your head on the exact terminology of ‘terminal’ or ‘console’, what we mean here is a command line interface.) When typing commands in a terminal that is part of a graphical interface like Gnome or KDE, then your terminal will be represented as /dev/pts/1 (1 can be another number).
On Linux you will find other special devices such as /dev/null which can be considered a black hole; it has unlimited storage, but nothing can be retrieved from it. Technically speaking, anything written to /dev/null will be discarded. /dev/null can be useful to discard unwanted output from commands. /dev/null is not a good location to store your backups ;-).
/proc is another special directory, appearing to be ordinary files, but not taking up disk space. It is actually a view of the kernel, or better, what the kernel manages, and is a means to interact with it directly. /proc is a proc filesystem.
user@Predator-pc:~$ mount -t proc
none on /proc type proc (rw)
When listing the /proc directory you will see many numbers (on any Unix) and some interesting files (on Linux)
user@predator-pc~$ ls /proc
1 2339 4724 5418 6587 7201 cmdline mounts
10175 2523 4729 5421 6596 7204 cpuinfo mtrr
10211 2783 4741 5658 6599 7206 crypto net
10239 2975 4873 5661 6638 7214 devices pagetypeinfo
141 29775 4874 5665 6652 7216 diskstats partitions
15045 29792 4878 5927 6719 7218 dma sched_debug
1519 2997 4879 6 6736 7223 driver scsi
1548 3 4881 6032 6737 7224 execdomains self
1551 30228 4882 6033 6755 7227 fb slabinfo
1554 3069 5 6145 6762 7260 filesystems stat
1557 31422 5073 6298 6774 7267 fs swaps
1606 3149 5147 6414 6816 7275 ide sys
31507 5203 6418 6991 7282 interrupts sysrq-trigger
3189 5206 6419 6993 7298 iomem sysvipc
3193 5228 6420 6996 7319 ioports timer_list
18898 3246 5272 6421 7157 7330 irq timer_stats 19799 3248 5291 6422 7163 7345 kallsyms tty
3253 5294 6423 7164 7513 kcore uptime
3372 5356 6424 7171 7525 key-users version
1987 4 5370 6425 7175 7529 kmsg version_signature
1989 42 5379 6426 7188 9964 loadavg vmcore 2 45 5380 6430 7189 acpi locks vmnet
20845 4542 5412 6450 7191 asound meminfo vmstat
221 46 5414 6551 7192 buddyinfo misc zoneinfo
2338 4704 5416 6568 7199 bus modules
Let’s investigate the file properties inside /proc. Looking at the date and time will display the current date and time showing the files are constantly updated (a view on the kernel).
user@Predator-pc:~$ date Mon Jan 29 18:06:32 EST 2007 user@Predator-pc:~$ ls -al /proc/cpuinfo
-r--r--r-- 1 root root 0 Jan 29 18:06 /proc/cpuinfo user@Predator-pc:~$ user@Predatorpc:~$ ...time passes...
user@Predator-pc:~$ user@Predatorpc:~$ date
Mon Jan 29 18:10:00 EST 2007 user@Predator-pc:~$ ls -al /proc/cpuinfo
-r--r--r-- 1 root root 0 Jan 29 18:10 /proc/cpuinfo
Most files in /proc are 0 bytes, yet they contain data--sometimes a lot of data. You can see this by executing cat on files like /proc/cpuinfo, which contains information about the CPU.
user@Predator-pc:~$ file /proc/cpuinfo
/proc/cpuinfo: empty
user@Predator-pc:~$ cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 15 model
: 43
Just for fun, here is /proc/cpuinfo on a Sun Sunblade 1000...
user@pasha:~$ cat /proc/cpuinfo cpu : TI UltraSparc III (Cheetah) fpu : UltraSparc III integrated FPU promlib : Version 3 Revision 2 prom : 4.2.2 type : sun4u ncpus probed : 2 ncpus active : 2
Cpu0Bogo : 498.68
Cpu0ClkTck : 000000002cb41780 Cpu1Bogo : 498.68
Cpu1ClkTck : 000000002cb41780
MMU Type : Cheetah State:
CPU0: online
CPU1: online
Most of the files in /proc are read only, some require root privileges, some files are writable, and many files in /proc/sys are writable. Let’s discuss some of the files in /proc.
On the x86 architecture, /proc/interrupts displays the interrupts.
user@Predator-pc:~$ cat /proc/interrupts
CPU0
0: 13876877 IO-APIC-edge timer
1: 15 IO-APIC-edge i8042
8: 1 IO-APIC-edge rtc
9: 0 IO-APIC-level acpi
12: 67 IO-APIC-edge i8042
14: 128 IO-APIC-edge ide0
15: 124320 IO-APIC-edge ide1
169: 111993 IO-APIC-level ioc0
177: 2428 IO-APIC-level eth0
NMI: 0
LOC: 13878037
ERR: 0
MIS: 0
On a machine with two CPU’s, the file looks like this.
user@predator-pc~$ cat /proc/interrupts
CPU0 CPU1
0: 860013 0 IO-APIC-edge timer
1: 4533 0 IO-APIC-edge i8042
7: 0 0 IO-APIC-edge parport0
8: 6588227 0 IO-APIC-edge rtc
10: 2314 0 IO-APIC-fasteoi acpi
12: 133 0 IO-APIC-edge i8042
14: 0 0 IO-APIC-edge libata
15: 72269 0 IO-APIC-edge libata
18: 1 0 IO-APIC-fasteoi yenta
19: 115036 0 IO-APIC-fasteoi eth0
20: 126871 0 IO-APIC-fasteoi libata, ohci1394
21: 30204 0 IO-APIC-fasteoi ehci_hcd:usb1, uhci_hcd:usb2
22: 1334 0 IO-APIC-fasteoi saa7133[0], saa7133[0]
24: 234739 0 IO-APIC-fasteoi nvidia
NMI: 72 42
LOC: 860000 859994
ERR: 0
The physical memory is represented in /proc/kcore. Do not try to cat this file instead use a debugger. The size of /proc/kcore is the same as your physical memory plus four bytes.
user@predator-pc~$ ls -lh /proc/kcore
-r-------- 1 root root 2.0G 2007-01-30 08:57 /proc/kcore user@predator-pc~$
The /sys directory was created for the Linux 2.6 kernel. Since 2.6, Linux uses sysfs to support usb and IEEE 1394 (FireWire) hot plug devices. See the manual pages of udev(8) (the successor of devfs) and hotplug(8) for more info (or visit http://linuxhotplug.sourceforge.net/ ).
Basically the /sys directory contains kernel information about hardware.
Although /usr is pronounced like user remember that it stands for Unix System Resources. The /usr hierarchy should contain shareable read only data. Some people choose to mount /usr as read only. This can be done from its own partition or from a read only NFS share (NFS is discussed later).
The /usr/bin directory contains a lot of commands.
user@deb508:~$ ls
/usr/bin | wc -l 1395
(On Solaris the /bin directory is a symbolic link to /usr/bin.)
The /usr/include directory contains general use include files for C.
user@ubu1010:~$ ls /usr/include/
aalib.h expat_config.h math.h search.h af_vfs.h expat_external.h mcheck.h semaphore.h aio.h expat.h memory.h setjmp. fcntl.h menu.h sgtty.h aliases. features.h mntent.h shadow.h ...
h AL
h
The /usr/lib directory contains libraries that are not directly executed by users or scripts.
user@deb508:~$ ls /usr/lib | head -7
4Suite ao apt arj aspell avahi bonobo
The /usr/local directory can be used by an administrator to install software locally.
user@deb508:~$ ls /usr/local/
bin etc games include lib man sbin share src user@deb508:~$ du -sh /usr/local/ 128K /usr/local/
The /usr/share directory contains architecture independent data. As you can see, this is a fairly large directory.
user@deb508:~$ ls /usr/share/ | wc -l
263 user@deb508:~$ du -sh /usr/share/ 1.3G /usr/share/
This directory typically contains /usr/share/man for manual pages.
user@deb508:~$ ls /usr/share/man
cs fr hu it.UTF-8 man2 man6 pl.ISO8859-2 sv de fr.ISO8859-1 id ja man3 man7 pl.UTF-8 tr es fr.UTF8 it ko man4 man8 pt_BR zh_CN fi gl it.ISO8859-
1 man1 man5 pl ru zh_TW
And it contains /usr/share/games for all static game data (so no high-scores or play logs).
user@ubu1010:~$ ls
/usr/share/games/
The /usr/src directory is the recommended location for kernel source files.
user@deb508:~$ ls -l /usr/src/
Files that are unpredictable in size such as log, cache and spool files, should be located in /var.
The /var/log directory serves as a central point to contain all log files.
You can use the set command to display a list of environment variables. On Debian stable and Debian systems, the set command will also list shell functions after the shell variables. Use set | more to see the variables then.
Use the unset command to remove a variable from your shell environment.
[user@RHEL4b ~]$ MyVar=8472
[user@RHEL4b ~]$ echo $MyVar
8472
[user@RHEL4b ~]$ unset MyVar
[user@RHEL4b ~]$ echo $MyVar
[user@RHEL4b ~]$
The $PS1 variable determines your shell prompt. You can use backslash escaped special characters like \u for the username or \w for the working directory. The bash manual has a complete reference.
In this example we change the value of $PS1 a couple of times.
user@deb503:~$ PS1=prompt prompt promptPS1=‘prompt ‘ prompt prompt PS1=‘> ‘ >
> PS1=‘\u@\h$ ‘ user@deb503$
user@deb503$ PS1=‘\u@\h:\W$’ user@deb503:~$
To avoid unrecoverable mistakes, you can set normal user prompts to green and the root prompt to red. Add the following to your .bashrc for a green user prompt:
# color prompt by user RED=‘\[\033[01;31m\]’
WHITE=‘\[\033[01;00m\]’
GREEN=‘\[\033[01;32m\]’ BLUE=‘\[\033[01;34m\]’ export
PS1=“${debian_chroot:+($debian_chroot)}$GREEN\u$WHITE@$BLUE\h$WHITE\w\$ “
The $PATH variable is determines where the shell is looking for commands to execute (unless the command is builtin or aliased). This variable contains a list of directories, separated by colons.
[[user@RHEL4b ~]$ echo $PATH /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:
The shell will not look in the current directory for commands to execute! (Looking for executables in the current directory provided an easy way to hack PC-DOS computers). If you want the shell to look in the current directory then add a . (dot) at the end of your $PATH.
[user@RHEL4b ~]$ PATH=$PATH:.
[user@RHEL4b ~]$ echo $PATH /usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:. [user@RHEL4b ~]$
Your path might be different when using su instead of su - because the latter will take on the environment of the target user. The root user typically has /sbin directories added to the $PATH variable.
[user@RHEL3 ~]$ su
Password:
[root@RHEL3 user]# echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
[root@RHEL3 user]# exit
[user@RHEL3 ~]$ su Password:
[root@RHEL3 ~]# echo $PATH
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin: [root@RHEL3 ~]#
The env command without options will display a list of exported variables. The difference with set with options is that set lists all variables, including those not exported to child shells.
But env can also be used to start a clean shell (a shell without any inherited environment). The env -i command clears the environment for the subshell. Notice in this screenshot that bash will set the $SHELL variable on startup.
[user@RHEL4b ~]$ bash -c ‘echo $SHELL $HOME $USER’
/bin/bash /home/user user
[user@RHEL4b ~]$ env -i bash -c ‘echo $SHELL $HOME $USER’
/bin/bash
[user@RHEL4b ~]$
You can use the env command to set the $LANG, or any other, variable for just one instance of bash with one command. The example below uses this to show the influence of the $LANG variable on file globbing (see the chapter on file globbing).
[user@RHEL4b test]$ env LANG=C bash -c ‘ls File[a-z]’
Filea Fileb
[user@RHEL4b test]$ env LANG=en_US.UTF-8 bash -c ‘ls File[a-z]’
Filea FileA Fileb FileB [user@RHEL4b test]$
You can export shell variables to other shells with the export command. This will export the variable to child shells.
[user@RHEL4b ~]$ var3=three
[user@RHEL4b ~]$ var4=four
[user@RHEL4b ~]$ export var4 [user@RHEL4b ~]$ echo $var3 $var4 three four
[user@RHEL4b ~]$ bash
[user@RHEL4b ~]$ echo $var3 $var4 four
But it will not export to the parent shell (previous screenshot continued).
[user@RHEL4b ~]$ export var5=five [user@RHEL4b ~]$ echo $var3 $var4 $var5 four five
[user@RHEL4b ~]$ exit exit
[user@RHEL4b ~]$ echo $var3 $var4 $var5 three four [user@RHEL4b ~]$
There are two predominant software paradigms: Free and Open Source Software (FOSS) and proprietary software. The criteria for differentiation between these two approaches is based on control over the software. With proprietary software control tends to lie more with the vendor, while with Free and Open Source Software tends to be more weighted towards the end user. But even though the paradigms differ, they use the same copyright laws to reach and enforce their goals. From a legal perspective, Free and Open Source Software can be considered as software to which users generally receive more rights via their license agreement than they would have with a proprietary software license, yet the underlying license mechanisms are the same.
Legal theory states that the author of FOSS, contrary to the author of public domain software, has in no way whatsoever given up his rights on his work. FOSS supports on the rights of the author (the copyright) to impose FOSS license conditions. The FOSS license conditions need to be respected by the user in the same way as proprietary license conditions. Always check your license carefully before you use third party software.
Examples of proprietary software are AIX from IBM, HP-UX from HP and Oracle Database 11g. You are not authorised to install or use this software without paying a licensing fee. You are not authorised to distribute copies and you are not authorised to modify the closed source code.
Software that is original in the sense that it is an intellectual creation of the author benefits copyright protection. Non-original software does not come into consideration for copyright protection and can, in principle, be used freely. Public domain software is considered as software to which the author has given up all rights and on which nobody is able to enforce any rights. This software can be used, reproduced or executed freely, without permission or the payment of a fee. Public domain software can in certain cases even be presented by third parties as own work, and by modifying the original work, third parties can take certain versions of the public domain software out of the public domain again.
Freeware is not public domain software or FOSS. It is proprietary software that you can use without paying a license cost. However, the often-strict license terms need to be respected.
Examples of freeware are Adobe Reader, Skype and Command and Conquer: Tiberian Sun (this game was sold as proprietary in 1999 and is since 2011 available as freeware).
Both the Free Software (translates to vrije software in Dutch and to Logiciel Libre in French) and the Open-Source Software movement largely pursue similar goals and endorse similar software licenses. But historically, there has been some perception of differentiation due to different emphases. Where the Free Software movement focuses on the rights (the four freedoms) which Free Software provides to its users, the Open Source Software movement points to its Open Source Definition and the advantages of peer-to-peer software development. Recently, the term free and open source software or FOSS has arisen as a neutral alternative. A lesser-used variant is free/libre/open source software (FLOSS), which uses libre to clarify the meaning of free as in freedom rather than as in at no charge.
Examples of free software are gcc, MySQL and gimp.
Detailed information about the four freedoms can be found here:
http://www.gnu.org/philosophy/free-sw.html
The open-source definition can be found at:
http://www.opensource.org/docs/osd
The above definition is based on the Debian Free Software Guidelines available here:
http://www.debian.org/social_contract#guidelines
More and more software is being released under the GNU GPL (in 2006 Java was released under the GPL). This license (v2 and v3) is the main license endorsed by the Free Software Foundation. It’s main characteristic is the copyleft principle. This means that everyone in the chain of consecutive users; in return for the right of use that is assigned, needs to distribute the improvements he makes to the software and his derivative works under the same conditions to other users, if he chooses to distribute such improvements or derivative works. In other words; software which incorporates GNU GPL software, needs to be distributed in turn as GNU GPL software (or compatible, see below). It is not possible to incorporate copyright protected parts of GNU GPL software in a proprietary licensed work. The GPL has been upheld in court.
You can use GPLv3 software almost without any conditions. If you solely run the software, you even don’t have to accept the terms of the GPLv3. However, any other use - such as modifying or distributing the software - implies acceptance.
In case you use the software internally (including over a network), you may modify the software without being obliged to distribute your modification. You may hire third parties to work on the software exclusively for you and under your direction and control. But if you modify the software and use it otherwise than merely internally, this will be considered as distribution. You must distribute your modifications under GPLv3 (the copyleft principle). Several more obligations apply if you distribute GPLv3 software. Check the GPLv3 license carefully. You create output with GPLv3 software: The GPLv3 does not automatically apply to the output.
There are several versions of the original Berkeley Distribution License. The most common one is the 3-clause license (“New BSD License” or “Modified BSD License”).
This is a permissive free software license. The license places minimal restrictions on how the software can be redistributed. This is in contrast to copyleft licenses such as the GPLv. 3 discussed above, which have a copyleft mechanism. This difference is of less importance when you merely use the software, but kicks in when you start redistributing verbatim copies of the software or your own modified versions.
FOSS or not, there are many kind of licenses on software. You should read and understand them before using any software.
When you use several sources or wishes to redistribute your software under a different license, you need to verify whether all licenses are compatible. Some FOSS licenses (such as BSD) are compatible with proprietary licenses, but most are not. If you detect a license incompatibility, you must contact the author to negotiate different license conditions or refrain from using the incompatible software.
This chapter will explain the use of man pages (also called manual pages) on your Unix or Linux computer.
You will learn the man command together with related commands like whereis, whatis and mandb.
Most Unix files and commands have pretty good man pages to explain their use. Man pages also come in handy when you are using multiple flavours of Unix or several Linux distributions since options and parameters sometimes vary.
Type man followed by a command (for which you want help) and start reading. Press q to quit the manpage. Some man pages contain examples (near the end).
user@predator-pc~$ man whois Reformatting whois(1), please wait...
Most configuration files have their own manual.
user@predator-pc~$ man syslog.conf Reformatting syslog.conf(5), please wait...
This is also true for most daemons (background programs) on your system.
user@predator-pc~$ man syslogd Reformatting syslogd(8), please wait...
man -k (apropos) man -k (or apropos) shows a list of man pages containing a string.
user@predator-pc~$ man -k syslog lm-syslog-setup (8) - configure laptop mode to switch syslog.conf ... logger (1) - a shell command interface to the syslog(3) ... syslog-facility (8) - Setup and remove LOCALx facility for sysklogd syslog.conf (5) - syslogd(8) configuration file syslogd (8) - Linux system logging utilities. syslogdlistfiles (8) - list system logfiles
To see just the description of a manual page, use whatis followed by a string.
user@u810:~$ whatis route
route (8) - show / manipulate the IP routing table
The location of a manpage can be revealed with whereis.
user@predator-pc~$ whereis -m whois whois: /usr/share/man/man1/whois.1.gz
This file is directly readable by man.
user@predator-pc~$ man /usr/share/man/man1/whois.1.gz
Therefor, when referring to the man page of the passwd command, you will see it written as passwd(1); when referring to the passwd file, you will see it written as passwd(5). The screenshot explains how to open the man page in the correct section.
[user@RHEL52 ~]$ man passwd # opens the first manual found
[user@RHEL52 ~]$ man 5 passwd # opens a page from section 5
If you want to know more about man, then Read The Fantastic Manual (RTFM).
Unfortunately, manual pages do not have the answer to everything...
user@predator-pc~$ man woman No manual entry for woman
Should you be convinced that a man page exists, but you can’t access it, then try running mandb on Debian/Mint.
root@predator-pc~# mandb 0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.
Or run makewhatis on CentOS/Redhat.
[root@centos65 ~]# apropos scsi scsi: nothing appropriate
[root@centos65 ~]# makewhatis [root@centos65 ~]# apropos scsi hpsa (4) - HP Smart Array SCSI driver lsscsi (8) - list SCSI devices (or hosts) and their attributes sd (4) - Driver for SCSI Disk Drives st (4) - SCSI tape device
This module is a brief overview of the most common commands to work with directories: pwd, cd, ls, mkdir and rmdir. These commands are available on any Linux (or Unix) system.
This module also discusses absolute and relative paths and path completion in the bash shell.
The you are here sign can be displayed with the pwd command (Print Working Directory). Go ahead, try it: Open a command line interface (also called a terminal, console or xterm) and type pwd. The tool displays your current directory.
user@debian8:~$ pwd /home/user
You can change your current directory with the cd command (Change Directory).
user@debian8$ cd /etc user@debian8$ pwd
/etc user@debian8$ cd /bin user@debian8$ pwd
/bin user@debian8$ cd /home/user/ user@debian8$ pwd /home/user
The cd is also a shortcut to get back into your home directory. Just typing cd without a target directory, will put you in your home directory. Typing cd ~ has the same effect.
user@debian8$ cd /etc user@debian8$ pwd /etc user@debian8$ cd user@debian8$ pwd
/home/user user@debian8$ cd ~ user@debian8$ pwd
/home/user
To go to the parent directory (the one just above your current directory in the directory tree), type cd .. .
user@debian8$ pwd /usr/share/games user@debian8$ cd .. user@debian8$ pwd /usr/share
To stay in the current directory, type cd . ;-) We will see useful use of the . character representing the current directory later.
Another useful shortcut with cd is to just type cd - to go to the previous directory.
user@debian8$ pwd
/home/user user@debian8$ cd /etc user@debian8$ pwd
/etc user@debian8$ cd -
/home/user user@debian8$ cd /etc
You should be aware of absolute and relative paths in the file tree. When you type a path starting with a slash (/), then the root of the file tree is assumed. If you don’t start your path with a slash, then the current directory is the assumed starting point.
The screenshot below first shows the current directory /home/user. From within this directory, you have to type cd /home instead of cd home to go to the /home directory.
user@debian8$ pwd /home/user
user@debian8$ cd home
bash: cd: home: No such file or directory user@debian8$ cd /home user@debian8$ pwd
/home
When inside /home, you have to type cd user instead of cd /user to enter the subdirectory user of the current directory /home.
user@debian8$ pwd
/home
user@debian8$ cd /user
bash: cd: /user: No such file or directory user@debian8$ cd user user@debian8$ pwd /home/user
In case your current directory is the root directory /, then both cd /home and cd home will get you in the /home directory.
user@debian8$ pwd
/
user@debian8$ cd home user@debian8$ pwd /home user@debian8$ cd / user@debian8$ cd /home user@debian8$ pwd /home
This was the last screenshot with pwd statements. From now on, the current directory will often be displayed in the prompt. Later in this book we will explain how the shell variable $PS1 can be configured to show this.
The tab key can help you in typing a path without errors. Typing cd /et followed by the tab key will expand the command line to cd /etc/. When typing cd /Et followed by the tab key, nothing will happen because you typed the wrong path (upper case E).
You will need fewer key strokes when using the tab key, and you will be sure your typed path is correct!
You can list the contents of a directory with ls.
user@debian8:~$ ls allfiles.txt dmesg.txt services stuff summer.txt user@debian8:~$
A frequently used option with ls is -a to show all files. Showing all files means including the hidden files. When a file name on a Linux file system starts with a dot, it is considered a hidden file and it does not show up in regular file listings.
user@debian8:~$ ls
allfiles.txt dmesg.txt services stuff summer.txt user@debian8:~$ ls -a
. allfiles.txt .bash_profile dmesg.txt .lesshst stuff
.. .bash_history .bashrc services .ssh summer.txt user@debian8:~$
Many times you will be using options with ls to display the contents of the directory in different formats or to display different parts of the directory. Typing just ls gives you a list of files in the directory. Typing ls -l (that is a letter L, not the number 1) gives you a long listing.
user@debian8:~$ ls -l total 17296
-rw-r--r-- 1 user user 17584442 Sep 17 00:03 allfiles.txt
-rw-r--r-- 1 user user 96650 Sep 17 00:03 dmesg.txt
-rw-r--r-- 1 user user 19558 Sep 17 00:04 services drwxr-xr-x 2 user user 4096 Sep 17 00:04 stuff
-rw-r--r-- 1 user user 0 Sep 17 00:04 summer.txt
Another frequently used ls option is -h. It shows the numbers (file sizes) in a more human readable format. Also shown below is some variation in the way you can give the options to ls. We will explain the details of the output later in this book.
Note that we use the letter L as an option in this screenshot, not the number 1.
user@debian8:~$ ls -l -h total 17M
-rw-r--r-- 1 user user 17M Sep 17 00:03 allfiles.txt
-rw-r--r-- 1 user user 95K Sep 17 00:03 dmesg.txt
-rw-r--r-- 1 user user 20K Sep 17 00:04 services drwxr-xr-x 2 user user 4.0K Sep 17 00:04 stuff
-rw-r--r-- 1 user user 0 Sep 17 00:04 summer.txt user@debian8:~$ ls -lh total 17M
-rw-r--r-- 1 user user 17M Sep 17 00:03 allfiles.txt
-rw-r--r-- 1 user user 95K Sep 17 00:03 dmesg.txt
-rw-r--r-- 1 user user 20K Sep 17 00:04 services drwxr-xr-x 2 user user 4.0K Sep 17 00:04 stuff
-rw-r--r-- 1 user user 0 Sep 17 00:04 summer.txt user@debian8:~$ ls -hl total 17M
-rw-r--r-- 1 user user 17M Sep 17 00:03 allfiles.txt
-rw-r--r-- 1 user user 95K Sep 17 00:03 dmesg.txt
-rw-r--r-- 1 user user 20K Sep 17 00:04 services drwxr-xr-x 2 user user 4.0K Sep 17 00:04 stuff
-rw-r--r-- 1 user user 0 Sep 17 00:04 summer.txt user@debian8:~$ ls -h -l total 17M
-rw-r--r-- 1 user user 17M Sep 17 00:03 allfiles.txt
-rw-r--r-- 1 user user 95K Sep 17 00:03 dmesg.txt
-rw-r--r-- 1 user user 20K Sep 17 00:04 services drwxr-xr-x 2 user user 4.0K Sep 17 00:04 stuff
-rw-r--r-- 1 user user 0 Sep 17 00:04 summer.txt user@debian8:~$
Walking around the Unix file tree is fun, but it is even more fun to create your own directories with mkdir. You have to give at least one parameter to mkdir, the name of the new directory to be created. Think before you type a leading / .
user@debian8:~$ mkdir mydir user@debian8:~$ cd mydir
user@debian8:~/mydir$ ls -al total 8 drwxr-xr-x 2 user user 4096 Sep 17 00:07 . drwxr-xr-x 48 user user 4096 Sep 17 00:07 .. user@debian8:~/mydir$ mkdir stuff user@debian8:~/mydir$ mkdir otherstuff user@debian8:~/mydir$ ls -l total 8
drwxr-xr-x 2 user user 4096 Sep 17 00:08 otherstuff drwxr-xrx 2 user user 4096 Sep 17 00:08 stuff user@debian8:~/mydir$
The following command will fail, because the parent directory of threedirsdeep does not exist.
user@debian8:~$ mkdir mydir2/mysubdir2/threedirsdeep
mkdir: cannot create directory ‘mydir2/mysubdir2/threedirsdeep’: No such fi\ le or directory
When given the option -p, then mkdir will create parent directories as needed.
When a directory is empty, you can use rmdir to remove the directory.
user@debian8:~/mydir$ ls -l total 8
drwxr-xr-x 2 user user 4096 Sep 17 00:08 otherstuff drwxr-xrx 2 user user 4096 Sep 17 00:08 stuff user@debian8:~/mydir$ rmdir otherstuff user@debian8:~/mydir$ cd ..
user@debian8:~$ rmdir mydir
rmdir: failed to remove ‘mydir’: Directory not empty
user@debian8:~$ rmdir mydir/stuff
user@debian8:~$ rmdir mydir
And similar to the mkdir -p option, you can also use rmdir to recursively remove directories.
user@debian8:~$ mkdir -p test42/subdir user@debian8:~$ rmdir -p
test42/subdir
In this chapter we learn how to recognise, create, remove, copy and move files using commands like file, touch, rm, cp, mv and rename.
Files on Linux (or any Unix) are case sensitive. This means that FILE1 is different from file1, and /etc/hosts is different from /etc/Hosts (the latter one does not exist on a typical Linux computer).
This screenshot shows the difference between two files, one with upper case W, the other with lower case w.
user@predator-pc~/Linux$ ls
winter.txt Winter.txt user@predator-pc~/Linux$ cat winter.txt
It is cold. user@predatorpc~/Linux$ cat Winter.txt It is very cold!
A directory is a special kind of file, but it is still a (case sensitive!) file. Each terminal window (for example /dev/pts/4), any hard disk or partition (for example /dev/sdb1) and any process are all represented somewhere in the file system as a file. It will become clear throughout this course that everything on Linux is a file. file
The file utility determines the file type. Linux does not use extensions to determine the file type. The command line does not care whether a file ends in .txt or .pdf. As a system administrator, you should use the file command to determine the file type. Here are some examples on a typical Linux system.
user@predator-pc~$ file pic33.png
pic33.png: PNG image data, 3840 x 1200, 8-bit/color RGBA, non-interlaced user@predator-pc~$ file /etc/passwd
/etc/passwd: ASCII text user@predator-pc~$ file HelloWorld.c HelloWorld.c: ASCII C program text
The file command uses a magic file that contains patterns to recognise file types. The magic file is located in /usr/share/file/magic. Type man 5 magic for more information. It is interesting to point out file -s for special files like those in /dev and /proc.
root@debian6~# file /dev/sda
/dev/sda: block special
root@debian6~# file -s /dev/sda
/dev/sda: x86 boot sector; partition 1: ID=0x83, active, starthead...
root@debian6~# file /proc/cpuinfo
/proc/cpuinfo: empty root@debian6~# file -s /proc/cpuinfo /proc/cpuinfo:
ASCII C++ program text
One easy way to create an empty file is with touch. (We will see many other ways for creating files later in this book.)
This screenshot starts with an empty directory, creates two files with touch and the lists those files.
user@debian7:~$ ls -l total 0
user@debian7:~$ touch file42 user@debian7:~$ touch file33 user@debian7:~$ ls l
total 0
-rw-r--r-- 1 user user 0 Oct 15 08:57 file33 -rw-r--r-- 1 user user 0 Oct 15 08:56 file42 user@debian7:~$
The touch command can set some properties while creating empty files. Can you determine what is set by looking at the next screenshot? If not, check the manual for touch.
user@debian7:~$ touch -t 200505050000 SinkoDeMayo user@debian7:~$ touch -t
130207111630 BigBattle.txt user@debian7:~$ ls -l total 0
-rw-r--r-- 1 user user 0 Jul 11 1302 BigBattle.txt
-rw-r--r-- 1 user user 0 Oct 15 08:57 file33
-rw-r--r-- 1 user user 0 Oct 15 08:56 file42 -rw-r--r-- 1 user user 0 May 5 2005 SinkoDeMayo user@debian7:~$
When you no longer need a file, use rm to remove it. Unlike some graphical user interfaces, the command line in general does not have a waste bin or trash can to recover files. When you use rm to remove a file, the file is gone. Therefore, be careful when removing files!
user@debian7:~$ ls
BigBattle.txt file33 file42 SinkoDeMayo user@debian7:~$ rm BigBattle.txt user@debian7:~$ ls file33 file42 SinkoDeMayo user@debian7:~$
To prevent yourself from accidentally removing a file, you can type rm -i.
user@debian7:~$ ls file33 file42 SinkoDeMayo
user@debian7:~$ rm -i file33
rm: remove regular empty file `file33’? yes user@debian7:~$ rm -i SinkoDeMayo
rm: remove regular empty file `SinkoDeMayo’? n user@debian7:~$ ls file42 SinkoDeMayo user@debian7:~$
By default, rm -r will not remove non-empty directories. However, rm accepts several options that will allow you to remove any directory. The rm -rf statement is famous because it will erase anything (providing that you have the permissions to do so). When you are logged on as root, be very careful with rm -rf (the f means force and the r means recursive) since being root implies that permissions don’t apply to you. You can literally erase your entire file system by accident.
user@debian7:~$ mkdir test
user@debian7:~$ rm test
rm: cannot remove `test’: Is a directory user@debian7:~$ rm -rf test user@debian7:~$ ls test
ls: cannot access test: No such file or directory user@debian7:~$
To copy a file, use cp with a source and a target argument.
user@debian7:~$ ls
file42 SinkoDeMayo
user@debian7:~$ cp file42 file42.copy user@debian7:~$ ls
file42 file42.copy SinkoDeMayo
If the target is a directory, then the source files are copied to that target directory.
user@debian7:~$ mkdir dir42 user@debian7:~$ cp SinkoDeMayo dir42
user@debian7:~$ ls dir42/ SinkoDeMayo
To copy complete directories, use cp -r (the -r option forces recursive copying of all files in all subdirectories).
user@debian7:~$ ls dir42 file42 file42.copy SinkoDeMayo user@debian7:~$ cp -r dir42/ dir33 user@debian7:~$ ls dir33 dir42 file42 file42.copy SinkoDeMayo user@debian7:~$ ls dir33/ SinkoDeMayo
You can also use cp to copy multiple files into a directory. In this case, the last argument (a.k.a. the target) must be a directory.
user@debian7:~$ cp file42 file42.copy SinkoDeMayo dir42/ user@debian7:~$ ls dir42/ file42 file42.copy SinkoDeMayo
To prevent cp from overwriting existing files, use the -i (for interactive) option.
user@debian7:~$ cp SinkoDeMayo file42 user@debian7:~$ cp SinkoDeMayo file42
user@debian7:~$ cp -i SinkoDeMayo file42 cp: overwrite `file42’? n user@debian7:~$
Use mv to rename a file or to move the file to another directory.
user@debian7:~$ ls dir33 dir42 file42 file42.copy SinkoDeMayo user@debian7:~$ mv file42 file33 user@debian7:~$ ls
dir33 dir42 file33 file42.copy SinkoDeMayo user@debian7:~$
When you need to rename only one file then mv is the preferred command to use.
The same mv command can be used to rename directories.
user@debian7:~$ ls -l total 8
drwxr-xr-x 2 user user 4096 Oct 15 09:36 dir33 drwxr-xr-x 2 user user 4096 Oct 15 09:36 dir42 -rw-r--r-- 1 user user 0 Oct 15 09:38 file33
-rw-r--r-- 1 user user 0 Oct 15 09:16 file42.copy -rw-r--r-- 1 user user 0 May 5 2005 SinkoDeMayo
user@debian7:~$ mv dir33 backup user@debian7:~$ ls -l total 8
drwxr-xr-x 2 user user 4096 Oct 15 09:36 backup drwxr-xr-x 2 user user 4096 Oct 15 09:36 dir42 -rw-r--r-- 1 user user 0 Oct 15 09:38 file33
-rw-r--r-- 1 user user 0 Oct 15 09:16 file42.copy -rw-r--r-- 1 user user 0 May 5 2005 SinkoDeMayo user@debian7:~$
mv -i
The mv also has a -i switch similar to cp and rm. this screenshot shows that mv -i will ask permission to overwrite an existing file.
user@debian7:~$ mv -i file33 SinkoDeMayo mv: overwrite
`SinkoDeMayo’? no user@debian7:~$
Rename
The rename command is one of the rare occasions where the Linux Fundamentals book has to make a distinction between Linux distributions. Almost every command in the Fundamentals part of this book works on almost every Linux computer. But rename is different.
Try to use mv whenever you need to rename only a couple of files.
The rename command on Debian uses regular expressions (regular expression or shor regex are explained in a later chapter) to rename many files at once.
Below a rename example that switches all occurrences of txt to png for all file names ending in .txt.
user@debian7:~/test42$ ls abc.txt file33.txt file42.txt user@debian7:~/test42$ rename ‘s/\.txt/\.png/’ *.txt user@debian7:~/test42$ ls abc.png file33.png file42.png
This second example switches all (first) occurrences of file into document for all file names ending in .png.
user@debian7:~/test42$ ls abc.png file33.png file42.png user@debian7:~/test42$ rename ‘s/file/document/’ *.png user@debian7:~/test42$ ls abc.png document33.png document42.png user@debian7:~/test42$
On Red Hat Enterprise Linux, the syntax of rename is a bit different. The first example below renames all *.conf files replacing any occurrence of .conf with .backup.
[user@centos7 ~]$ touch one.conf two.conf three.conf
[user@centos7 ~]$ rename .conf .backup *.conf
[user@centos7 ~]$ ls
one.backup three.backup two.backup [user@centos7 ~]$
The second example renames all (*) files replacing one with ONE.
[user@centos7 ~]$ ls
one.backup three.backup two.backup [user@centos7 ~]$ rename one ONE *
[user@centos7 ~]$ ls
ONE.backup three.backup two.backup
[user@centos7 ~]$
You can use head to display the first ten lines of a file.
user@debian7~$ head /etc/passwd
root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/bin/sh man:x:6:12:man:/var/cache/man:/bin/sh lp:x:7:7:lp:/var/spool/lpd:/bin/sh mail:x:8:8:mail:/var/mail:/bin/sh news:x:9:9:news:/var/spool/news:/bin/sh root@debian7~#
The head command can also display the first n lines of a file.
user@debian7~$ head -4 /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh user@debian7~$
And head can also display the first n bytes.
user@debian7~$ head -c14 /etc/passwd root:x:0:0:roouser@debian7~$
Similar to head, the tail command will display the last ten lines of a file.
user@debian7~$ tail /etc/services vboxd 20012/udp binkp 24554/tcp # binkp fidonet protocol asp
27374/tcp # Address Search Protocol asp 27374/udp
csync2 30865/tcp # cluster synchronization tool dircproxy 57000/tcp # Detachable IRC Proxy tfido 60177/tcp # fidonet EMSI over telnet fido 60179/tcp # fidonet EMSI over TCP
# Local services
user@debian7~$
You can give tail the number of lines you want to see.
user@debian7~$ tail -3 /etc/services fido 60179/tcp # fidonet EMSI over TCP
# Local services user@debian7~$
The tail command has other useful options, some of which we will use during this course. cat
The cat command is one of the most universal tools, yet all it does is copy standard input to standard output. In combination with the shell this can be very powerful and diverse. Some examples will give a glimpse into the possibilities. The first example is simple, you can use cat to display a file on the screen. If the file is longer than the screen, it will scroll to the end.
user@debian8:~$ cat /etc/resolv.conf domain linux-training.be search linux-training.be nameserver 192.168.1.42
cat is short for concatenate. One of the basic uses of cat is to concatenate files into a bigger (or complete) file.
user@debian8:~$ echo one >part1 user@debian8:~$ echo two >part2 user@debian8:~$ echo three
>part3
user@debian8:~$ cat part1 one
user@debian8:~$ cat part2 two
user@debian8:~$ cat part3 three
user@debian8:~$ cat part1 part2 part3 one two three
user@debian8:~$ cat part1 part2 part3 >all user@debian8:~$ cat all one two three user@debian8:~$
You can use cat to create flat text files. Type the cat > winter.txt command as shown in the screenshot below. Then type one or more lines, finishing each line with the enter key. After the last line, type and hold the Control (Ctrl) key and press d.
user@debian8:~$ cat > winter.txt It is very cold today!
user@debian8:~$ cat winter.txt
It is very cold today! user@debian8:~$
The Ctrl d key combination will send an EOF (End of File) to the running process ending the cat command.
You can choose an end marker for cat with << as is shown in this screenshot. This construction is called a here directive and will end the cat command.
user@debian8:~$ cat > hot.txt <<stop > It is hot today!
> Yes it is summer.
> stop
user@debian8:~$ cat hot.txt It is hot today!
Yes it is summer. user@debian8:~$
In the third example you will see that cat can be used to copy files. We will explain in detail what happens here in the bash shell chapter.
user@debian8:~$ cat winter.txt
It is very cold today! user@debian8:~$ cat winter.txt > cold.txt user@debian8:~$ cat cold.txt
It is very cold today! user@debian8:~$
Just one example will show you the purpose of tac (cat backwards).
user@debian8:~$ cat count one two three four
user@debian8:~$ tac count four three two one
When between two pipes, the cat command does nothing (except putting stdin on stdout).
[user@RHEL4b pipes]$ tac count.txt | cat | cat | cat | cat | cat five four three two one
[user@RHEL4b pipes]$
Writing long pipes in Unix is fun, but sometimes you may want intermediate results. This is were tee comes in handy. The tee filter puts stdin on stdout and also into a file. So tee is almost the same as cat, except that it has two identical outputs.
[user@RHEL4b pipes]$ tac count.txt | tee temp.txt | tac one two three four five
[user@RHEL4b pipes]$ cat temp.txt five four three two one
[user@RHEL4b pipes]$
grep
The grep filter is famous among Unix users. The most common use of grep is to filter lines of text containing (or not containing) a certain string.
[user@RHEL4b pipes]$ cat tennis.txt
Amelie Mauresmo, Fra
Kim Clijsters, BEL
Justine Henin, Bel
Serena Williams, usa
Venus Williams, USA
[user@RHEL4b pipes]$ cat tennis.txt | grep Williams
Serena Williams, usa
Venus Williams, USA
You can write this without the cat.
[user@RHEL4b pipes]$ grep Williams tennis.txt
Serena Williams, usa
Venus Williams, USA
One of the most useful options of grep is grep -i which filters in a case insensitive way.
[user@RHEL4b pipes]$ grep Bel tennis.txt
Justine Henin, Bel
[user@RHEL4b pipes]$ grep -i Bel tennis.txt
Kim Clijsters, BEL
Justine Henin, Bel
[user@RHEL4b pipes]$
Another very useful option is grep -v which outputs lines not matching the string.
[user@RHEL4b pipes]$ grep -v Fra tennis.txt
Kim Clijsters, BEL
Justine Henin, Bel
Serena Williams, usa
Venus Williams, USA
[user@RHEL4b pipes]$
And of course, both options can be combined to filter all lines not containing a case insensitive string.
[user@RHEL4b pipes]$ grep -vi usa tennis.txt
Amelie Mauresmo, Fra
Kim Clijsters, BEL
Justine Henin, Bel
[user@RHEL4b pipes]$
With grep -A1 one line after the result is also displayed.
user@debian5:~/pipes$ grep -A1 Henin tennis.txt
Justine Henin, Bel
Serena Williams, usa
With grep -B1 one line before the result is also displayed.
user@debian5:~/pipes$ grep -B1 Henin tennis.txt
Kim Clijsters, BEL
Justine Henin, Bel
With grep -C1 (context) one line before and one after are also displayed. All three options (A,B, and C) can display any number of lines (using e.g. A2, B4 or C20).
user@debian5:~/pipes$ grep -C1 Henin tennis.txt
Kim Clijsters, BEL
Justine Henin, Bel
Serena Williams, usa
Counting words, lines and characters is easy with wc.
[user@RHEL4b pipes]$ wc tennis.txt
5 15 100 tennis.txt
[user@RHEL4b pipes]$ wc -l tennis.txt 5 tennis.txt
[user@RHEL4b pipes]$ wc -w tennis.txt 15 tennis.txt
[user@RHEL4b pipes]$ wc -c tennis.txt
100 tennis.txt
[user@RHEL4b pipes]$
The sort filter will default to an alphabetical sort.
user@debian5:~/pipes$ cat music.txt
Queen
Brel
Led Zeppelin Abba user@debian5:~/pipes$ sort music.txt
Abba
Brel
Led Zeppelin
Queen
But the sort filter has many options to tweak its usage. This example shows sorting different columns (column 1 or column 2).
[user@RHEL4b pipes]$ sort -k1 country.txt
Belgium, Brussels, 10
France, Paris, 60
Germany, Berlin, 100
Iran, Teheran, 70
Italy, Rome, 50
[user@RHEL4b pipes]$ sort -k2 country.txt
Germany, Berlin, 100
Belgium, Brussels, 10
France, Paris, 60 Italy, Rome, 50
Iran, Teheran, 70
The screenshot below shows the difference between an alphabetical sort and a numerical sort (both on the third column).
[user@RHEL4b pipes]$ sort -k3 country.txt
Belgium, Brussels, 10
Germany, Berlin, 100 Italy, Rome, 50
France, Paris, 60
Iran, Teheran, 70
[user@RHEL4b pipes]$ sort -n -k3 country.txt
Belgium, Brussels, 10
Italy, Rome, 50
France, Paris, 60
Iran, Teheran, 70
Germany, Berlin, 100
With uniq you can remove duplicates from a sorted list.
user@debian5:~/pipes$ cat music.txt
Queen
Brel
Queen Abba user@debian5:~/pipes$ sort music.txt
Abba
Brel Queen Queen user@debian5:~/pipes$ sort music.txt |uniq
Abba
Brel
Queen
The find command can be very useful at the start of a pipe to search for files. Here are some examples. You might want to add 2>/dev/null to the command lines to avoid cluttering your screen with error messages.
Find all files in /etc and put the list in etcfiles.txt
find /etc > etcfiles.txt
Find all files of the entire system and put the list in allfiles.txt
find / > allfiles.txt
Find files that end in .conf in the current directory (and all subdirs).
find . -name “*.conf”
Find files of type file (not directory, pipe or etc.) that end in .conf.
find . -type f -name “*.conf”
Find files of type directory that end in .bak .
find /data -type d -name “*.bak”
Find files that are newer than file42.txt
find . -newer file42.txt
The locate tool is very different from find in that it uses an index to locate files. This is a lot faster than traversing all the directories, but it also means that it is always outdated. If the index does not exist yet, then you have to create it (as root on Red Hat Enterprise Linux) with the updatedb command.
[user@RHEL4b ~]$ locate Samba warning: locate: could not open database: /var/lib/slocate/slocate.db:... warning: You need to run the ‘updatedb’ command (as root) to create th...
Please have a look at /etc/updatedb.conf to enable the daily cron job. [user@RHEL4b ~]$ updatedb fatal error: updatedb: You are not authorized to create a default sloc... [user@RHEL4b ~]$ su Password:
[root@RHEL4b ~]# updatedb
[root@RHEL4b ~]#
Most Linux distributions will schedule the updatedb to run once every day. date
The date command can display the date, time, time zone and more.
user@rhel55 ~$ date Sat Apr 17
12:44:30 CEST 2010
A date string can be customised to display the format of your choice. Check the man page for more options.
user@rhel55 ~$ date +’%A %d-%m-%Y’ Saturday
17-04-2010
Time on any Unix is calculated in number of seconds since 1969 (the first second being the first second of the first of January 1970). Use date +%s to display Unix time in seconds.
user@rhel55 ~$ date +%s 1271501080
When will this seconds counter reach two thousand million ?
user@rhel55 ~$ date -d ‘1970-01-01 + 2000000000 seconds’ Wed May 18 04:33:20 CEST 2033
The cal command displays the current month, with the current day highlighted.
user@rhel55 ~$ cal April 2010
Su Mo Tu We Th Fr Sa
1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30
You can select any month in the past or the future.
user@rhel55 ~$ cal 2 1970
February 1970
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
The sleep command is sometimes used in scripts to wait a number of seconds. This example shows a five second sleep.
user@rhel55 ~$ sleep 5 user@rhel55 ~$
The time command can display how long it takes to execute a command. The date command takes only a little time.
user@rhel55 ~$ time date Sat Apr 17 13:08:27 CEST 2010
real 0m0.014s user 0m0.008s sys 0m0.006s
The sleep 5 command takes five real seconds to execute, but consumes little cpu time.
user@rhel55 ~$ time sleep 5
real 0m5.018s user 0m0.005s sys 0m0.011s
This bzip2 command compresses a file and uses a lot of cpu time.
user@rhel55 ~$ time bzip2 text.txt
real 0m2.368s user 0m0.847s sys 0m0.539s
Users never have enough disk space, so compression comes in handy. The gzip command can make files take up less space.
user@rhel55 ~$ ls -lh text.txt
-rw-rw-r-- 1 user user 6.4M Apr 17 13:11 text.txt user@rhel55 ~$ gzip text.txt user@rhel55
~$ ls -lh text.txt.gz
-rw-rw-r-- 1 user user 760K Apr 17 13:11 text.txt.gz
You can get the original back with gunzip.
user@rhel55 ~$ gunzip text.txt.gz user@rhel55
~$ ls -lh text.txt
-rw-rw-r-- 1 user user 6.4M Apr 17 13:11 text.txt
To repeat the last command in bash, type !!. This is pronounced as bang bang.
user@debian5:~/test42$ echo this will be repeated > file42.txt user@debian5:~/test42$ !! echo this will be repeated > file42.txt user@debian5:~/test42$
You can repeat other commands using one bang followed by one or more characters. The shell will repeat the last command that started with those characters.
user@debian5:~/test42$ touch
file42
user@debian5:~/test42$ cat
user@debian5:~/test42$ !to touch file42 user@debian5:~/test42$
file42
To see older commands, use history to display the shell command history (or use history n to see the last n commands).
user@debian5:~/test$ history 10
mkdir test
cd test
touch file1
echo hello > file2
echo It is very cold today > winter.txt
ls
ls -l
cp winter.txt summer.txt
ls -l
history 10
When typing ! followed by the number preceding the command you want repeated, then the shell will echo the command and execute it.
user@debian5:~/test$ !43
ls file1 file2 summer.txt winter.txt
Another option is to use ctrl-r to search in the history. In the screenshot below i only typed ctrl-r followed by four characters apti and it finds the last command containing these four consecutive characters.
user@debian5:~$ (reverse-i-search)`apti’: sudo aptitude
install screen
The $HISTSIZE variable determines the number of commands that will be remembered in your current environment. Most distributions default this variable to 500 or 1000.
user@debian5:~$ echo $HISTSIZE 500
You can change it to any value you like.
user@debian5:~$ HISTSIZE=15000 user@debian5:~$ echo $HISTSIZE 15000
The $HISTFILE variable points to the file that contains your history. The bash shell defaults this value to ~/.bash_history.
user@debian5:~$ echo $HISTFILE /home/user/.bash_history
A session history is saved to this file when you exit the session!
Closing a gnome-terminal with the mouse, or typing reboot as root will NOT save your terminal’s history.
The number of commands kept in your history file can be set using $HISTFILESIZE.
user@debian5:~$ echo $HISTFILESIZE 15000
You can prevent a command from being recorded in history using a space prefix.
user@debian8:~/github$ echo abc abc
user@debian8:~/github$ echo def def
user@debian8:~/github$ echo ghi ghi user@debian8:~/github$ history 3
echo abc
echo ghi
It is possible to use regular expressions when using the bang to repeat commands. The screenshot below switches 1 into 2.
user@debian5:~/test$ cat
user@debian5:~/test$ !c:s/1/2 cat file2
hello
user@debian5:~/test$
file1
Repeating a command in the Korn shell is very similar. The Korn shell also has the history command, but uses the letter r to recall lines from history.
This screenshot shows the history command. Note the different meaning of the parameter.
$ history 17
clear
echo hoi
history 12
echo world
history 17
Repeating with r can be combined with the line numbers given by the history command, or with the first few letters of the command.
$ r e echo world world $ cd /etc
$ r cd /etc
$
This little chapter will teach you how to identify your user account on a Unix computer using commands like who am i, id, and more.
In a second part you will learn how to become another user with the su command.
And you will learn how to run a program as another user with sudo.
The whoami command tells you your username.
[user@centos7 ~]$ whoami user [user@centos7 ~]$
The who command will give you information about who is logged on the system.
[user@centos7 ~]$ who root pts/0 2014-10-10 23:07 (10.104.33.101) user pts/1
2014-10-10 23:30 (10.104.33.101) laura pts/2 2014-10-10 23:34 (10.104.33.96) tania pts/3 2014-10-10 23:39 (10.104.33.91) [user@centos7 ~]$
With who am i the who command will display only the line pointing to your current session.
[user@centos7 ~]$ who am i
user pts/1 2014-10-10 23:30 (10.104.33.101) [user@centos7 ~]$
The w command shows you who is logged on and what they are doing.
[user@centos7 ~]$ w
23:34:07 up 31 min, 2 users, load average: 0.00, 0.01, 0.02
USER TTY LOGIN@ IDLE JCPU PCPU WHAT root pts/0 23:07 15.00s 0.01s 0.01s top user pts/1 23:30 7.00s 0.00s 0.00s w [user@centos7 ~]$
The id command will give you your user id, primary group id, and a list of the groups that you belong to.
user@debian7:~$ id uid=1000(user) gid=1000(user) groups=1000(user)
On RHEL/CentOS you will also get SELinux context information with this command.
[root@centos7 ~]# id uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r\ :unconfined_t:s0-s0:c0.c1023
The su command allows a user to run a shell as another user.
laura@debian7:~$ su tania Password:
tania@debian7:/home/laura$
Yes you can also su to become root, when you know the root password.
laura@debian7:~$ su root Password:
root@debian7:/home/laura#
You need to know the password of the user you want to substitute to, unless your are logged in as root. The root user can become any existing user without knowing that user’s password.
root@debian7:~# id uid=0(root) gid=0(root) groups=0(root) root@debian7:~# su -
valentina valentina@debian7:~$
By default, the su command maintains the same shell environment. To become another user and also get the target user’s environment, issue the su - command followed by the target username.
root@debian7:~# su laura laura@debian7:/root$
exit
exit root@debian7:~# su - laura laura@debian7:~$ pwd /home/laura
When no username is provided to su or su -, the command will assume root is the target.
tania@debian7:~$ su - Password: root@debian7:~#
The sudo program allows a user to start a program with the credentials of another user. Before this works, the system administrator has to set up the /etc/sudoers file. This can be useful to delegate administrative tasks to another user (without giving the root password).
The screenshot below shows the usage of sudo. User user received the right to run useradd with the credentials of root. This allows user to create new users on the system without becoming root and without knowing the root password.
First the command fails for user.
user@debian7:~$ /usr/sbin/useradd -m valentina useradd: Permission denied. useradd: cannot lock /etc/passwd; try again later.
But with sudo it works.
user@debian7:~$ sudo /usr/sbin/useradd -m valentina
[sudo] password for user: user@debian7:~$
Check the man page of visudo before playing with the /etc/sudoers file. Editing the sudoers is out of scope for this fundamentals book.
user@rhel65:~$ apropos visudo
visudo (8) - edit the sudoers file user@rhel65:~$
On some Linux systems like Debian stable and XDebian stable, the root user does not have a password set. This means that it is not possible to login as root (extra security). To perform tasks as root, the first user is given all sudo rights via the /etc/sudoers. In fact all users that are members of the admin group can use sudo to run all commands as root.
root@predator-pc~# grep admin /etc/sudoers
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
The end result of this is that the user can type sudo su - and become root without having to enter the root password. The sudo command does require you to enter your own password. Thus the password prompt in the screenshot below is for sudo, not for su.
user@predator-pc~$ sudo su - Password: root@predator-pc~#
Using sudo without authorization will result in a severe warning:
user@rhel65:~$ sudo su -
We trust you have received the usual lecture from the local System Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo] password for user: user is not in the sudoers file. This incident will be reported. user@rhel65:~$
The root user can see this in the /var/log/secure on Red Hat and in /var/log/auth.log on Debian).
root@rhel65:~# tail /var/log/secure | grep sudo | tr -s ‘ ‘
Apr 13 16:03:42 rhel65 sudo: user : user NOT in sudoers ; TTY=pts/0 ; PWD=\
/home/user ; USER=root ; COMMAND=/bin/su - root@rhel65:~#
This chapter will teach you how to use useradd, usermod and userdel to create, modify and remove user accounts. You will need root access on a Linux computer to complete this chapter.
User management on Linux can be done in three complementary ways. You can use the graphical tools provided by your distribution. These tools have a look and feel that depends on the distribution. If you are a novice Linux user on your home system, then use the graphical tool that is provided by your distribution. This will make sure that you do not run into problems.
Another option is to use command line tools like useradd, usermod, gpasswd, passwd and others. Server administrators are likely to use these tools, since they are familiar and very similar across many different distributions. This chapter will focus on these command line tools.
A third and rather extremist way is to edit the local configuration files directly using vi (or vipw/vigr). Do not attempt this as a novice on production systems!
The local user database on Linux (and on most Unixes) is /etc/passwd.
[root@RHEL5 ~]# tail /etc/passwd inge:x:518:524:art dealer:/home/inge:/bin/ksh ann:x:519:525:flute player:/home/ann:/bin/bash frederik:x:520:526:rubius poet:/home/frederik:/bin/bash steven:x:521:527:roman emperor:/home/steven:/bin/bash pascale:x:522:528:artist:/home/pascale:/bin/ksh geert:x:524:530:kernel developer:/home/geert:/bin/bash wim:x:525:531:master damuti:/home/wim:/bin/bash sandra:x:526:532:radish stresser:/home/sandra:/bin/bash
As you can see, this file contains seven columns separated by a colon. The columns contain the username, an x, the user id, the primary group id, a description, the name of the home directory, and the login shell.
More information can be found by typing man 5 passwd.
[root@RHEL5 ~]# man 5 passwd
The root user also called the superuser is the most powerful account on your Linux system. This user can do almost anything, including the creation of other users. The root user always has userid 0 (regardless of the name of the account).
[root@RHEL5 ~]# head -1 /etc/passwd root:x:0:0:root:/root:/bin/bash
You can add users with the useradd command. The example below shows how to add a user named yanina (last parameter) and at the same time forcing the creation of the home directory (-m), setting the name of the home directory (-d), and setting a description (-c).
[root@RHEL5 ~]# useradd -m -d /home/yanina -c “yanina wickmayer” yanina [root@RHEL5 ~]# tail -1 /etc/passwd yanina:x:529:529:yanina wickmayer:/home/yanina:/bin/bash
The user named yanina received userid 529 and primary group id 529.
Both Red Hat Enterprise Linux and Debian/Debian stable have a file called /etc/default/useradd that contains some default user options. Besides using cat to display this file, you can also use useradd -D.
[root@RHEL4 ~]# useradd -D
GROUP=100
HOME=/home INACTIVE=-1
EXPIRE=
SHELL=/bin/bash
SKEL=/etc/skel
You can delete the user yanina with userdel. The -r option of userdel will also remove the home directory.
[root@RHEL5 ~]# userdel -r yanina
You can modify the properties of a user with the usermod command. This example uses usermod to change the description of the user harry.
[root@RHEL4 ~]# tail -1 /etc/passwd harry:x:516:520:harry potter:/home/harry:/bin/bash
[root@RHEL4 ~]# usermod -c ‘wizard’ harry
[root@RHEL4 ~]# tail -1 /etc/passwd harry:x:516:520:wizard:/home/harry:/bin/bash
The easiest way to create a home directory is to supply the -m option with useradd (it is likely set as a default option on Linux).
A less easy way is to create a home directory manually with mkdir which also requires setting the owner and the permissions on the directory with chmod and chown (both commands are discussed in detail in another chapter).
[root@RHEL5 ~]# mkdir /home/laura
[root@RHEL5 ~]# chown laura:laura /home/laura
[root@RHEL5 ~]# chmod 700 /home/laura [root@RHEL5 ~]# ls -ld /home/laura/
drwx------ 2 laura laura 4096 Jun 24 15:17 /home/laura/
When using useradd the -m option, the /etc/skel/ directory is copied to the newly created home directory. The /etc/skel/ directory contains some (usually hidden) files that contain profile settings and default values for applications. In this way /etc/skel/ serves as a default home directory and as a default user profile.
[root@RHEL5 ~]# ls -la /etc/skel/ total 48 drwxr-xr-x 2 root root 4096 Apr 1 00:11 . drwxr-xr-x 97 root root 12288 Jun 24 15:36 ..
-rw-r--r-- 1 root root 24 Jul 12 2006 .bash_logout
-rw-r--r-- 1 root root 176 Jul 12 2006 .bash_profile
-rw-r--r-- 1 root root 124 Jul 12 2006 .bashrc
The -r option of userdel will make sure that the home directory is deleted together with the user account.
[root@RHEL5 ~]# ls -ld /home/wim/ drwx------ 2 wim wim 4096 Jun 24 15:19 /home/wim/
[root@RHEL5 ~]# userdel -r wim
[root@RHEL5 ~]# ls -ld /home/wim/ ls:
/home/wim/: No such file or directory
The /etc/passwd file specifies the login shell for the user. In the screenshot below you can see that user annelies will log in with the /bin/bash shell, and user laura with the /bin/ksh shell.
[root@RHEL5 ~]# tail -2 /etc/passwd annelies:x:527:533:sword fighter:/home/annelies:/bin/bash laura:x:528:534:art dealer:/home/laura:/bin/ksh
You can use the usermod command to change the shell for a user.
[root@RHEL5 ~]# usermod -s /bin/bash laura
[root@RHEL5 ~]# tail -1 /etc/passwd dealer:/home/laura:/bin/bash
laura:x:528:534:art
Users can change their login shell with the chsh command. First, user harry obtains a list of available shells (he could also have done a cat /etc/shells) and then changes his login shell to the Korn shell (/bin/ksh). At the next login, harry will default into ksh instead of bash.
[laura@centos7 ~]$ chsh -l
/bin/sh
/bin/bash
/sbin/nologin /usr/bin/sh
/usr/bin/bash
/usr/sbin/nologin
/bin/ksh
/bin/tcsh
/bin/csh
[laura@centos7 ~]$
Note that the -l option does not exist on Debian and that the above screenshot assumes that ksh and csh shells are installed.
The screenshot below shows how laura can change her default shell (active on next login).
[laura@centos7 ~]$ chsh -s /bin/ksh Changing shell for laura.
Password:
Shell changed.
This chapter will tell you more about passwords for local users.
Three methods for setting passwords are explained; using the passwd command, using openssel passwd, and using the crypt function in a C program.
The chapter will also discuss password settings and disabling, suspending or locking accounts.
Passwords of users can be set with the passwd command. Users will have to provide their old password before twice entering the new one.
[tania@centos7 ~]$ passwd
Changing password for user tania.
Changing password for tania.
(current) UNIX password:
New password:
BAD PASSWORD: The password is shorter than 8 characters New password:
BAD PASSWORD: The password is a palindrome New password:
BAD PASSWORD: The password is too similar to the old one passwd:
Have exhausted maximum number of retries for service
As you can see, the passwd tool will do some basic verification to prevent users from using too simple passwords. The root user does not have to follow these rules (there will be a warning though). The root user also does not have to provide the old password before entering the new password twice.
root@debian7:~# passwd tania Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully
User passwords are encrypted and kept in /etc/shadow. The /etc/shadow file is read only and can only be read by root. We will see in the file permissions section how it is possible for users to change their password. For now, you will have to know that users can change their password with the /usr/bin/passwd command.
[root@centos7 ~]# tail -4 /etc/shadow
user:$6$ikp2Xta5BT.Tml.p$2TZjNnOYNNQKpwLJqoGJbVsZG5/Fti8ovBRd.VzRbiDSl7TEq\ IaSMH.TeBKnTS/SjlMruW8qffC0JNORW.BTW1:16338:0:99999:7:::
tania:$6$8Z/zovxj$9qvoqT8i9KIrmN.k4EQwAF5ryz5yzNwEvYjAa9L5XVXQu.z4DlpvMREH\ eQpQzvRnqFdKkVj17H5ST.c79HDZw0:16356:0:99999:7:::
laura:$6$glDuTY5e$/NYYWLxfHgZFWeoujaXSMcR.Mz.lGOxtcxFocFVJNb98nbTPhWFXfKWG\ SyYh1WCv6763Wq54.w24Yr3uAZBOm/:16356:0:99999:7:::
valentina:$6$jrZa6PVI$1uQgqR6En9mZB6mKJ3LXRB4CnFko6LRhbh.v4iqUk9MVreui1lv7\ GxHOUDSKA0N55ZRNhGHa6T2ouFnVno/0o1:16356:0:99999:7:::
[root@centos7 ~]#
The /etc/shadow file contains nine colon separated columns. The nine fields contain (from left to right) the user name, the encrypted password (note that only inge and laura have an encrypted password), the day the password was last changed (day 1 is January 1, 1970), number of days the password must be left unchanged, password expiry day, warning number of days before password expiry, number of days after expiry before disabling the account, and the day the account was disabled (again, since 1970). The last field has no meaning yet.
All the passwords in the screenshot above are hashes of hunter2.
Passwords are stored in an encrypted format. This encryption is done by the crypt function. The easiest (and recommended) way to add a user with a password to the system is to add the user with the useradd -m user command, and then set the user’s password with passwd.
[root@RHEL4 ~]# useradd -m xavier
[root@RHEL4 ~]# passwd xavier Changing password for user xavier.
New UNIX password: Retype new UNIX password:
passwd: all authentication tokens updated successfully.
[root@RHEL4 ~]#
Another way to create users with a password is to use the -p option of useradd, but that option requires an encrypted password. You can generate this encrypted password with the openssl passwd command.
The openssl passwd command will generate several distinct hashes for the same password, for this it uses a salt.
user@rhel65:~$ openssl passwd hunter2
86jcUNlnGDFpY user@rhel65:~$ openssl passwd hunter2 Yj7mDO9OAnvq6 user@rhel65:~$ openssl passwd hunter2
YqDcJeGoDbzKA user@rhel65:~$
This salt can be chosen and is visible as the first two characters of the hash.
user@rhel65:~$ openssl passwd -salt 42 hunter2 42ZrbtP1Ze8G.
user@rhel65:~$ openssl passwd -salt 42 hunter2 42ZrbtP1Ze8G.
user@rhel65:~$ openssl passwd -salt 42 hunter2
42ZrbtP1Ze8G. user@rhel65:~$
This example shows how to create a user with password.
root@rhel65:~# useradd -m -p $(openssl passwd hunter2) mohamed
Users can be listed in groups. Groups allow you to set permissions on the group level instead of having to set permissions for every individual user.
Every Unix or Linux distribution will have a graphical tool to manage groups. Novice users are advised to use this graphical tool. More experienced users can use command line tools to manage users, but be careful: Some distributions do not allow the mixed use of GUI and CLI tools to manage groups (YaST in Novell Suse). Senior administrators can edit the relevant files directly with vi or vigr.
Groups can be created with the groupadd command. The example below shows the creation of five (empty) groups.
root@predator-pc~#
groupadd
tennis
root@predator-pc~#
groupadd
football
root@predator-pc~#
groupadd
snooker
root@predator-pc~#
groupadd
formula1
root@predator-pc~# groupadd salsa
Users can be a member of several groups. Group membership is defined by the /etc/group file.
root@predator-pc~# tail -5 /etc/group tennis:x:1006: football:x:1007: snooker:x:1008: formula1:x:1009: salsa:x:1010: root@predator-pc~#
The first field is the group’s name. The second field is the group’s (encrypted) password (can be empty). The third field is the group identification or GID. The fourth field is the list of members, these groups have no members.
A user can type the groups command to see a list of groups where the user belongs to.
[harry@RHEL4b ~]$ groups harry sports
[harry@RHEL4b ~]$
Group membership can be modified with the useradd or usermod command.
root@predator-pc~# usermod -a -G tennis inge root@predator-pc~# usermod -a -G tennis katrien root@predator-pc~# usermod -a -G salsa katrien root@predator-pc~# usermod -a -G snooker sandra root@predator-pc~# usermod -a -G formula1 annelies root@predator-pc~# tail -5 /etc/group
tennis:x:1006:inge,katrien football:x:1007:
snooker:x:1008:sandra formula1:x:1009:annelies salsa:x:1010:katrien root@predator-pc~#
Be careful when using usermod to add users to groups. By default, the usermod command will remove the user from every group of which he is a member if the group is not listed in the command! Using the -a (append) switch prevents this behaviour.
You can change the group name with the groupmod command.
root@predator-pc~# groupmod -n darts snooker root@predator-pc~# tail -5 /etc/group tennis:x:1006:inge,katrien football:x:1007: formula1:x:1009:annelies salsa:x:1010:katrien darts:x:1008:sandra
You can delegate control of group membership to another user with the gpasswd command. In the example below we delegate permissions to add and remove group members to serena for the sports group. Then we su to serena and add harry to the sports group.
[root@RHEL4b ~]# gpasswd -A serena sports
[root@RHEL4b ~]# su - serena [serena@RHEL4b ~]$ id harry uid=516(harry) gid=520(harry) groups=520(harry)
[serena@RHEL4b ~]$ gpasswd -a harry sports
Adding user harry to group sports [serena@RHEL4b ~]$ id harry uid=516(harry) gid=520(harry) groups=520(harry),522(sports)
[serena@RHEL4b ~]$ tail -1 /etc/group sports:x:522:serena,venus,harry [serena@RHEL4b ~]$
Group administrators do not have to be a member of the group. They can remove themselves from a group, but this does not influence their ability to add or remove members.
[serena@RHEL4b ~]$ gpasswd -d serena sports
Removing user serena from group sports [serena@RHEL4b ~]$ exit
Information about group administrators is kept in the /etc/gshadow file.
[root@RHEL4b ~]# tail -1 /etc/gshadow sports:!:serena:venus,harry [root@RHEL4b ~]#
To remove all group administrators from a group, use the gpasswd command to set an empty administrators list.
[root@RHEL4b ~]# gpasswd -A ““ sports
You can start a child shell with a new temporary primary group using the newgrp command.
root@rhel65:~# mkdir prigroup
root@rhel65:~# cd prigroup/ root@rhel65:~/prigroup# touch standard.txt root@rhel65:~/prigroup# ls -l total 0
-rw-r--r--. 1 root root 0 Apr 13 17:49 standard.txt root@rhel65:~/prigroup# echo $SHLVL
1
root@rhel65:~/prigroup# newgrp tennis
root@rhel65:~/prigroup# echo $SHLVL
2
root@rhel65:~/prigroup# touch newgrp.txt root@rhel65:~/prigroup# ls -l total 0
-rw-r--r--. 1 root tennis 0 Apr 13 17:49 newgrp.txt -rw-r--r--. 1 root root 0 Apr 13 17:49 standard.txt root@rhel65:~/prigroup# exit exit
root@rhel65:~/prigroup#
Logged on users have a number of preset (and customized) aliases, variables, and functions, but where do they come from ? The shell uses a number of startup files that are executed (or rather sourced) whenever the shell is invoked. What follows is an overview of startup scripts.
Both the bash and the ksh shell will verify the existence of /etc/profile and source it if it exists.
When reading this script, you will notice (both on Debian and on Red Hat Enterprise Linux) that it builds the PATH environment variable (among others). The script might also change the PS1 variable, set the HOSTNAME and execute even more scripts like /etc/inputrc This screenshot uses grep to show PATH manipulation in /etc/profile on Debian.
root@debian7:~# grep PATH /etc/profile
PATH=“/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin” PATH=“/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games” export PATH root@debian7:~#
This screenshot uses grep to show PATH manipulation in /etc/profile on RHEL7/CentOS7.
[root@centos7 ~]# grep PATH /etc/profile case “:${PATH}:” in
PATH=$PATH:$1 PATH=$1:$PATH export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL [root@centos7 ~]#
The root user can use this script to set aliases, functions, and variables for every user on the system.
When this file exists in the home directory, then bash will source it. On Debian Linux 5/6/7 this file does not exist by default.
RHEL7/CentOS7 uses a small ~/.bash_profile where it checks for the existence of ~/.bashrc and then sources it. It also adds $HOME/bin to the $PATH variable.
[root@rhel7 ~]# cat /home/user/.bash_profile
# .bash_profile
# Get the aliases and functions if [ -f ~/.bashrc ]; then
. ~/.bashrc fi # User specific environment and startup programs
PATH=$PATH:$HOME/.local/bin:$HOME/bin
export PATH [root@rhel7 ~]#
When .bash_profile does not exist, then bash will check for ~/.bash_login and source it.
Neither Debian nor Red Hat have this file by default.
When neither ~/.bash_profile and ~/.bash_login exist, then bash will verify the existence of ~/.profile and execute it. This file does not exist by default on Red Hat. On Debian this script can execute ~/.bashrc and will add $HOME/bin to the $PATH variable.
root@debian7:~# tail -11 /home/user/.profile if [ -n “$BASH_VERSION” ]; then
# include .bashrc if it exists
if [ -f “$HOME/.bashrc” ]; then . “$HOME/.bashrc” fi fi
# set PATH so it includes user’s private bin if it exists
if [ -d “$HOME/bin” ] ; then
PATH=“$HOME/bin:$PATH” fi
RHEL/CentOS does not have this file by default.
The ~/.bashrc script is often sourced by other scripts. Let us take a look at what it does by default.
Red Hat uses a very simple ~/.bashrc, checking for /etc/bashrc and sourcing it. It also leaves room for custom aliases and functions.
[root@rhel7 ~]# cat /home/user/.bashrc
# .bashrc
# Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc
fi
# Uncomment the following line if you don’t like systemctl’s auto-paging feature:
# export SYSTEMD_PAGER=
# User specific aliases and functions
On Debian this script is quite a bit longer and configures $PS1, some history variables and a number af active and inactive aliases.
root@debian7:~# wc -l /home/user/.bashrc
110 /home/user/.bashrc
When exiting bash, it can execute ~/.bash_logout.
Debian use this opportunity to clear the console screen.
serena@deb503:~$ cat .bash_logout # ~/.bash_logout: executed by bash(1) when login shell exits. # when leaving the console clear the screen to increase privacy
if [ “$SHLVL” = 1 ]; then
[ -x /usr/bin/clear_console ] && /usr/bin/clear_console -q fi
Red Hat Enterprise Linux 5 will simple call the /usr/bin/clear command in this script.
[serena@rhel53 ~]$ cat .bash_logout
# ~/.bash_logout /usr/bin/clear
Red Hat Enterprise Linux 6 and 7 create this file, but leave it empty (except for a comment).
user@rhel65:~$ cat .bash_logout
# ~/.bash_logout
