Tasks in Bash

Bash is the GNU shell, commonly used in Linux. This guide is intended to demonstrate simple (though not necessarily obvious!) tasks using the Bash shell by example; it is not a manual. Some of this may be applicable to other shells, but if you're using an obscure shell you're unlikely to need this information... I'm still working on this; let me know if you think of anything I should add.

Commands to by typed at the shell are preceded here by '$'; anything else is intended to be the output from the command.

Finding and renaming files

Finding files

Many Linux systems will have either locate or slocate installed, which search a database of the system's files for a given pattern. (If slocate is installed, locate is likely to link to it so the actual command that runs is always slocate.) This is fast, but its accuracy depends on when the database was last rebuilt (typically daily; this is not controllable by the user).

$ locate resolv.conf
/etc/resolv.conf

The locate database will not contain files that are not world-readable. If slocate is installed, its database will contain all files readable by the user.

Often more useful is the slower find method, which allows the user to specify the root directory for the search and can read all files readable by the user:

$ find . -name 'lostfile'   # first argument is root path for searching
./files/lostfile

Find can also execute a command and pass to it matching filenames:

$ find . -name 'lostfile' -exec grep regex {} \;

In the above example, '{}' is replaced by find with the matching filename, and all text following '-exec' is passed to the command until the ';' is reached. (Note that the ';' must be escaped to prevent expansion by the shell.) To print the matching file name in addition to the command output, use '-print' before '-exec'.

Search paths

The Bash path is a list of directories that are searched when a command is typed. By default Bash does not include the current directory (.). To execute a.out, use:

$ ./a.out

To add the current directory to the path, use:

PATH="${PATH}":.

You should never specify "." before the existing path, as this could cause the wrong commands to be inadvertently run. For instance, if the current directory contained a file called rm, it would be run in preference to the command of that name to delete files. The same syntax can be used to place a user's own 'bin' directory in the search path; again it should be placed before the existing path.

To make a change to the path permanent, the file to edit is ~/.bash_profile. Add to it somewhere the 'PATH=' line, and it will be executed every time a new Bash is started.

To find out precisely where in the path the command perl is, use:

$ which perl
/usr/bin/perl

Renaming files

Renaming a single file is simple:

$ mv oldfilename newfilename

If newfile exists, it will by default be overwritten. Some systems are configured by default to prompt before doing this; see aliases to change this behaviour.

It can be useful to rename several files at once; for example to change all files ending in .htm to .html. This cannot be done using a simple wildcard. Instead:

$ for a in *.htm; do mv $a "$a"l; done

(Note that it is necessary to use "$a" and not '$a'. Only the former expands the variable before passing to 'mv'.)

For more complex renaming operations, regular expressions are needed. Bash can handle these itself, but if you are already familiar with regular expressions it may be more intuitive to use sed. To change *.html to *.txt use:

$ for a in *.html; do mv "$a" "$(echo "$a" | sed 's/html$/txt/')"; done

Here, the expression 'html$' ensures that sed only locates that string at the end of the name. Use of sed this way allows for arbitrary renaming operations to be performed using regular expressions. For simple renamings, using Bash may be easier; the above example may be written:

$ for a in *.html; do mv "$a" "${a/html/txt}"; done

but note that this does not anchor the regular expression to the end of the filename. For 'greedy' matching (every occurrence substituted, rather than only the first), use:

$ for a in *.html; do mv "$a" "${a//html/txt}"; done

Input/output features

Redirection & pipes

When a command is entered, its output is sent by default to the terminal. This is known as standard output. Any errors are generally also sent to the screen, and they are standard error. It is possible to redirect either or both of these to files:

$ ls > ls.txt       #save standard output to ls.txt
$ ls 2> ls.txt      #save standard error to ls.txt
$ ls &> ls.txt      #save both standard output and error to ls.txt
$ ls &> /dev/null   #discard all output

By default this will overwrite the file if it exists already. To append to the file (or create it if it does not exist), use:

$ ls >> ls.txt

To send the output of a command directly to another command, use a pipe:

$ ls | grep regex

The above will send the output of ls to grep and will only show files matching the regular expression given.

To send a file to another command, the following are equivalent:

$ cat ls.txt | grep regex 
$ grep regex < ls.txt
$ grep regex ls.txt        # only if command accepts a file as argument
$ grep regex - < ls.txt    # use '-' as filename to specify stdin if argument is mandatory

Quoting

It is useful to understand the way Bash handles quoting; that is, how to remove the 'special' meaning from characters such as "|" and "#" that are interpreted by the shell.

To quote a single character, use a single backslash. In the following example, the "#" is taken by the shell to denote a comment unless it is 'escaped' (quoted with a "\").

$ echo #1

$ echo \#1
#1

Anything in single quotation marks ' ' is interpreted literally. You can not embed more single quotation marks within them.

$ echo '# \ | $ & > ('
# \ | $ & > (

Double quotation marks " " are used to quote all characters except "$", "`" and "\". Variable and backtick expansion therefore still occur. The "\" can be used to escape any of those characters and the " itself.

$ echo "Terminal type is \"$TERM\""
Terminal type is "xterm"
$ echo "The terminal type variable is \$TERM"
The terminal type variable is $TERM
$ echo "The date is `date +%d/%m/%y`"
The date is 22/01/04

Note that quoting protects characters from Bash, but certain commands require their own quoting, particularly in regular expressions. Compare the following, where sed interprets "(" literally but "[" must be escaped: (Note that in Perl, a literal "(" must be escaped; this is a difference between BRE and PCRE.)

$ echo '[Hello] world' | sed 's/\[\|\]//g'
Hello world
$ echo '(Hello) world' | sed 's/(\|)//g'
Hello world

Default editor and vi

vi is a powerful but not particularly intuitive text editor in which the user can easily become inadvertently trapped. This may happen because no preferred text editor has been set by the user, and the system defaults to vi. Set the editor to a more sensible program:

$ EDITOR=emacs

The key-sequence to exit vi is 'ESC :q!' followed by enter. This will not make any changes to the file being edited.

Saving time

Tab completion and history expansion

Bash has various features to allow the user to save time. Tab completion will complete file paths, command names, and (if enabled) can even choose files based on their extensions. For example:

$ ls
a.html  b.txt  c.pdf
$ xpdf <TAB>

will choose c.pdf without the user entering the filename. The general behaviour of tab completion is easily learnt by trial and error.

History expansion saves time when entering commands that have been typed previously. Typing

$ !something

will execute the command last typed that began with 'something'. The entire history list can be viewed using the command history; note that each command has an associated number and you can recall any command. For example:

$ history | tail
[...] 
607 ls 
608 xpdf c.pdf 
609 history | tail 
$ !607 
ls 
a.html b.txt c.pdf 

Bash can also perform incremental history searches. Press ctrl-r for a reverse search (searching backwards from the most recent command), and ctrl-s for a forward search (searching forwards from the oldest command). As soon as your typing matches a command in the history, the command will be displayed and can be chosen by pressing enter. To cancel, use ctrl-g.

Recalling the last command and its arguments

The last-typed command can be accessed with '!!':

$ ls -ltr /usr/share
[output from ls]
$ !!
ls -ltr /usr/share         #Bash repeats what it's about to do
[output from ls]           #Runs the last command again

The last argument from the last command can be accessed with '!$':

$ cp oldfile.dat newfile.dat
$ cat !$
cat newfile.dat           #This is what Bash does
[contents of newfile.dat]

More powerfully, the n-th argument from the last command can be accessed with '!:n' and the n-th to m-th arguments with '!:n-m':

$ cp oldfile.dat newfile.dat
$ rm !:1
rm oldfile.dat
$ touch file1 file2 file3
$ rm !:1-2
rm file1 file2

Aliases

An alias is an instruction to Bash to substitute something entered at the command line for something else. For example, the following command:

$ alias rm='rm -i'

will change the behaviour of rm so that it will always prompt before removing files. Note that in general any options appearing later override those appearing earlier, so typing

$ rm -f a.out  #alias makes this actually mean rm -i -f a.out

will not ask for confirmation, because the later option takes precedence.

To get a list of all aliases currently in force, enter:

$ alias

To remove an alias, such as the one added previously, use:

$ unalias rm

To make aliases 'permanent', add the alias commands to ~/.bashrc.