Basic Linux Commands and Capabilities

Abstract

The original Unix operating system and its successor Linux were designed to support the common tasks and needs of software developers and computer users. This laboratory exercise reviews some basic commands and capabilities that match likely needs of CSC 161 students.

Introduction

The Linux operating system has evolved from Unix, an operating system developed at AT&T Bell Labs in 1969. At the time, Bell Labs was a major research and development enterprise, averaging approximately a patent a day over its first 40 years of existence. With this impressive track record, the company was delighted to encourage development of a computing environment that would support research and development; common computing tasks needed to be easy and quick. As a result, the original Unix was extremely efficient, although sometimes cryptic. While professionals found Unix wonderfully powerful, beginners sometimes found that it took some time to master.

The current Linux operating system maintains the efficiency and power of the original Unix, with many extensions and revisions for modern needs. At the same time, Linux also has interfaces that make it much easier to use than the original Unix—although it sometimes still reflects the original cryptic style.

This lab covers the following basic capabilities and commands.

The Terminal Window

When Unix was initially developed, users entered information by typing on a keyboard, and results were printed on paper or a screen. Almost all work was character based, with very little use of graphics. (Images require extensive memory resources that were beyond the capabilities of most early computers.) With this history, many of the early Unix capabilities depended upon typing commands into a terminal. On current machines, the terminal window continues to provide a wide range of powerful capabilities for many processing needs.

Opening the Terminal Window

Although you may be accustomed to a graphical user interface (GUI) from your past computer usage, you also need to become comfortable with a command-line interface. Historically, commands were typed into windows (actually, the entire terminal had just one screen—the only window). Unix/Linux provides several powerful tools for this type of interaction. To run various commands, you must invoke them by name. The computer program that reads and responds to such invocations is called the shell, and your interactions with the shell take place in a window generated by a program called a terminal emulator.

You may already have a terminal window open on your screen. If not, you can start one at any time by moving the pointer onto the small monitor icon on the bottom row of the front panel, and click with the left mouse button. Shortly a window will appear, displaying the shell prompt—the name of the workstation on which the shell is running, followed by a percentage sign (%) or dollar sign ($). This prompt indicates that the shell is ready to receive instructions.

You enter such instructions using the keyboard. Move the mouse pointer into the terminal window to make it active. (You many need to click on the window, once your mouse is in the desired area.) The window frame changes color, indicating that the window has become active.

If you want to set the window aside for the moment, with the possibility of returning to it later, look closely at the upper right-hand corner of the window, where the frame contains a small bar or underscore character; if you move the pointer into that square and click on the left mouse button, you minimize the window, closing it up into a small rectangular icon along the bottom control panel on the screen. A minimized window can be restored by moving the pointer onto its icon and clicking the left mouse button twice in rapid succession.

Changing Your Password

You should change the password associated with your account shortly after you receive it and every few months thereafter. The /net/bin/passphrase program lets you make this change.

1. Choose a new password. Make it something that you can easily remember, but not an English word or a name, since it is easy for system crackers to break in by guessing your password if you choose it from one of those categories. Using a multi-word phrase can be helpful.
2. Open a terminal window, move the pointer into it, and type /net/bin/passphrase. The passphrase program will prompt you once for your old password—the one you logged in with — and twice for your new password. If you give your old password correctly and the two copies of your new password match, the program will substitute the new password for the old one in the table that the login program consults. The old password will be discarded and will not be recognized in subsequent logins. (If the attempt to change the password fails for any reason, however, the old password will be retained.)

After running /net/bin/passphrase, the shell takes over again and issues another prompt. You can invoke as many programs as you like from the shell, one after another.

Cutting and Pasting Between Windows

In working on a workstation on the MathLAN (Campus Linux Network), you can select and paste material from one window to another. Through this semester, this can be particularly helpful, when you want to work with material that appears on the lab directions in the Iceweasel web browser.

To select material from Iceweasel (or any other window), move the cursor to the beginning of a section and push down the left mouse button. Then, holding the button down, move the mouse to the end of the section. (The entire section now should be highlighted.) When the desired section is highlighted, stop pressing on the left mouse button—the section should stay highlighted. Now move the mouse to where you want to paste the material, and click the middle mouse button.

Terminal Utilities

Although the basic running of programs within a terminal window may seem quite straightforward, a few nifty shortcuts can simplify your work considerably.

Stopping a Long or Infinite Loop

Suppose you wrote a program that contained an infinite loop, and you ran it from the terminal window. How could you stop the program?

One way is to close the window, but a better way is to type ctrl+c (i.e., hold down the ctrl key while you type c). This should kill the program without making you re-open the window and re-navigate to the directory you were working in.

The Arrow Keys

Suppose you want to run a command that you ran a short while ago? If the command is complicated, it would be nice not to have to re-type it, and you don't have to! Within the terminal window, this is accomplished using the ↑ (up arrow) key to retrieve previous commands. Once you have viewed some earlier commands, note that the ↓ (down arrow) also works, which can be helpful if you go too far.

The history Command

But what if the command you want was several commands ago, say 10? Who wants to type that many arrows? To see another option, first type the following to get a list of the commands you have used most recently.

history

Typically, a terminal window remembers the last 50 commands (although a user can change this number). At the start, the history list might look something like this.

505 sleep 10
506 history

In this listing, each command is numbered for future reference. For example, with this listing, if you want to re-issue the sleep command, you could do so by typing "!505" to repeat command number 505. Better yet (since you don't need to know the command number), you can repeat the command with "!s". This will re-issue the most recent command that begins with the prefix "s". Note that the prefix can be of any length; it is not limited to one character.

cat

It is often convenient to look at the contents of a text file without having to open it in an editor. The simplest command for this purpose is cat. For example,

cat .bashrc

would display the contents of the .bashrc file in your terminal window. Although cat works well for short files that can be viewed all on one screen, we will see shortly that other commands may work better for longer files.

As an aside, note that the command name cat stands for "concatenate" for reasons we will explore later.

Autocompletion

Another very useful keyboard shortcut is called autocompletion. Suppose you want to do something with a file called my-test-file in your home directory. That file name seems a bit long to type very often, so it's nice that you don't have to.

To use autocompletion, type the command and the first part of the file name (e.g., cat ~/my-t), followed by pressing the tab key. You will find that when you press tab, the system completes the (unique) file name for you.

If there isn't a unique completion, the system may add a few letters and beep. This tells you that there are multiple files that have these beginning letters. On the other hand, if no files begin with the given file name, the system likely will just beep at you.

Background Processes

Sometimes you want to start a task in a terminal window and the continue with other work. For example, suppose you want to read a C program quarts-espeak.c with the emacs editor. Type

emacs /home/weinman/public_html/courses/CSC161/2017F/modules/getting-started/src/quarts-espeak.c

This will open emacs just fine, but now switch back to your terminal window. You will notice that it is unavailable for further use (i.e., you won't get another command prompt) until emacs is closed.

Now close the editor, and then re-open it from the terminal window, but this time add an ampersand character (&) to the end of the command:

emacs /home/weinman/public_html/courses/CSC161/2017F/modules/getting-started/src/quarts-espeak.c &

Now when you return to your terminal window, a prompt is waiting for you, making it easy to do multiple tasks at once. Adding the ampersand character to a command causes the command to be launched as a "background process"—separate from the window process, allowing you to continue working with your terminal window.

Closing the Terminal Window

One way to shut down the terminal window is to type exit.

However, a quicker (and more common) approach is just to type ctrl+d—that is, hold down either of the keys marked ctrl, just below the shift keys, and simultaneously press the d key. (On our workstations' keyboards, the keys marked ctrl ("control") and alt ("alternate" or "meta") are somewhat like shift keys, in the sense that they modify the effect of other keys that are pressed simultaneously.) The shell program interprets ctrl+d as a signal that you have no more instructions for it and halts, and the terminal terminal emulator closes the window automatically once the shell stops running.

Although the shortcut ctrl+d is convenient, note that this key combination is used in multiple contexts in the Linux system to indicate the "end-of-input" character. Thus, when using a simple terminal window, pressing ctrl+d at a command prompt indicates that there will be no more commands forthcoming, and the terminal window should close. However, if you are using another application, the ctrl+d once may exit the application and a second ctrl+d will close the window.

The Manual

Linux includes an on-line help system, called the manual. There is a "man page" describing each Linux command and each C library function. Don't worry if you don't understand everything you see in a man page. They are often long and cryptic, but you can learn a lot without having to understand everything.

For example, try typing man cat to read about cat. You should see the command name and a quick synopsis of how to use the command. For example, the synopsis cat [OPTION] [FILE] ... tells you that cat (optionally) takes a file as its input, and it also can take (optional) options as parameters. A list of the options follows the synopsis. To exit a man page, type q.

Flags

The manual for a program will also describe all the options available. Options are usually prefixed by a - (dash) and referred to as a flag. If a flag is a single letter it only needs one dash. However, longer flags are prefixed by two dashes. This convention allows multiple single letter flags to be smashed together behind one single dash. For example, you can type:

ls -l -a -h

to list all the files in a directory in the long-listing human-readable format. Or, you can simply type:

ls -lah

Another handy use for the man pages is finding commands or C functions when you don't remember, or never knew, their names. Suppose you wanted to find a function for computing the square root: you could guess that such a function might exist, but you might not know its name. To print a list of man pages that include the word "square" in the name or description fields, you could use the following.

man -k square

Of course, keyword searches are more useful for some commands than others:

man -k print

may reap you more results than you want to deal with!

Chapters

The Linux "man" pages system is derived from the original paper manual for the UNIX System from Bell Laboratories. As a result, it shares the same organization of pages into sections, or "chapters." For example, Chapter 1 was Commands (things you would type in the terminal), while Chapter 3 was Subroutines (or functions you would call from C). Chapter 7 was Games.

The chapters in the Linux manual have similar names and categorizations. When you ask to see a man page, the very top heading reveals the name, chapter number, and chapter title just was it was in the original paper manual.

man cat

CAT(1)                           User Commands                        CAT(1)  

NAME
       cat - concatenate files and print on the standard output

...

As you may notice, the chapter number is given in parentheses immediately after the manual page's name. For example, you might read, cat(1), as(1), fork(2), puts(3), mouse(4), whois(5), huntd(6),time(7), halt(8).

The chapter numbers give you a clue about the purpose, but also help resolve ambiguities when entries by the same name appear in two different chapters, as in the sleep(1) command and sleep(3) library function, or time(1) command and time(2) operating system call. By default, you'll get the entry from the lowest number chapter.

man sleep

SLEEP(1)                         User Commands                        SLEEP(1)

NAME
       sleep - delay for a specified amount of time

SYNOPSIS
       sleep NUMBER[SUFFIX]...
       sleep OPTION

...

To force a manual page from a particular chapter, give the number before the entry, as follows.

man 3 sleep

SLEEP(3)                   Linux Programmer's Manual                  SLEEP(3)

NAME
       sleep - sleep for the specified number of seconds

SYNOPSIS
       #include <unistd.h>

       unsigned int sleep(unsigned int seconds);

...

Directory and File Commands

The Linux Directory/File Hierarchy

Linux maintains directories and files in a hierarchical structure, called a tree structure. This section explores this organization.

Pathnames

When you first open a terminal window, the shell expects that you are in your home directory. At the start, this is called your current working directory (i.e., your position within the directory tree).

A relative pathname for a file is a name that is given "relative" to your current working directory. For example, if your current working directory is csc161, then labs/hw2.c could be a relative pathname for a file named hw2.c located in a directory named labs that was itself inside csc161.

An absolute pathname, such as /home/username/csc161/labs/hw2.c, includes the file's complete path starting with the system's "root" directory, which is always named / on a Linux system. Just like it sounds, the root directory is the topmost directory in the file system tree.

Each directory in a Linux system contains two special files . (dot) and .. (dot dot) that can be useful when constructing relative pathnames. The file named . (dot) means "the current directory," and the file named .. (dot dot) means "the parent directory." For example, ../c could be a directory that is a sibling of your current working directory (i.e., c could be a directory that has the same parent directory your current working directory does). Then ../c/labs/hw2.c could refer to a file farther down that branch of the tree. Similarly, ../../pix/grinnell could be a directory that is a cousin of your current directory in the file system tree.

The tilde character is also useful for specifying pathnames, but it works a little differently. Used alone, it specifies your home directory, so ~/csc161/hw is a short name for /home/username/csc161/hw.

Finally, ~username refers to the home directory belonging to username. Thus, you can print a listing of the files in Mr. Walker's public_html directory with

ls -l ~walker/public_html

Root Directory and its Subdirectories

While we are poking around the Linux file system, take a look at the files in the root directory /. You should see directories with names like /bin, /home, /lib, and /usr.

Again, list the files in each of these directories. They contain many, many files, organized as follows.

/bin

These are the executable programs that comprise the GNU/Linux utilities. For example, there is an executable file here named ls that is run when you issue the command ls.

/home

You won't be surprised to hear that user accounts are stored in this directory.

/lib

This directory is the home of several libraries that can be used by programmers.

/usr

The name of this directory is pronounced "user", and it generally contains application programs, libraries, and other files that are not part of the GNU/Linux system (i.e., optional resources intended for and requested by users). For example, the Linux C library for the gcc compiler is found as file libgcc.a in the subdirectory /usr/lib/gcc/x86_64-linux-gnu/4.7/.

Sometimes locating a program can be something of a challenge, but the commands which and whereis can help you solve this problem. For example, both

whereis emacs

and

which emacs

will give you the absolute path name of the emacs program.

Search Paths

When you type the name of a command in a terminal window, the operating system must determine the appropriate program to run. If you type a full path name, the operating simply uses that program. For example, if you type /bin/ls, the operating system will run the ls program located in the /bin directory.

Similarly, you can clarify that the operating system should use a relative path by starting a command with a period (. represents the current directory). For example, ./test tells the operating system to run the program test in the current directory.

However, if you type a command without specifying its location, the operating system must determine where to find the relevant program. To facilitate this process, the operating system maintains a variable $PATH for each terminal window. For example, the variable might be set to

/usr/bin:/bin:/usr/X11R6/bin:/usr/local/bin:/net/bin:.

This string is interpreted as a sequence of directories, separated by a colon :. For example, suppose you type pwd in a terminal window. The operating system first tries to find a program pwd in directory /usr/bin. However, no such program exists there, so the operating system looks at the next directory, /bin. In this case, a program pwd is found, so the operating system uses this program to respond to the command in the terminal window. (If file pwd was not present in /bin, the operating system would continue its search with /usr/X11R6/bin, etc.)

In this example, note that the last option is . (the abbreviation for a program in the current directory). Thus, the operating would run a program in your current directory—but only if there was not a program of the same name in one of the earlier directories listed in the $PATH variable.

The practice of putting your working directory in your default search path is considered a security risk in many circles. You would do well also to avoid this configuration. (Your default MathLAN configuration does not include dot in PATH).

File Utilities

Some common file management commands are listed in the table below. When you have the chance you should try each of these to determine just how they work.

The following variants can be particularly handy. (Although some details may not be obvious now, we will see shortly how to find out more information about such commands.)

cd ..
cp -p
ls -l
mkdir -p

pushd and popd

Two more commands that can be quite useful for moving around the file system are pushd and popd. They let you jump back and forth between distant directories quickly and easily. For example, suppose your current working directory is courses/csc161/hw/ and you want to jump to a directory in another branch of your file system tree, say public_html/courses, and afterward you want to return to your original directory.

The following command will push the name of your current directory onto a stack of directory names that Linux maintains behind the scenes, and then change your current directory to the one named in the command:

pushd public_html/courses

When you are ready to return to your previous directory, you simply type popd. This pops the most recent directory name off the stack and then makes it your current working directory.

Go ahead and give these command a try. Of course, if you like, you can use pushd several times in a row (pushing multiple directory names onto the stack), and then backtrack through the sequence in reverse order.

As one application, you might use pushd and popd when jumping back and forth between labs or homework assignments.

Displaying Text Files

It is often convenient to look at the contents of a text file without having to open it in an editor. Previously in this lab, we saw that cat can be used for this purpose, but it is most useful for short files that can be viewed all on one screen.

GNU/Linux provides several other utilities that are useful for "paging" through text files (i.e., for viewing files one page at a time). Several of these commands are outlined in the following table.

Printing

You likely have have printed files in the Campus Linux Network already. This is frequently done from within another program, say a web browser, PDF reader, or text editor. With the following commands, you can also print files, and manage print queues, from the command line.

Please don't test these out now, unless you have something you really want printed, but keep them in mind for next time you do.

You can accomplish printing a C file in at least two ways:

• Unix command lpr
• Print option within emacs

For this course, printing from emacs is not satisfactory. emacs adds a header that confuses the splitting of a file into pages. For multi-page files, printing from emacs therefore often produces awkward page breaks and partial pages.

For this course, always use lpr in a terminal window to print a C program. For example, you could print file quarts.c with the command

lpr quarts.c

Directory and File Permissions

From a user's perspective within Unix or Linux, the world of files and directories is divided into three categories:

• the user
• the user's group (e.g., CS faculty or student)
• everyone else

For each of these categories, users in these categories can have three types of permissions:

• read a file
• write or modify a file
• execute the file (e.g., run a program)

To clarify how these permissions work, we consider a long listing of files in a subdirectory for Marge Coahran:

ls -l ~coahranm/public_html

total 40
drwxr-xr-x 4 coahranm mathfac 4096 2007-04-25 16:32 csc105
drwxr-xr-x 6 coahranm mathfac 4096 2007-01-19 16:04 csc152
drwxr-xr-x 4 coahranm mathfac 4096 2007-01-19 19:10 csc201
drwxr-xr-x 4 coahranm mathfac 4096 2007-09-01 15:43 csc211
drwxr-xr-x 4 coahranm mathfac 4096 2007-08-31 17:56 csc301
-rw-r--r-- 1 coahranm mathfac 5808 2007-09-01 20:38 index.html
-rw-r--r-- 1 coahranm mathfac 5808 2007-08-31 19:02 index.html~
drwxr-xr-x 2 coahranm mathfac 4096 2008-01-02 15:55 mmc_files

In the listing, the first character indicates whether the item is a directory (d) or a regular file (-). Thus, in this directory, csc211 is a directory (the line starts with d), and index.html is a regular file.

The next characters list specific file permissions (r=read, w=write, x=execute) for the user, group, and world, in that order. Thus, immediately after the d character for the csc105 directory, the sequence rwx indicates that user coahranm has all three permissions for these capabilities. However, both the group and any others have permissions r-x, indicateing that they can read and execute the directory but cannot modify (write) the directory.

Following this permission information, a number appears for the "number of links" involved with the file. Interpreting this is beyond the scope of the course, but ask again in CSC 213 on operating systems!

The rest of the line gives the username and groupname of the file's owner (coahranm and mathfac), the file's size, the date and time the file was last modified date, and file name.

You may belong to more than one group on the MathLAN. To see a list of all of your groups, use the command groups.

If you get really confused, you can ask for your own username with the command whoami.

Setting Permissions

You can set the permissions of the files you own using the chmod command. The simplest approach is to assign numbers to each capability (4 for read, 2 for write, 1 for execute) and then to use addition when combining numbers. Thus, 6 = 4+2 (read plus write permission), and 7 = 4+2+1 (all three permissions added together).

Within this framework, you set permissions for a file by specifying the desired capabilities for the user, group, and world (in that order). Thus, when she set up her directory for CSC 105 above, Ms. Coahran might have issued the command

chmod 755 csc105

Here, the user (Ms. Coahran) has full permissions (7=read+write+execute); while the group and others can read and execute, but not write (5).

Setting Terminal Defaults

Bash Runtime Configuration:

As you may have noticed, the system sets up many elements for you whenever you open a window. This setup is accomplished by the .bashrc file located in your home directory. As you will see, .bashrc is just a regular file that can be edited, and from time to time you may want to tailor this file to meet your special needs or interests.

For now, we mention only two special commands that you may find within .bashrc.

umask allows you to set the default permissions on a file. That is, when you create a new file, the permissions are given according to the specification by umask. In using umask, however, note that the permissions given specify what is to be turned off rather than what is turned on. That is, if you want files not to be writable by others, but you want full privileges, then the appropriate command would be

umask 077

As noted, this statement yields the same result as using chmod to give new files the permissions 700.

alias allows you to make your own commands, putting commands, options, and programs together in a convenient way. For example, on my home computer, I want to be able to log into the Campus Linux Network easily. To do this task, the full command is:

ssh -Y -l walker ssh.cs.grinell.edu

(I want the ssh command, providing a special interface—the Y option—logging in as walker, and connecting to the server ssh.cs.grinnell.ed) Although this command does the right task, it is awkward to type. Thus, in the .bashrc file of my home computer, I include a line of the form

alias my-ssh="ssh -Y -l walker ssh.cs.grinnell.edu"

With this command defined, I can just type my-ssh into a terminal window to connect over the Internet to the Campus Linux Network.

Redirection and Pipes

Some of the most powerful aspects of shell use are the abilities to redirect input or output and chain commands together with pipes. We look at each of these capabilities in turn.

I/O Redirection

Redirection allows programs to get input from a file, rather than the terminal (called standard input or stdin), and to write its output to a file, rather than the terminal (standard output, or stdout).

To take the input or list of commands from a file, rather than the terminal, one uses the input redirection operator <. To direct the standard output to a file one uses the output redirection operator >. Here are three examples; the first uses input redirection, the second uses output redirection, and the third uses both.

command < input.txt
command > output.txt
command < other_input.txt > other_output.txt

The file names are determined by your particular circumstances, and could be anything (though of course the input file specified must exist). Note that output redirection will overwrite any existing file of the name you specify. You can append (rather than replace) an existing file by using the output operator with two angle brackets:

command >> appended_output.txt

for ((inputSize=1 ; inputSize<=1024 ; inputSize=2*inputSize))
do
  echo -n "$inputSize " >> data.txt
 ./runMyTest $inputSize >> data.txt           
done

Pipes

The output of one program may be connected as the input of another program via the use of pipes. This fabulous aspect of the shell environment makes it possible to combine small, modular programs into larger computations that give very specific results from a few general building blocks. As an example, the following command lists all the users running processes on a machine.

ps au | cut -d\  -f 1 | sort | uniq

USER
dan
dols
interiot
pergamon
root
weinman
www-data

In this example, the ps command gives us lots of information on all the processes running on the machine. The output from ps is piped to cut, which (in this example) extracts the first field (the username) as separated by spaces of each line. That output is in turn piped into sort, which rearranges the entries into lexicographic order. Finally, this sorted list is processed by uniq so that all repeated values are removed, leaving us only with a unique list of active users.