| Purchase | Copyright © 2002 Paul Sheer. Click here for copying permissions. | Home |
|
|
| |
From this chapter you will get an idea about what is happening under the hood of your UNIX system, but go have some coffee first.
On UNIX, when you run a program (like any of the shell commands you have been using), the actual computer instructions are read from a file on disk from one of the bin/ directories and placed in RAM. The program is then executed in memory and becomes a process. A process is some command/program/shell-script that is being run (or executed) in memory. When the process has finished running, it is removed from memory. There are usually about 50 processes running simultaneously at any one time on a system with one person logged in. The CPU hops between each of them to give a share of its execution time. [Time given to carry out the instructions of a particular program. Note this is in contrast to Windows or DOS where the program itself has to allow the others a share of the CPU: under UNIX, the process has no say in the matter. ]Each process is given a process number called the PID (process ID). Besides the memory actually occupied by the executable, the process itself seizes additional memory for its operations.
In the same way that a file is owned by a particular user and group, a process also has an owner--usually the person who ran the program. Whenever a process tries to access a file, its ownership is compared to that of the file to decide if the access is permissible. Because all devices are files, the only way a process can do anything is through a file, and hence file permission restrictions are the only kind of restrictions ever needed on UNIX. [There are some exceptions to this.] This is how UNIX access control and security works.
The center of this operation is called the UNIX kernel. The kernel is what actually does the hardware access, execution, allocation of process IDs, sharing of CPU time, and ownership management.
Log in on a terminal and type the command ps. You should get some output like:
|
PID TTY STAT TIME COMMAND5995 2 S 0:00 /bin/login -- myname5999 2 S 0:00 -bash6030 2 R 0:00 ps |
ps with no options shows three processes to be running. These are the only three processes visible to you as a user, although there are other system processes not belonging to you. The first process was the program that logged you in by displaying the login prompt and requesting a password. It then ran a second process call bash, the Bourne Again shell [The Bourne shell was the original UNIX shell] where you have been typing commands. Finally, you ran ps, which must have found itself when it checked which processes were running, but then exited immediately afterward.
The shell has many facilities for controlling and executing processes--this is called job control. Create a small script called proc.sh:
|
#!/bin/shecho "proc.sh: is running"sleep 1000 |
Run the script with chmod 0755 proc.sh and then ./proc.sh. The shell blocks, waiting for the process to exit. Now press ^Z. This will cause the process to stop (that is, pause but not terminate). Now do a ps again. You will see your script listed. However, it is not presently running because it is in the condition of being stopped. Type bg (for background). The script will now be ``unstopped'' and run in the background. You can now try to run other processes in the meantime. Type fg, and the script returns to the foreground. You can then type ^C to interrupt the process.
Create a program that does something a little more interesting:
5 |
#!/bin/shecho "proc.sh: is running"while true ; do echo -e '\a' sleep 2done |
Now perform the ^Z, bg, fg, and ^C operations from before. To put a process immediately into the background, you can use:
|
./proc.sh & |
The JOB CONTROL section of the bash man page ( bash(1)) looks like this(footnote follows) [Thanks to Brian Fox and Chet Ramey for this material.]: (the footnotes are mine)
JOB CONTROL
Job control refers to the ability to selectively stop (suspend) the execution of processes and continue (resume) their execution at a later point. A user typically employs this facility via an interactive interface supplied jointly by the system's terminal driver and bash.
The shell associates a job with each pipeline. [What does this mean? It means that each time you execute something in the background, it gets its own unique number, called the job number.]It keeps a table of currently executing jobs, which may be listed with the jobs command. When bash starts a job asynchronously (in the background), it prints a line that looks like:
[1] 25647
indicating that this job is job number 1 and that the process ID of the last process in the pipeline associated with this job is 25647. All of the processes in a single pipeline are members of the same job. Bash uses the job abstraction as the basis for job control.
To facilitate the implementation of the user interface to job control, the system maintains the notion of a current terminal process group ID. Members of this process group (processes whose process group ID is equal to the current terminal process group ID) receive keyboard-generated signals such as SIGINT. These processes are said to be in the foreground. Background processes are those whose process group ID differs from the terminal's; such processes are immune to keyboard-generated signals. Only foreground processes are allowed to read from or write to the terminal. Background processes which attempt to read from (write to) the terminal are sent a SIGTTIN (SIGTTOU) signal by the terminal driver, which, unless caught, suspends the process.
If the operating system on which bash is running supports job control, bash allows you to use it. Typing the suspend character (typically ^Z, Control-Z) while a process is running causes that process to be stopped and returns you to bash. Typing the delayed suspend character (typically ^Y, Control-Y) causes the process to be stopped when it attempts to read input from the terminal, and control to be returned to bash. You may then manipulate the state of this job, using the bg command to continue it in the background, the fg command to continue it in the foreground, or the kill command to kill it. A ^Z takes effect immediately, and has the additional side effect of causing pending output and typeahead to be discarded.
There are a number of ways to refer to a job in the shell. The character % introduces a job name. Job number n may be referred to as %n. A job may also be referred to using a prefix of the name used to start it, or using a substring that appears in its command line. For example, %ce refers to a stopped ce job. If a prefix matches more than one job, bash reports an error. Using %?ce, on the other hand, refers to any job containing the string ce in its command line. If the substring matches more than one job, bash reports an error. The symbols %% and %+ refer to the shell's notion of the current job, which is the last job stopped while it was in the foreground. The previous job may be referenced using %-. In output pertaining to jobs (e.g., the output of the jobs command), the current job is always flagged with a +, and the previous job with a -.
Simply naming a job can be used to bring it into the foreground: %1 is a synonym for ``fg %1'', bringing job 1 from the background into the foreground. Similarly, ``%1 &'' resumes job 1 in the background, equivalent to ``bg %1''.
The shell learns immediately whenever a job changes state. Normally, bash waits until it is about to print a prompt before reporting changes in a job's status so as to not interrupt any other output. If the -b option to the set builtin command is set, bash reports such changes immediately. (See also the description of notify variable under Shell Variables above.)
If you attempt to exit bash while jobs are stopped, the shell prints a message warning you. You may then use the jobs command to inspect their status. If you do this, or try to exit again immediately, you are not warned again, and the stopped jobs are terminated.
To terminate a process, use the kill command:
|
kill <PID> |
The kill command actually sends a termination signal to the process. The sending of a signal simply means that the process is asked to execute one of 30 predefined functions. In some cases, developers would not have bothered to define a function for a particular signal number (called catching the signal); in which case the kernel will substitute the default behavior for that signal. The default behavior for a signal is usually to ignore the signal, to stop the process, or to terminate the process. The default behavior for the termination signal is to terminate the process.
To send a specific signal to a process, you can name the signal on the command-line or use its numerical equivalent:
|
kill -SIGTERM 12345 |
or
|
kill -15 12345 |
which is the signal that kill normally sends when none is specified on the command-line.
To unconditionally terminate a process:
|
kill -SIGKILL 12345 |
or
|
kill -9 12345 |
which should only be used as a last resort. Processes are prohibited from ever catching the SIGKILL signal.
It is cumbersome to have to constantly look up the PID of a process. Hence the GNU utilities have a command, killall, that sends a signal to all processes of the same name:
|
killall -<signal> <process_name> |
This command is useful when you are sure that there is only one of a process running, either because no one else is logged in on the system or because you are not logged in as superuser. Note that on other UNIX systems, the killall command kills all the processes that you are allowed to kill. If you are root, this action would crash the machine.
The full list of signals can be gotten from signal(7), and in the file /usr/include/asm/signal.h.
All processes are allocated execution time by the kernel. If all processes were allocated the same amount of time, performance would obviously get worse as the number of processes increased. The kernel uses heuristics [Sets of rules.] to guess how much time each process should be allocated. The kernel tries to be fair--two users competing for CPU usage should both get the same amount.
Most processes spend their time waiting for either a key press, some network input, some device to send data, or some time to elapse. They hence do not consume CPU.
On the other hand, when more than one process runs flat out, it can be difficult for the kernel to decide if it should be given greater priority than another process. What if a process is doing some operation more important than another process? How does the kernel tell? The answer is the UNIX feature of scheduling priority or niceness. Scheduling priority ranges from +20 to -20. You can set a process's niceness with the renice command.
|
renice <priority> <pid>renice <priority> -u <user>renice <priority> -g <group> |
A typical example is the SETI program. [SETI stands for Search for Extraterrestrial Intelligence. SETI is an initiative funded by various obscure sources to scan the skies for radio signals from other civilizations. The data that SETI gathers has to be intensively processed. SETI distributes part of that data to anyone who wants to run a seti program in the background. This puts the idle time of millions of machines to ``good'' use. There is even a SETI screen-saver that has become quite popular. Unfortunately for the colleague in my office, he runs seti at -19 instead of +19 scheduling priority, so nothing on his machine works right. On the other hand, I have inside information that the millions of other civilizations in this galaxy and others are probably not using radio signals to communicate at all :-)] Set its priority to +19 with:
|
renice +19 <pid> |
to make it disrupt your machine as little as possible.
Note that nice values have the reverse meaning that you would expect: +19 means a process that eats little CPU, while -19 is a process that eats lots. Only superuser can set processes to negative nice values.
Mostly, multimedia applications and some device utilities are the only processes that need negative renicing, and most of these will have their own command-line options to set the nice value. See, for example, cdrecord(1) and mikmod(1) -- a negative nice value will prevent skips in your playback. [LINUX will soon have so called real time process scheduling. This is a kernel feature that reduces scheduling latency (the gaps between CPU execution time of a process, as well as the time it takes for a process to wake). There are already some kernel patches that accomplish this goal.]
Also useful are the -u and -g options, which set the priority of all the processes that a user or group owns.
Further, we have the nice command, which starts a program under a defined niceness relative to the current nice value of the present user. For example,
|
nice +<priority> <pid>nice -<priority> <pid> |
Finally, the snice command can both display and set the current niceness. This command doesn't seem to work on my machine.
|
snice -v <pid> |
The top command sorts all processes by their CPU and memory consumption and displays the top twenty or so in a table. Use top whenever you want to see what's hogging your system. top -q -d 2 is useful for scheduling the top command itself to a high priority, so that it is sure to refresh its listing without lag. top -n 1 -b > top.txt lists all processes, and top -n 1 -b -p <pid> prints information on one process.
top has some useful interactive responses to key presses:
The top man page describes the field meanings. Some of these are confusing and assume knowledge of the internals of C programs. The main question people ask is: How much memory is a process using? The answer is given by the RSS field, which stands for Resident Set Size. RSS means the amount of RAM that a process consumes alone. The following examples show totals for all processes running on my system (which had 65536 kilobytes of RAM at the time). They represent the total of the SIZE, RSS, and SHARE fields, respectively.
5 10 |
echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \ awk '{print "+" $5}' | sed -e 's/M/\\*1024/'` | bc68016 echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \ awk '{print "+" $6}' | sed -e 's/M/\\*1024/'` | bc58908 echo `echo '0 ' ; top -q -n 1 -b | sed -e '1,/PID *USER *PRI/D' | \ awk '{print "+" $7}' | sed -e 's/M/\\*1024/'` | bc30184 |
The SIZE represents the total memory usage of a process. RSS is the same, but excludes memory not needing actual RAM (this would be memory swapped to the swap partition). SHARE is the amount shared between processes.
Other fields are described by the top man page (quoted verbatim) as follows:
Each process that runs does so with the knowledge of several var =value text pairs. All this means is that a process can look up the value of some variable that it may have inherited from its parent process. The complete list of these text pairs is called the environment of the process, and each var is called an environment variable. Each process has its own environment, which is copied from the parent process's environment.
After you have logged in and have a shell prompt, the process you are using (the shell itself) is just like any other process with an environment with environment variables. To get a complete list of these variables, just type:
|
set |
This command is useful for finding the value of an environment variable whose name you are unsure of:
|
set | grep <regexp> |
Try set | grep PATH to see the PATH environment variable discussed previously.
The purpose of an environment is just to have an alternative way of passing parameters to a program (in addition to command-line arguments). The difference is that an environment is inherited from one process to the next: for example, a shell might have a certain variable set and may run a file manager, which may run a word processor. The word processor inherited its environment from file manager which inherited its environment from the shell. If you had set an environment variable PRINTER within the shell, it would have been inherited all the way to the word processor, thus eliminating the need to separately configure which printer the word processor should use.
Try
|
X="Hi there"echo $X |
You have set a variable. But now run
|
bash |
You have now run a new process which is a child of the process you were just in. Type
|
echo $X |
You will see that X is not set. The reason is that the variable was not exported as an environment variable and hence was not inherited. Now type
|
exit |
which breaks to the parent process. Then
|
export Xbashecho $X |
You will see that the new bash now knows about X.
Above we are setting an arbitrary variable for our own use. bash (and many other programs) automatically set many of their own environment variables. The bash man page lists these (when it talks about unsetting a variable, it means using the command unset <variable>). You may not understand some of these at the moment, but they are included here as a complete reference for later.
The following is quoted verbatim from the bash man page. You will see that some variables are of the type that provide special information and are read but never never set, whereas other variables configure behavioral features of the shell (or other programs) and can be set at any time(footnote follows) [Thanks to Brian Fox and Chet Ramey for this material.].
Shell Variables
The following variables are set by the shell:
The following variables are used by the shell. In some cases, bash assigns a default value to a variable; these cases are noted below.
``.:~:/usr''.
MAILPATH='/usr/spool/mail/bfox?"You have mail":~/shell-mail?"$_ has mail!"'
Bash
supplies a default value for this variable, but the location of the user
mail files that it uses is system dependent (e.g., /usr/spool/mail/$USER).