This tool uses C−like language to manipulate structured data one line at a time. awk command files are scripts composed of one or many functional blocks enclosed in curly brackets. Each individual block within the file may be associated with a conditional expression that is used to determine whether that block should be executed upon the current line. As you might guess, when there is no conditional expression, the block is executed on every line in the file. The very straightforward example awk script shown in Listing 17.8 takes a file and numbers its lines by a specified increment. Although simple, this operation is useful when you are debugging a script or some C source code, because syntax−error messages usually include the line number. After the BEGIN keyword, the variables are given values. The next block actually prints the number and the line represented by $0. The next line increments the line counter by the specified increment. If there were code that needed to be executed just before the file is closed, a block labeled END could be added to the end of the script. Listing 17.8: A Sample awk Script BEGIN { line = 1; increment = 1; } { # Prints the number and echoes the line. printf("%5d %s\n", line, $0); line += increment; } To execute this from a file, save the code to a file with the extension .awk and issue the following command. (The .awk extension is not required, but including it clarifies which type of script you are calling.) $awk −f file.awk file > newfile Each line matching the conditional expression is prefixed with a line number. If there are no lines matching the conditional expression, each line of the file is echoed to the new file, and no line numbers are added. Here is another example, with the actual actions added to the command instead of in a file: $ ls *.zip |awk '{print "mv "$0" "$0".vs"}'|/bin/bash This command will list all files ending in the suffix .zip, inputting the list into the awk command. The awk command will use the mv command to rename the files ending in .zip so that they now end in .zip.vs. Sample listings are shown below: Before After /tmp/etc.zip /tmp/etc.zip.vs /tmp/tmp.zip /tmp/tmp.zip.vs /tmp/var.zip /tmp/var.zip.vs This is useful if you set up your system to automatically remove all files ending in .zip from /tmp once a week. 439 Several books are available that illustrate the use of awk. A popular one is Sed & Awk (Nutshell Handbook), by Dale Dougherty and Arnold Robbins (O'Reilly, 1997). The awk utility, or the GNU version called GAWK, is included with most Linux distributions and available for most flavors of Unix. sed sed is a stream editor, hence the name. It allows for the search and replacement of regular expressions and more—a valuable capability when the task at hand involves updating a file. You can use sed to automate the update of a text file by searching for a string and changing it as required. It is also quite useful when you need to change the output of a command before piping it into another command. If you have ever issued a command such as :s/phrase/newphrase/ to replace one phrase with another within an editor, then you know how to use sed commands. When using sed, however, this change is automatically applied globally because sed works on each line in the file. sed can be executed with the actions as a command−line argument. It is sometimes beneficial to do this if, for example, the output of a very long file needs slight alteration so that it can be printed to a new file in the format requested by the boss. Here is a fairly simple example: $cat /tmp/calendar |sed −e 's/Jan/&uary/' | sed −e 's/Feb/&ruary/' | sed −e 's/Mar/&ch/' | sed −e 's/Apr/&il' | sed −e 's/Jun/&e/'| sed −e 's/Jul/&y/' > newfile In this example, a calendar file containing dates within the first seven months of the year needs to be altered so that the abbreviated month name is replaced by the full name of that month. The edited calendar is to be written out to newfile. First we cat the file into our sequence of sed commands. The −e tells sed that the next thing is a sed command to be executed. This command's placement within single tick marks distinguishes it from the rest of the line. The s means that we wish to substitute the second phrase for the first. Thus this command line meets our file−alteration needs, its length well justified when it saves many keystrokes in a file with say, 1000 lines. Like awk, the sed command is included with most Linux distributions and available for most flavors of Unix. System Initialization Scripts Often when you are using some recently developed utility, you need to start the utility at boot time and aren't able to depend upon an existing script to do this. For these situations, you need to write your own initialization script and add it to the initialization scripts that we discussed in Chapter 3, "Startup and Shutdown." This is actually quite easy to do if you follow the example of an existing script. These scripts are all Bash shell scripts and share a common format. Writing an Initialization Script Let's say that you have downloaded a tarball of mynewutility from an Internet site. Obviously, there is no mynewutility script in the /etc/rc.d hierarchy, since the utility isn't part of the standard distribution. When you have finished installing the utility and configuring it, you'll probably want to make it start at boot time like the other daemons in the /etc/rc.d hierarchy. To accomplish this, you'll need to write an initialization script to manage the new utility. 440 Let's look at the existing script for the printer daemon and adapt it for use to manage mynewutility. Red Hat 7.3's printer startup script is /etc/rc.d/init.d/lpd, shown in Listing 17.9. Listing 17.9: A Sample Initialization Script for lpd #!/bin/sh # # lpd This shell script takes care of starting and stopping \ # lpd (printer daemon). # # chkconfig: 2345 60 60 # description: lpd is the print daemon required for lpr to work properly. \ # It is basically a server that arbitrates print jobs to printer(s). # processname: /usr/sbin/lpd # config: /etc/printcap # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration and check that networking is up. if [ −f /etc/sysconfig/network ] ; then . /etc/sysconfig/network [ ${NETWORKING} = "no" ] && exit 0 fi [ −x /usr/sbin/lpd ] || exit 0 prog=lpd RETVAL=0 start () { echo −n $"Starting $prog: " # Is this a printconf system? if [[ −x /usr/sbin/printconf−backend ]]; then # run printconf−backend to rebuild printcap and spools if ! /usr/sbin/printconf−backend ; then # If the backend fails, we dont start no printers defined echo −n $"No Printers Defined" echo_success echo return 0 fi fi if ! [ −e /etc/printcap ] ; then echo_failure echo return 1 fi # run checkpc to fix whatever lpd would complain about /usr/sbin/checkpc −f # start daemon daemon /usr/sbin/lpd RETVAL=$? echo [ $RETVAL = 0 ] && touch /var/lock/subsys/lpd return $RETVAL } stop () { # stop daemon echo −n $"Stopping $prog: " 441 killproc /usr/sbin/lpd RETVAL=$? echo [ $RETVAL = 0 ] && rm −f /var/lock/subsys/lpd return $RETVAL } restart () { stop start RETVAL=$? return $RETVAL } # See how we were called. case "$1" in start) start ;; stop) stop ;; status) status /usr/sbin/lpd RETVAL=$? ;; restart) restart ;; condrestart) # only restart if it is already running [ −f /var/lock/subsys/lpd ] && restart || : ;; reload) echo −n $"Reloading $prog: " killproc /usr/sbin/lpd −HUP RETVAL=$? echo ;; *) echo $"Usage: $0 {start|stop|restart|condrestart|reload|status}" RETVAL=1 esac exit $RETVAL The format of initialization scripts is fairly uniform. The standard SysV initialization scripts have some features that we really like. At the beginning of the script is a chkconfig line that tells the system which run levels the script is to be set up for and a comment detailing what the script is intended to do. Note It's also helpful to add comments that identify any specific system characteristics the script requires to run successfully, although this is not done in most of the existing scripts. This might be a comment such as "In order for the lpd script to run successfully, the printers must have been configured, and a properly formatted /etc/printcap must exist." This setup is not critical to the functioning of the script, but it simplifies things for a new system administrator trying to gain a deeper awareness of the computer system. 442 After the comments, the /etc/init.d/functions library script is run. This script sets up functions that are used within initialization scripts. The library script contains functions that are useful to multiple initialization scripts and otherwise have to be duplicated in each script. Next, the script checks certain conditions that are required for the printer daemon to function properly. First, the script checks whether the network is up. If the network variable in the network script is set to no, the lpd script exits, because lpr and lpd communicate using the network facilities. Next, the script verifies that lpd exists and is an executable file. It also verifies that an /etc/printcap file exists. The next section contains functions that are to be called later in the script. In the case of the lpd script, these are start, stop, and restart. The next part of the script contains the menu functionality that allows you to specify which one of the script functions you'd like to perform. This is a case statement like those in C and other languages, setting the following alternatives: If the script was called with the start argument, then call the start function.• If the script was called with the stop argument, then call the stop function.• If the script was called with the status argument, then use the status function from the /etc/rc.d/init.d/functions library to determine whether the line printer daemon is running. • If lpd was called with the restart argument, then run the restart function.• Starting lpd with the condrestart argument causes the system to restart lpd only if it is currently running. • Running the lpd script with the reload argument tells the script to send a HUP signal to the line printer daemon. • The last line in the lpd script is the return of the $RETVAL variable, which has been generated by the function that the script performed. Now that you understand this existing script, creating your own initialization script for mynewutility will be relatively easy. Let's assume that mynewutility requires networking to be active and that the executable is located at /usr/sbin/mynewutilityd. We only need start, restart, stop, and status functionality in this case. The new script will look something like Listing 17.10. Listing 17.10: An Initialization Script for mynewutility #!/bin/sh # # mynewutilityd This shell script takes care of starting # and stopping mynewutilityd. # # chkconfig: 345 96 96 # description: mynewutilityd is a fictitious daemon used # to illustrate startup script functions. # processname: /usr/sbin/mynewutilityd # Source function library. . /etc/init.d/functions # Source networking configuration and check that networking is up. if [ −f /etc/sysconfig/network ] ; then . /etc/sysconfig/network [ ${NETWORKING} = "no" ] && exit 0 fi 443 [ −x /usr/sbin/mynewutilityd ] || exit 0 RETVAL=0 start () { echo −n "Starting mynewutilityd: " # start daemon daemon /usr/sbin/mynewutilityd RETVAL=$? echo [ $RETVAL = 0 ] && touch /var/lock/subsys/mynewutilityd return $RETVAL } stop () { # stop daemon echo −n "Stopping mynewutilityd: " killproc /usr/sbin/mynewutilityd RETVAL=$? echo [ $RETVAL = 0 ] && rm −f /var/lock/subsys/mynewutilityd return $RETVAL } restart () { stop start RETVAL=$? return $RETVAL } # See how we were called. case "$1" in start) start ;; stop) stop ;; status) status /usr/sbin/mynewutilityd RETVAL=$? ;; restart) restart ;; *) echo "Usage: mynewutilityd {start|stop|restart RETVAL=1 esac exit $RETVAL Of course, this script will not run unless you use the chkconfig or ntsysv utility to add it to the scripts for the run levels specified in the chkconfig line in the script's comments, thus: #chkconfig mynewutilityd on 444 Tailoring the rc.local Script The rc.local script is likely to need your editing, because it is the script where you place so−called local changes—those that are specific to one computer and don't represent daemons that are added as part of a package. In fact, it's common to start unusual daemons (like the fictitious mynewutility discussed above) by adding them to rc.local. Let's take a look at the default version of rc.local for Red Hat 7.3 (Listing 17.11). Listing 17.11: The rc.local Script for Red Hat 7.3 #!/bin/sh # # This script will be executed *after* all the other init # scripts. # You can put your own initialization stuff in here if you # don't want to do the full Sys V style init stuff. if [ −f /etc/redhat−release ]; then R=$(cat /etc/redhat−release) arch=$(uname −m) a="a" case "_$arch" in _a*) a="an";; _i*) a="an";; esac NUMPROC=`egrep −c "^cpu[0−9]+" /proc/stat` if [ "$NUMPROC" −gt "1" ]; then SMP="$NUMPROC−processor " if [ "$NUMPROC" = "8" −o "$NUMPROC" = "11" ]; then a="an" else a="a" fi fi # This will overwrite /etc/issue at every boot. So, # make any changes you want to make to /etc/issue here # or you will lose them when you reboot. echo "" > /etc/issue echo "$R" >> /etc/issue echo "Kernel $(uname −r) on $a $SMP$(uname −m)" >> /etc/issue cp −f /etc/issue /etc/issue.net echo >> /etc/issue fi The default rc.local file is simple. It only contains code to create the /etc/issue and /etc/issue.net files. The /etc/issue file is used to write the text introduction to the login prompt. The result on an x86 computer using Red Hat 7.3 looks like this: Red Hat Linux release 7.3 (Valhalla) Kernel 2.4.18−3 on an i686 The rc.local file first declares its interpreter, just like all the other scripts that we've looked at in this chapter. Next, rc.local checks whether the file /etc/redhat−release exists and is a regular file. If the 445 file is both, the variable R is set to the content of the file. The case statement that follows the redhat−release check exists to determine whether the correct word in the upcoming string should grammatically be a or an; if the value of $arch has an initial vowel, the script uses an; otherwise, it uses a. Next, an if statement is used to determine whether there is more than one processor; and second, if there is more than one processor, whether the word for the number of processors begins with a vowel, as in 8 (eight) and 11 (eleven). This determination is only important if the system is using multiple processors, in which case the /etc/issue will look something like this: Red Hat Linux release 7.3 (Valhalla) Kernel 2.4.18−3 on a 2−processor i686 The last few lines of the script simply write out the data to the /etc/issue file and copy it to /etc/issue.net. If you have not created the mynewutilityd script, you can start the mynewutility daemon in the rc.local file by adding the following line at the end of the file: daemon mynewutilityd Administrator's Logbook: Changes to Red Hat 7.3 default /etc/rc.d/rc.local script Added a line to start the daemon mynewutilityd, which is part of the mynewutility package downloaded from www.mynewutility.org, to add basic newutility functionality to our network. VS 16 Jul 02 Cautions about Changing Existing Scripts Many system administrators advise that it is not a good idea to revise and adapt existing scripts. Actually changing an existing script on your system is sometimes necessary, although certainly you should exercise caution when you change one. Should you decide to tailor an existing script to your specific usage, it is a good practice to start by making a copy of the script with an extension like .orig to identify it as the original version. You also need to add comments noting within the tailored script what specifically was changed and why. It is advisable to keep a backup of the original script in case of accidental overwrite. Generally, anything that makes your system different from the default arrangement should also be included in an administrative log entry. When you are troubleshooting a problem, this helps to clarify differences between a machine that is working properly and one that isn't. Sometimes it is the change that caused the malfunction, and other times it might be the lack of this change that is problematic. Consider the example discussed in this chapter, where you have added a check to the mynewutility script. This check determines whether a process called necessary is running, and the script only continues if the process is running. A log entry to accompany this might look like the one in the Administrator's Logbook example on this page, for "Changes to Red Hat 7.3 default /etc/rc.d/init.d/mynewutilityd script." Administrator's Logbook: Changes to Red Hat 7.3 default /etc/rc.d/init.d/mynewutilityd script 446 Edited /etc/rc.d/init.d/mynewutilityd adding a check to determine whether the necessary process was currently running. If necessary is not running, the script attempts to run it. If it runs successfully, proceed with the initialization of the mynewutility daemon. VS 14 Aug 02 #check for necessary process if ! pgrep necessary then /usr/bin/necessary if ! pgrep necessary then exit fi fi Using the cron Facility The cron daemon makes it very easy to set up a recurring event based on the need to run it at a given time, day, or date. cron searches /var/spool/cron for crontab files, which are named after existing user accounts; any crontabs that are found are loaded into memory. The daemon also searches for /etc/crontab and the files in the /etc/cron.d/ directory. The /etc/crontab file contains entries that run files in /etc/cron.hourly on an hourly basis, files in /etc/cron.daily on a daily basis, files in /etc/cron.weekly once a week, and files in /etc/cron.monthly once a month. Listing 17.12 shows the default /etc/crontab file for Red Hat 7.3. Listing 17.12: The Default Red Hat 7.3 /etc/crontab SHELL=/bin/bash PATH=/sbin:/bin:/usr/sbin:/usr/bin MAILTO=root HOME=/ # run−parts 01 * * * * root run−parts /etc/cron.hourly 02 4 * * * root run−parts /etc/cron.daily 22 4 * * 0 root run−parts /etc/cron.weekly 42 4 1 * * root run−parts /etc/cron.monthly 0−59/5 * * * * root /usr/bin/mrtg /etc/mrtg/mrtg.cfg In the first line, the shell is established as the Bash shell, and a path is set up. The MAILTO line sets up who is to receive the output from these commands, including error messages and simple notifications that the files were run. The home directory is set to /. The first five fields of the run−parts lines establish the run times for the files contained in the listed directory. The five fields used in /etc/crontab are as follows: minute 0–59 hour 0–23 (24−hour format) day of the month 1–31 month 1–12 day of week 0–7 (where both 0 and 7 are Sunday) 447 Asterisks represent any value. In the /etc/cron.hourly line, then, the string 01 * * * * indicates that the files contained in the /etc/cron.hourly directory should be run at one minute after each hour. The /etc/cron.daily line uses 02 4 * * * which indicates that the files contained therein should be run at two minutes after the fourth hour of each day. The /etc/cron.weekly line 22 4 * * 0 identifies that the files it contains are to be run at 4:22 on Sunday. The /etc/cron.monthly directory contains files that are to be run at 4:42 on the first day of the month, based on the string: 42 4 1 * * Values can be lists, ranges, or steps. Lists are comma−separated values as follows: 1,3,5,7,9 Ranges look like this: 1−5 You can combine the two like this: 1−5,7,9 Steps are ranges in which a certain set of numbers are skipped. The form is N/m, where N is typically a range and m represents the numbers to skip. Thus the following in the hour field indicates that the line should be run every other hour: 0−23/2 To indicate every other possible value for the field, you can use */2 The sixth field of the run−parts lines identifies the user in whose name the command should be run; in this case, root. The seventh field is the command itself. You can simply create an executable script and place it in the appropriate directory. If the cron job only applies to one user, it should be added to that user's personal crontab file via the crontab command. This command installs, uninstalls, or lists the user's crontab file, /var/spool/cron/$USER, depending upon the argument you use. The two forms for the crontab command are as follows: crontab [−u username] file crontab [−u username] [−l] [−r] [−e] The first crontab syntax is used to replace the user's crontab file with another, so a filename with 448 [...]... capabilities that Linux offers The versatility of scripts in the Linux world is one of the strengths of Linux Next we'll move onto a discussion of troubleshooting Even the best maintained systems have difficulties now and then Diagnosing the problem and finding a solution is to many the most intriguing and enjoyable part of Linux system administration 453 Chapter 18: Troubleshooting Your Linux System Overview... Remember that an ext3 filesystem is just an ext2 filesystem with journaling added Creating a Linux Filesystem on a Floppy Disk To create an ext3 filesystem on a 3.5", high−density floppy, issue the following command: # /sbin/mke2fs −j /dev/fd0 The system will reply: mke2fs 1.27 (8−Mar 2002) Filesystem label= OS type: Linux Block size =102 4 (log=0) Fragment size =102 4 (log=0) 184 inodes, 1440 blocks 72 blocks... Filesystem Problems or Questions The filesystem is covered in great detail in Chapter 6 As a Linux system administrator, the more you know about the ext3 filesystem the better You should get familiar with the inode concept and the filesystem layout especially, including the intended uses of the directories off / Here are a few of the most common filesystem−related questions Remember that an ext3 filesystem... prompt 458 timeout=50 message=/boot/message linear default =linux image=/boot/vmlinuz−2.4.7 10 label =linux read−only image=/boot/vmlinuz−2.4.7 10 new label =linux_ new read−only GRUB GRUB uses a number from 0 to one less than the number of configured kernel images to denote which image is to be the default To change the default boot kernel from linux to linux_ new using GRUB, edit the /boot/grub/grub.conf file... requiring both a natural analytical ability and an extensive knowledge of the Linux system It is also one of the most high−profile aspects of the Linux system administrator's job If something isn't working, you can bet there will be "shoulder surfers" around to watch while you fix the problem The more familiar you are with the Linux filesystem covered in Chapter 7, the booting and initialization processes... check the Swap line, because if a great deal of swap space is in use, the system' s performance will be degraded as a result If you're still convinced that Linux isn't seeing all of your system' s memory, the most likely explanation is that the system has a BIOS that's not working well with Linux' s memory probes Typically, such systems report just 64MB, or sometimes only 16MB, of RAM The fix is to use... X−CD−Roast main screen Figure 9.5: The Taper main screen Figure 9.6: The BRU main screen Chapter 10: Printers and the Spooling Subsystem Figure 10. 1: Laser printing Figure 10. 2: Red Hat's Printconf facility Figure 10. 3: You can elect to define a local printer or one of four different types of network printers Figure 10. 4: The Select Print Driver dialog box lets you select the model of the printer you are... Problems Because gaining access to your systems is so important, you must be able to diagnose and repair login problems Knowing the login process step by step is the key to this ability Once you've been doing Linux system administration for a while, these problems will be easy for you to fix Lost Password One of the most frustrating problems occurs when you have a system and no one seems to remember the... Making the System Boot a New Kernel Once you get a kernel that has everything that you need, you might want it to be booted by default After compiling and thoroughly testing a new kernel, edit the configuration file for the boot loader to change the default LILO To change the default boot kernel from linux to linux_ new, edit the /etc/lilo.conf file to change the default =linux line to default =linux_ new... the GNU Hurd filesystem, the developers of the FDISK utility have assigned the partition code 0x63 This partition type is not currently used, so if you can't get FDISK to recognize your Hurd−owned filesystem, use the standard 0x83 if the filesystem contains an ext2 or ext3 filesystem Do not use the new 0x63 code Making a New Boot Floppy to Replace a Lost One Sometime in your career as a system administrator, . BIOS's disk geometry settings and those used by Linux. (See below for a description of disk geometry settings.) LI1 010, etc. When LILO returns a string of LI1 0101 0… at bootup, the problem is that LILO. solution is to many the most intriguing and enjoyable part of Linux system administration. 453 Chapter 18: Troubleshooting Your Linux System Overview Troubleshooting is an art, requiring both a. natural analytical ability and an extensive knowledge of the Linux system. It is also one of the most high−profile aspects of the Linux system administrator's job. If something isn't working,