MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


Learning the Mac OS X Terminal

Learning the Mac OS X Terminal, Part 3

by Chris Stone
02/26/2002

Editor's note -- Chris Stone has recently posted an update to this tutorial for readers using Mac OS X 10.1.5 or later. The update focuses on Sendmail. All other features in this article continue to work as originally discussed.

The series continues in Learning the Terminal in Mac OS X, Automating Mail from the Mac OS X Terminal, Configuring Email from the Mac OS X Terminal, Customizing the Mac OS X Terminal, and Synchronizing Drives with Cron and the Mac OS X Terminal.

However, none of the first three parts of the series will work in Mac OS X 10.2 (Jaguar) or newer. Jaguar brought several major changes to the OS that require significant changes to the procedure. I'll be posting Jaguar compatible updates to these articles shortly.

Now that you have the regular maintenance cron jobs running at more reasonable times and emailing you their reports, you would probably like to know what those jobs do, and what the reports tell you.

Today we'll first look at both of these things, and then finish up with a couple of ďvariations on the themeĒ -- additional procedures to make this process even easier (though with their own trade-offs and caveats).

The Daily Script

To begin, let's have another look at the system crontab, specifically the command for the daily job:


sh /etc/daily   2>&1 | tee /var/log/daily.out  
   | mail -s "`hostname` daily output"  root

As you might remember, cron uses the Bourne shell (sh) to run the script found in /etc/daily. The real meat of this cron job, then, exists in that file, which is rather long and complex and would require several articles like this to fully explain. However, the script does contain comments describing each general task, so having a good look at this is informative.

The /etc/daily file is too long to view with cat. Instead, youíll need an application that can display the files one bite-sized one-screen chunk, or "page," at a time. Such applications are called pagers, and one of the most common pagers is more.

As you did with cat, enter "more" and the fileís pathname to view the first page of the file. To view the daily script, then, enter:

more /etc/daily

To move down to the next page, press the space bar. (To go back up, press B.) At the bottom of the more display, youíll see the name of the file being viewed, as well as an indicator of the percentage of the file read so far. To exit more, type Q.

As you peruse the script, keep in mind that lines beginning with the #character are ignored as the shell runs through the script. Typically, such lines contain descriptive comments about the script, but they can also be actual command lines that for various reasons are turned off, or "commented out."

terminal Window
A look inside the more window with the percentage of the file read so far.

Also, the command lines beginning with echo provides some of the output that ends up in the report. For example, these first two lines begin the daily report with a blank line and the quoted line of text:


echo ""
echo "Removing scratch and junk files:"

Again, I canít detail fully this script here, but what follows are the most pertinent highlights. The skipped parts of the script mostly relate to processes not applicable to a "stock" OS X system.

The first section of the script removes "scratch and junk files" from your system. Specifically, some of these items are:

  1. Files existing in /tmp that havenít been accessed or changed in at least the last three days. The /tmp directory contains, among other things, the Temporary Items directory used by many GUI applications, so itís often chock-full of good trash fodder.
  2. Files existing in /var/tmp that havenít been accessed in at least a week, or changed in at least the last three days. Some Unix processes leave junk here as well.

The daily script also performs one of the most important tasks of any in these scripts: backing up your NetInfo database. Should your NetInfo database ever need restoring, you can easily do so using NetInfo Manager (in Applications-> Utilities) and the backed-up database, called local.nibak.

The heading of the next section of the script is, as you can see in the cron report, "Checking subsystem status."

Checking subsystem status:


disks:
Filesystem   1K-blocks     Used    Avail Capacity  Mounted on
/dev/disk0s5  40016844 33274772  6742072    83%    /
fdesc                1        1        0   100%    /dev
/dev/disk2s1s2    354552   354552      0   100%    /Volumes/Q3TA

Related Reading

Mac OS X: The Missing Manual
By David Pogue

This output is a result of the df -k -l command in the daily script, which reports the used and free space on all local disks. This example shows a 40-gigabyte system disk (which will always show as "mounted on" /, or "root") with about 6.7 gigabytes of free space.

You can ignore the fdesc line, which doesnít refer to any actual disk, but part of the filesystem plumbing.

Any other local volumes you have mounted will also show up in this list. This example shows a disk (in fact, a CD-ROM) mounted on /Volumes/Q3TA. This attribute, known as the disk's "mount point," shows you the path you would take to reach that disk via the CLI. For example, to peek inside the Q3TA CD you would enter ls /Volumes/Q3TA.

You will find all local disks and most network volumes mounted within /Volumes.

The next relevant command simply checks sendmailís queue for any undeliverable messages. If the report doesnít show this directory as empty (and the procedure in this tutorial is your only use of sendmail) , then itís likely you have some sendmail issue.

The daily script then runs the netstat -i command, which outputs the network statistics to the report, a few lines of which look like this:


network:
Name  Mtu   Network       Address            Ipkts Ierrs    Opkts Oerrs  Coll
lo0   16384 <Link>                            2015     0     2015     0     0
lo0   16384 127           localhost           2015     0     2015     0     0
en1   1500  <Link>      00.30.65.xx.xx.xx        0     0        0     0     0
en0   1500  <Link>      00.30.65.xx.xx.xx   646365     0   670550     0     0
en0   1500  192.168       192.168.0.40      646365     0   670550     0     0

The netstat -i command lists your network interfaces in rows, showing traffic statistics for each (since activation) in the columns. This example actually shows just two hardware interfaces: en1 is an inactive Airport card, and en0 is the active Ethernet port.

You will see some other lines for the IP interfaces, including the "local loop" interface (lo0). At this point, youíll be fine to only focus on the lines showing "<link>" in the network column. The other pertinent columns are the actual packet-count columns:


Ipkts -- Incoming packets 
Ierrs -- Incoming packet errors
Opkts -- Outgoing packets
Oerrs -- Outgoing packet errors
Coll -- Packet collisions 

What you should be concerned with, of course, are any non-zero entries in the error and/or collision columns. I wonít go into troubleshooting your network here, but this page might be a good place to start if something does turn up: http://www.princeton.edu/~unix/Solaris/troubleshoot/netstat.html.

Comment on this articleNow that you're up to your knees in command lines, how's it going?
Post your comments

The next important job of the daily script is the "rotation" of the system log. This log file, system.log, records the status and error messages from the large number of processes that comprise the OS.

In the case where no backups of system.log yet exist, the script makes the first backup, compresses it using gzip, and numbers it 0. This results in a file called system.log.0.gz. By "rotating" this log file on subsequent days, the script will first rename system.log.0.gz to system.log.1.gz, and then create a new system.log.0.gz.

Each day, the script creates a new system.log.0.gz file after incrementing the other backup filenames by one. Once system.log.7.gz is created, however, there will be no ninth backup file. Instead, on the subsequent backup the system.log.6.gz file (renamed to system.log.7.gz) just overwrites the previous system.log.7.gz file.

This procedure, then, ensures that youíll have over a weekís worth of logs to refer to in case problems arise, but not so many as to waste disk space, and likely none too large to not view easily.

Next, the script "cleans" the Web server log files by deleting any rotated files that have been around longer than a week.

The Weekly Script

The weekly script performs three important tasks, none of which provides any output to the report except a statement that the command was performed.

Locate

One of the most useful Unix CLI utilities is locate, a lightning-fast file finder. locate does its magic by searching through a database of filenames created by indexing every pathname on your system. Instead of scanning your disks to find a file, locate just whips through its pre-indexed database, and returns results almost immediately.

However, locate results are only as accurate as its database. Files added after the database has been built will not be "located." locate is not the tool for every search, but with weekly database rebuilding, itís great for quickly finding that long-lost file you know is tucked away somewhere on your drive. The first task of the weekly script, then, is to rebuild the locate database.

If you are antsy to try locate for yourself, have a look at the short locate tutorial included in the article found here.

The weekly script updates another important database used by the whatis command. whatis is a nifty little memory jogger that quickly shows you the function of a given command, like this:


[localhost:~] chris% whatis netstat
netstat(1)               - show network status

What whatis displays is, in fact, the first line of a commandís "man page". If youíre not already familiar with man pages, you should be. These comprise the massive collection of online Unix documentation included with Mac OS X. Look here for a great tutorial on using man pages.

The weekly script, then, creates a whatis database from all man pages it finds, allowing whatis to return an answer faster than you can say, "Duh!"

Finally, the weekly script also rotates several other log files, including those for ftp and netinfo.

The Monthly Script

Thereís actually very little to the monthly script, but what it does provide can be pretty interesting, if you like to know where all your time goes. The monthly scriptís first task is to run the "connect time accounting" utility ac. ("Connect" here means "logged in.")

When run from the monthly script, ac will report the cumulative time, in hours, each user account has been logged in since the last time the script ran, as well as the total for all users:


Doing login accounting:
	total      714.22
	chris      548.76
	miho       101.77
	andy        54.39
	jonny        9.18
	test1        0.06
	ftp          0.06


ac calculates these totals by reading the current wtmp log file, which logs every login and logout. You can view this list anytime with the last command:


[localhost:/var/log] chris% last 
chris     ttyp2                     Thu Feb 21 16:18   still logged in
chris     ttyp1                     Thu Feb 21 16:16   still logged in
chris     console  localhost        Thu Feb 21 16:02   still logged in
reboot    ~                         Thu Feb 21 16:01 

So how does ac know to restart the accounting each month? Well, right after ac reports its findings, the monthly script rotates the wtmp logs, creating a new empty wtmp file to start logging to. The next time the monthly script runs, then, ac will do its accounting based on this new file.

Now that you have a good idea of what the three cron jobs do, and what to look for in the reports, I want to take a final look back at the procedure we followed and give a couple of "variations on the theme."

Using Other Mail Clients

Of course, not everyone uses Mail.app as their regular email client, and launching it just to read the cron reports is not very convenient. Itís not an entirely hopeless situation, given these two alternatives:

First, use the command line mail utility. Since by now, you probably always have Terminal open anyway :-), you can just type in mail to read any incoming reports. And as you may have noticed, you are alerted of any new mail to your Unix account whenever you open a new Terminal window. Hereís one tutorial on using mail: http://wss.berkeley.edu/unix/faq/mail.html.

Second, add your regular email address to the ~root/.forward file. If youíre lucky enough to have an email account that this works with, this is your best solution. The reasons it might not work are complicated, but the easiest way to test is to use the CLI mail utility to send yourself an email:


[localhost:~] chris% mail chris@oreilly.com
Subject: Test message
Hey, it worked!
.
EOT
[localhost:~] chris%

If you do get the message, then you can simply add the address to the .forward file, or, if youíll never be checking your Unix mail, replace the one thatís there using pico. Hereís the easy command to get right into it:

[localhost:~] chris% sudo pico ~root/.forward

To add an address, follow the previous one with a comma and a space before entering the next address:

chris, chris@oreilly.com

Mail forwarded to your regular account, then, will show up in whatever email client you use. Using Eudora or Entourage, for example, you could also create a rule to route the cron reports into their own folder.

Eliminating sendmail Permissions Errors

As your remember, we needed to change the permissions for the root directory to get sendmail to work. While in most cases this does work, itís really just a workaround that comes with a few of its own trade-offs. For one, some Apple updaters will revert the permissions, so youíll need to chmod again to get sendmail back on track.

Secondly, and more importantly, there's a reason why Apple wants the root directory to be group-writable. Many Classic installer applications (and even some native ones) are programmed to place items in the root directory, and unless youíve given these applications admin privileges to do this, they just might choke during an installation. Appleís workaround for this possibility, then, is to keep the root directory group-writable.

Of course, that causes problems for sendmail, which requires these permissions for security reasons. At this point, then, you might feel stuck between a rock and a hard place. But wouldnít it be great if you could just tell sendmail (as Ben Franklin might), "Hey, Iím willing to give up a little security if you just give me the liberty to keep my permissions!"

Previously in the Series

Learning the Terminal in Jaguar, Part 3

Learning the Terminal in Jaguar, Part 2

Learning the Terminal in Jaguar, Part 1

Learning the Mac OS X Terminal, Part 5

In fact, sendmail allows you to set just that option by adding a single line to its configuration file. This is from the sendmail documentation:

"You may have to tweak your environment to make it safer for sendmail to run. If you find that some of the safeties in sendmail are too restrictive for your environment, they can be turned off by setting the option DontBlameSendmail. The option is appropriately named as sendmail is not to be blamed for problems resulting from unsafe permissions on directories and files."

As long as youíre using sendmail as described in the tutorial and are the primary user of the machine, the security risk is small in setting this option. If, however, you arenít able to control access to your machine either physically or remotely, and you are compromised, please donít blame me either ;-)

So if youíre ready to go, the file you need to edit is /etc/mail/sendmail.cf. Youíll first want to make a backup. Since the /etc/mail directory is only root-writable, youíll need sudo:

sudo cp -p sendmail.cf sendmail.cf.bak

Note the use of the -p option flag in this command line, which preserves the permissions settings of the original in the copy of the file. This will make things a little easier, should you need to quickly restore the file.

You can then edit this file using pico, as you have with others. Since sendmail.cf is only writable by root, youíll need to use sudo here as well.

sudo pico /etc/mail/sendmail.cf

This file is over 1,200 lines long and might be intimidating, but since youíll just be adding a single line near the top and then getting the heck out, you should have nothing to worry about.

The line youíre looking for is a commented-out "DonBlameSendmail" line about 70 lines down from the top. The quickest way to get there in pico is by pressing Contol + W, entering "DontBlame", and pressing Return. You should then see these lines:


# level 9 config file format
V9/Berkeley

# override file safeties - setting this option compromises system security,
# addressing the actual file configuration problem is preferred
# need to set this before any file actions are encountered in the cf file
#O DontBlameSendmail=safe   
# default LDAP map specification

Next, add a new line after the found line and enter (or paste in) this line:


O DontBlameSendmail=GroupWritableDirPathSafe 

When youíre done, the lines should look like this:


# override file safeties - setting this option compromises system security,
# addressing the actual file configuration problem is preferred
# need to set this before any file actions are encountered in the cf file
#O DontBlameSendmail=safe   
O DontBlameSendmail=GroupWritableDirPathSafe
# default LDAP map specification

As usual, press Control + O to save the file, Return to confirm the name, and Control + X to exit pico. You can now reset the root directoryís permissions to the factory default with this command:

[localhost:/etc/mail] chris% sudo chmod g+w /

If everything went well, sendmail will in fact send its mail the next time itís beckoned, even with a group-writable root directory.

If youíre still having problems with anything, make sure to look at the TalkBack sections for all parts of this tutorial, where readers and I have covered most of the common problems and made some corrections.

Also, if you would like to learn lots more about cron, hereís another tutorial for you.

Now that your feet (or even your knees) are wet working with Terminal and Unix, you have an entire ocean left to explore. I hope this tutorial has given you the confidence to dive in. There are other articles here on the Mac DevCenter you should now be ready for, as well as plenty more around the Internet.

I also plan to have some more of my own articles in the near future, so feel free to make any requests in the TalkBacks. See you there!

Special thanks to Fred Coffman for his help with this article.

Chris Stone is a Senior Macintosh Systems Administrator for O'Reilly, coauthor of Mac OS X in a Nutshell and contributing author to Mac OS X: The Missing Manual, which provides over 40 pages about the Mac OS X Terminal.


Return to the Mac DevCenter.

Copyright © 2009 O'Reilly Media, Inc.