MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


An Introduction to Tiger Terminal, Part 5

by Mary Norbury-Glaser
11/15/2005

In previous parts of this Tiger Terminal series (Part 1, Part 2, Part 3, & Part 4), I introduced the OS 10.4 version of the terminal.app, shell basics, CLI network utilities, and shell scripting. In Part 5, I'll show you how Tiger Mac OS X runs regularly scheduled commands and scripts to execute recurring jobs, like system maintenance and backups by comparing the "old" way, using the Unix tool called cron (for chronological), with the new Tiger method of using the launchd daemon.

A daemon is a background process or program that runs unattended. It's usually committed to perform systemwide functions. cron is a Unix daemon that enables users to execute scripts or commands automatically at a specified day and time. With Tiger OS X 10.4, Apple introduced a new tool called launchd that is intended to replace cron and, as we'll soon see, many other traditional Unix-style services. To best explain, I'll look at cron because it's so ubiquitous in the Unix/Linux world and then I'll discuss launchd, including how it works, how to use it and what it may mean for the future. But first . . .

Why launchd?

There are many processes and subsystems involved in booting a traditional Unix-style operating system (including previous versions of OS X), in launching applications at specific times, and in launching on demand: init, mach_init, (x)inetd, cron, rc, watchdog, at, startup items, and login items.

The first process on a Unix system is init but its role varies between different systems: on Linux boxes, init maintains system run levels and launches services on demand; on BSD systems, init runs startup scripts and periodically launches getty (which reads and invokes login). In previous versions of OS X, mach_init would run core system services (see Figure 1); StartupItems, init.d, or rc/rc.local would start boot time services; xinetd started network services; login items would come from a user's .profile or .xinitrc or from StartupItems; and cron would run scheduled services. These are all still available in Tiger but the intention is clearly to move away from using the "old" methods for new services and transfer them to launchd instead.

If you were to run the ps command (get the process status of all users; man ps) on a Panther system, you would see that the first two PIDs (Process IDs) are init and mach_init (although mach_init has a PID of 2, it is started first on the system and init is started second). Run the ps command on a Tiger system, and you'd see that launchd has replaced both these processes and has a PID of 1.

figure 1
Figure 1. Panther ps ax vs. Tiger ps ax

With Tiger, Apple took a daring approach by deciding to supersede and eventually replace these and other services with one system startup daemon, launchd. By providing a unified interface for managing programs started by the system, launchd can do all the jobs that the myriad Unix programs described above can: launchd can manage daemons for the entire system or for a single user; it can launch daemons on demand and monitor them. It can, therefore, speed up the boot process and provide efficient administration for managing services.

The Old . . .

As an example of how scheduling maintenance tasks have changed in Tiger, let's concentrate on one of the daemons, Vixie cron (the version of cron in OS X), how it functioned in Panther, OS X 10.3, and its role in Tiger, 10.4.

In Panther, cron is started at boot by SystemStarter and runs continuously in the background (it's a daemon, remember?). cron checks tables (called crontab files) in two directories for new instructions or tasks that need to be executed. One directory is /var/cron/tabs/ which contains the user crontab files (visible only by root). The other is the one we're interested in: /etc/crontab. This is the crontab file for the entire system (anybody can view this file but only root can edit it). Figure 2 is a crontab for Panther. To open this file, you'll need to cd to the /etc directory and then type sudo pico crontab at the command prompt (remember, you have to be running OS X 10.3 to get this, and remember pico is the basic default editor in Panther).

figure 2
Figure 2. Panther crontab

This should look pretty familiar to you if you've been following along with Parts 1 - 4 of this series! The top line is a comment line indicating the location and name of the file. The next few lines are our shell and path descriptions. The next set of commented lines are read left to right and represent minute, hour, day of the month (mday), month, and weekday (wday) in numerical format. The asterisk represents "every." The user of the job (which tells cron to run it as if root were running it) and the task to be run (periodic daily, periodic weekly or periodic monthly) completes the line. So the periodic daily task will run at 3:15 a.m. on every day of the month, every month, and every day of the week.

Learning Unix for Mac OS X Tiger

Related Reading

Learning Unix for Mac OS X Tiger
By Dave Taylor

You can edit the times if you want the tasks to run at more reasonable times. Use the arrow keys to move the cursor to the field you want to change, delete number and type in a new one. For example, if you wanted to change the weekly script from 4:30 a.m. to 10:00 p.m. and from day 6 (Saturday) to day 7 (Sunday), you would change this line:

30 4 * * 6 root periodic weekly

to this line:

0 22 * * 7 root periodic weekly

Each of the three command lines in the crontab file runs the periodic command along with the argument of daily, weekly, or monthly. The argument (daily, weekly, or monthly) to periodic is the name of a directory inside the /etc/periodic directory. For example, periodic daily means the periodic command looks inside the /etc/periodic/daily directory and alphabetically executes every file in that directory. There are two files in daily (100.clean-logs and 500.daily) and one (500.daily) in each of the weekly and monthly directories.

Where is cron in Tiger? Well, cron is still there on the system, although deprecated, most likely for backward compatibility issues due to the evolving nature of launchd. Open crontab using nano at the command line (sudo nano /etc/crontab) and you'll be prompted for your password:

figure 3
Figure 3. sudo nano /etc/crontab

Then you'll see this message:

figure 4
Figure 4. System crontab message

Or, since you're just reading the file, you could have used the cat command to look at the contents of the crontab file:

figure 5
Figure 5. cat /etc/crontab

In OS X 10.4, instead of starting at boot and checking in every minute, cron is only started when called. The system crontab still exists in the /etc directory and cron will continue to work if you add a new cron job to the directory /var/cron/tabs or if you change the crontab file in the /etc directory.

In order to utilize cron in Tiger and create your own scheduled job, you would use the crontab command with the -e option (for edit). This command uses the vi editor by default. Since we're more comfortable using nano, let's see about changing that because we'll be creating our own cron job. First, open terminal.app and let's see what environment variables are defined by using the command, env.

figure 6
Figure 6. Looking at environment variables using the env command

We can set our default editor for crontab by adding a $VISUAL variable and naming nano.

figure 7
Figure 7. Adding $VISUAL variable

We've set our editor using the export VISUAL=nano command to send the variable into the environment. This means that the variable will be maintained if our shell creates any child processes (but not if we close the terminal!). Then, we tested it by using the echo command.

If you want to make this change permanent, you would edit the bash shell config file in your home directory (refer to Part 1 if you need a refresher).

figure 8
Figure 8. Edit your bash config file

Add the environment variable at the end of your .bash_profile configuration file.

figure 9
Figure 9. Add the environment variable

Save it (^X, confirm with Y then hit the Enter key to write to .bash_profile).

Close the terminal.app window, open a new window and test your changes by typing echo $VISUAL.

figure 10
Figure 10. echo $VISUAL

Now we can create a simple cron job and have it run every week at a set day and time. Our job will repair permissions on our boot volume and then output the results to a file on our desktop. The diskutil command is the CLI version of the Disk Utility app, found in /Applications/Utilities/. Type crontab -e at the command line.

figure 11
Figure 11. crontab -e

Recall that we made nano our default editor above so you won't have to learn vi just for this exercise. We'll be telling cron to run our job every particular weekday at a particular time. Since this is an exercise and we don't want to wait forever for this to run, I'm going to set mine for 5 minutes from now. If you need a bit more time, plan accordingly. You'll tab between fields, with the fields running: minute (0-59), hour (0-23), day of the month (1-31), month (1-12), day of the week (0-7 with Monday=1, Sunday=7), and command to be executed. The "/" after the repairPermissions option indicates that the command is run on the startup volume.

Set this for a few minutes from now. Note that my username (norburym) is indicated. Use yours instead! Don't worry about the ugly file name!

Here is a screen shot of the command in nano:

figure 12
Figure 12. cron job

If you wanted to see the display as output, type crontab -l at the command line:

05 20 * * 1 /usr/sbin/diskutil repairPermissions / >> /Users/norburym/Desktop/fixed.txt

When cron runs this job, you'll see a text file appear on your desktop called fixed.txt:

figure 13
Figure 13. fixed.txt file generated by our cron job

To remove the cron job, go to the command line and type:

crontab -r

Check to make sure it's gone by typing crontab -l and you should see:

crontab: no crontab for yourusername

The New . . .

Now it's time to look at launchd. The launchd framework/system is actually made up of the daemon launchd and launchctl, the utility that interfaces with launchd. With launchctl, we can load and unload daemons/agents (also known as jobs) and otherwise control launchd through environment settings, logging, system resource usage stats and starting/stopping launchd jobs (also known as agents!). Now, let's use launchctl to see what jobs are loaded into launchd:

figure 14
Figure 14. sudo launchctl list

When Mac OS X boots, the BootROM checks a test of the hardware and then activates Open Firmware which selects the operating system. Next, BootX loads the kernel and drivers and then the kernel loads launchd. launchd runs /etc/rc which runs SystemStarter which, in turn, runs a series of scripts found in both /System/Library and /Library. launchd then scans through /System/Library/LaunchDaemons and /Library/LaunchDaemons for plist files (more on this in a bit), runs the necessary plist files and then starts loginwindow. launchd manages daemons during startup and continues running until shutdown.

Let's head over to /System/Library/LaunchDaemons . . . via the command line, now! Type ls to see what's here. Notice the difference between the daemons available (Figure 15 below) and the daemons actually loaded (Figure 14 above).

figure 15
Figure 15. /System/Library/LaunchDaemons

So we see that each of the periodic jobs have been replaced by three launch daemons in plist format. Huh? The Property List, or plist, format is used in Mac OS X/Darwin to store application user preferences and system configuration files in XML (eXtensible Markup Language). plists are human- and computer-readable plain text files. You create an XML file using the plist format. Then, you use launchctl to load the file into launchd.

There are different specific locations for the launchd XML files depending on whether they are for system-wide agents (/Library/LaunchDaemons), per-user agents (provided by the user: ~/Library/LaunchAgents; provided by the administrator: /Library/LaunchAgents), system-wide installed by the OS (/System/Library/LaunchAgents), etc.

How to Edit launchd Agent plists

Like most things, you have a choice between the CLI and the GUI. Graphically, you can use a plist editor to view and edit plist files. If you have Apple Developer Tools installed, you will have Property List Editor in /Developer/Applications. There is also a very nice third party developer plist editor specifically for launchd plist editing called Launchd Editor by codepoetry. This is a $5 shareware app and well worth it.

Read the man pages for the cryptically called launchd.plist for options and parameters (launchd.plist doesn't actually exist; it's merely a reference name).

Go to /System/Library/LaunchDaemons and double-click the com.apple.periodic-daily.plist file. It should automatically open Property List Editor.

figure 16
Figure 16. com.apple.periodic-daily.plist in Property List Editor

There are over 30 different keys that can be set (and, IMHO, the syntax is ugly). The most important are Label and Program or ProgramArguments. The Label needs to be unique and Apple prefers (but doesn't require) that you use a domain scheme for naming. The Program or ProgramArguments key indicates the path to the executable that you want to run. The difference between the two lies in how you choose to specify any arguments: if you choose to use Program, you give the full path to the program and then indicate the arguments in ProgramArguments; if you choose to use only ProgramArguments, the first value in the array is the executable (see /usr/sbin/periodic in Figure 16, above) and the subsequent array values are arguments (see daily in Figure 16, above).

Using Launchd Editor: File - Open, navigate to the file (or: navigate to the file, control-click it, select Open With... and choose Launchd Editor).

figure 17
Figure 17. com.apple.periodic-daily.plist in Launchd Editor, General tab

figure 18
Figure 18. com.apple.periodic-daily.plist in Launchd Editor, Startup tab

figure 19
Figure 19. com.apple.periodic-daily.plist in Launchd Editor, Configuration tab

Remember that these files belong to the global system so don't mess with these jobs! The StartCalendarInterval key specifies when the job will be run. This launch daemon, com.apple.periodic-daily.plist, runs its corresponding scripts in the /etc/periodic/daily directory: 100.clean-logs and 500.daily. If you want to run these outside of the hours indicated in the plist file (i.e. on demand), you can use the periodic command.

figure 20
Figure 20. Run the daily/weekly/monthly jobs

You'll be asked for your password.

figure 21
Figure 21. Enter your password

The jobs will take several minutes to run.

figure 22
Figure 22. Jobs running

You'll be put back at the command prompt when the jobs have completed.

Roll Your Own

You now have enough information to create a launchd job. Our launchd example will run a simple shell script (review Part 4) that tells us to take a break from all this scripting and walk the dogs.

1. First, create the shell script. cd to your ~/Documents/scripts folder and nano walkdogs.sh as in the next figure.

figure 23
Figure 23. Create walkdogs shell script

2. Type in the following script (substitute the names of your own dog(s), or kids, for that matter!). Crank up the volume on your Mac and experiment with some of the other voices in the Speech pref pane, under the Text to Speech tab.

figure 24
Figure 24. walkdogs.sh shell script

3. Save it with ^X, Y for yes and then type in the name of the script (walkdogs.sh) and hit enter to write the file. Change permissions with chmod!

figure 25
Figure 25. chmod +x walkdogs.sh

Now we can write a plist file using Property List Editor or Launchd Editor or we can open another plist file that already exists and modify it for our own use. Hmmmm . . . decisions, decisions. Let's look at an existing plist file and then use that as our template in Property List Editor (since it's installed free with Apple Developers Tools; but don't discount Launchd Editor, it's great).

You'll need to create a user LaunchAgents directory (our job only affects the user, not the entire system, so it will live in ~/Library/LaunchAgents). It's very cool that launchd can run on a per user basis (to replace user login items) but the directory is not there by default so you'll need to create it.

figure 26
Figure 26. Create user-specific LaunchAgent directory

5. Now copy the com.apple.periodic-daily.plist file from /System/Library/LaunchDaemons to this new directory (no stinking GUI!) then check that it's there: cd to ~/Library/LaunchAgents the directory and then do ls to see the file listed:

tiger12:~ norburym$ cp /System/Library/LaunchDaemons/com.apple.periodic-daily.plist /Users/norburym/Library/LaunchAgents/walkdogs.plist
tiger12:~ norburym$ cd ~/Library/LaunchAgents
tiger12:~/Library/LaunchAgents norburym$ ls
walkdogs.plist
tiger12:~/Library/LaunchAgents norburym$

6. Open your new walkdogs file by double-clicking it. It will open in Property List Editor by default. Remember that this is actually the cron plist file.

figure 27
Figure 27. Open walkdogs/periodic-daily plist file

7. We'll make a few changes to it, save it and let it run. First, expand the arrow in front of Root.

figure 28
Figure 28. Expanded

8. The Label key is the launchd identifier so let's name this file walkdogs. Double-click the field with com.apple.periodic-daily and it will become editable. Type in walkdogs.

9. Under Root, delete the LowPriorityIO key and the Nice key.

10. Change the ProgramArguments string field to the full path location to our walkdogs.sh script. Delete the key labeled "1"

11. Delete the StartCalendarInterval (StartCalendarInterval is buggy: it works once and then not again so we'll avoid it in this exercise). The minimum you need to create a plist file is the Label key and the ProgramArguments key.

figure 29
Figure 29. Finished walkdogs plist file

12. Save the file. From the terminal, cat the file to see what it looks like.

figure 30
Figure 30. cat ~/Library/LaunchAgents/walkdogs.plist

13. Let's load the job into launchd's list of jobs with launchctl:

tiger12:~ norburym$ launchctl load /Users/norburym/Library/LaunchAgents/walkdogs.plist
tiger12:~ norburym$

14. You can check this by running launchctl list and you will see your new walkdogs plist:

tiger12:~ norburym$ launchctl list
walkdogs
tiger12:~ norburym$

15. To test this, run the launchctl start command:

tiger12:~ norburym$ launchctl start walkdogs
tiger12:~ norburym$

Notice that I don't have to be in the directory where the file is located to start it. Volume cranked up? You should hear "Hey! Blue and Titus need a walk! It's (the current time)."

Final Thoughts

launchd boots the system and launches root-controlled daemons like xinetd, ssh, ftp, etc. and it can also launch user-specific agents, like our walkdogs example above. launchd now gives us a unified service management in OS X.

Although launchd provides similar capabilities to cron, it doesn't allow for the same level of control over commands and scripts that cron does. At least not right now! Plus, there are still some bugs like StartCalendarInterval working one time only. So cron is still required for some jobs for the time being. But with time, launchd will grow more robust, and backwards compatibility will no longer be necessary.

Mary Norbury-Glaser is the IT director at a University of Colorado affiliate center. She has over 15 years of experience in cross-platform systems administration in the education sector. She loves fast cars and geocaching.


Return to the Mac DevCenter

Copyright © 2009 O'Reilly Media, Inc.