MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


A Briefing on Syncing

by Michael E. Cohen
09/05/2006

Editor's note: The following article is an excerpt from Take Control of Syncing in Tiger, a $10 electronic book available for download from TidBITS Electronic Publishing. This 135-page e-book covers tasks like syncing data between your Mac and mobile phone, iPod, or PDA with Apple and third-party software; offers suggestions for syncing effectively; and provides practical troubleshooting advice.

A Briefing on Syncing

The idea behind synchronizing, or syncing, is simple: take information you collect on one device, like the calendars and contact lists you keep on your laptop or mobile phone, and combine it with similar information on other devices, like your desktop Mac or your iPod, so that the two sets of information match.

Though the idea behind syncing may be simple, putting it into practice requires coming up with reliable solutions to some knotty problems. Here are the biggies:

All tough questions, but ones you don't have to answer because the developers who create syncing applications already have—in fact, Apple itself has answered these questions several times in several ways over the last few years.

The answers that Apple came up with in Tiger, and the path it took to get them, can help you better understand how syncing works (the first step in taking control of syncing), regardless of whether Apple or a third-party developer supplies the solution you end up choosing for your syncing needs.

Apple Syncing's Evolution

Apple's first stab at a syncing solution came in 2002, in the form of iSync for Jaguar, a one-stop syncing solution for portable devices and Macs. Apple designed iSync to work with Jaguar's new personal information applications—iCal and Address Book—providing an easy way for Mac users to sync calendar and contact information with other Macs (via Apple's .Mac online subscription service) and with mobile phones, PDAs, and iPods.

iSync for Panther added more compatible devices, fixed some bugs, and incorporated some minor new features, but it didn't change the basic concept: if you wanted to sync something, you used iSync.

But Apple had already begun to realize that there was more to syncing than just addresses and calendars, and that a more general, adaptable approach to syncing—one that third-party developers as well as Apple could exploit—was needed.

With Mac OS X 10.4 Tiger, Apple has introduced such a general, adaptable syncing strategy: syncing has become a system service that any application can use. In the process, iSync has become only one tool among several that you can use to sync information—the one-stop syncing solution is no more.

Apple's brave new world of system-wide syncing has its pros and cons. On one hand, you can expect to see more applications that take advantage of Tiger's syncing service and that can sync their data between Macs and other devices—for example, as this e-book was being written, Microsoft introduced a new version of Entourage that exploits Tiger's syncing service. On the other hand, you'll have to work harder to remember which applications sync which kinds of information, and with which devices.

No one said that living in the future would be easy.

Apple's Syncing Model

To make syncing into a system-wide service, Apple has devised a model that describes what syncing is and how it works, a model flexible enough to be useful in a variety of programs and devices. If you understand this model, you'll be better able to take control of syncing on your Macintosh and less apt to be confused by any hiccups in the syncing process.

The Players

Apple's syncing model includes three kinds of participants in the syncing process:

Note: Each user account has its own truth database, located in ~/Library/
Application Support/SyncServices/Local
. If you're a .Mac user, part of your truth database also resides among the folders in your iDisk's /Library/Application Support/SyncServices folder. Now that you know this, restrain yourself from modifying the files that make up this database.

The Process

The dance for these three participants involves four steps, which always take place in the same order:

  1. Negotiating: In this first step, a client requests that the engine set up a syncing session between them; the request includes the kind of sync that needs to take place—see "Types of sync sessions,"
    on the next page.

  2. Pushing: Next, the client sends whatever new or changed information it has accrued to the sync engine.

  3. Mingling: The sync engine consults the truth database, inserting new data, marking outdated data, and noting any conflicts that might arise between the pushed information and the truth database's contents. The engine tries to resolve conflicts but might also instruct the client to ask the user (that's you) to resolve them.

  4. Pulling: Finally, the client receives the changes—what's new, what's modified, what's deleted—from the sync engine. The client reformats the information it receives if necessary and records it in the appropriate place, like the Mac's Address Book, or a connected PDA's contact list.

Note: Although the syncing process has four steps that occur in the order given, not all of the steps must be performed in any session. For example, a client can negotiate a session, push nothing, and then pull updates, or it can push updates but pull nothing.

Types of Sync Sessions

A sync session takes place either automatically (usually because you have established a schedule) or manually (usually because you specifically requested it or because you connected a device to your Mac that triggers a sync). Sync sessions can take any of the following forms:

Note: The sync engine, not the client, decides what type of sync session takes place: the engine may perform the type of sync session requested by the client, but it doesn't have to. The sync engine must evaluate the state of all of the clients involved and the state of the truth database when establishing a sync session. Don't be surprised, therefore, when a sync sometimes seems to take much more—or less—time than a seemingly similar sync has previously taken.

The Elements of Syncing

When you look at the stuff on your Mac that you might be interested in syncing, it typically breaks down into two forms:

Syncing whole files and syncing chunks of structured information require different syncing approaches—Choose How to Sync, in the full e-book, provides a summary of the applications you'll use for these two intrinsically different kinds of syncing.

Syncing Structured Information

Most structured information consists of the types of things that traditional database programs manage: bits of text, some numbers, maybe a small image or sound bite. Apple's syncing model implicitly assumes you're syncing such structured, "database-friendly" information. After all, at the heart of the Tiger sync engine is a database: the truth database.

Note: Behind the scenes, Tiger's sync services use another Tiger innovation to manage the truth database's contents: a set of software components known collectively as Core Data. Core Data offers programs various ways to store multiple chunks of information in structured form, including a traditional SQL database.

For both performance and storage capacity reasons, the amount of data handled by the sync engine can't be so large that it makes the truth database unwieldy. After all, the truth database must contain all the current synced data from every client as well as every client's state. If you were to use the truth database to store copies of many large ungainly objects, like Photoshop files or PowerPoint presentations, the database would consume a significant portion of your hard disk and bog down the performance of the sync engine.

In short, Apple's syncing model is designed to work with items from structured information collections like Address Book's contact lists or iCal's event and to-do lists, rather than with whole files.

Syncing Files

When it comes to syncing files, the Tiger sync engine with its truth database just doesn't cut it. Luckily, a database like the truth database isn't needed to sync files between devices.

A file-syncing application typically performs these steps:

  1. It examines each device participating in a sync and locates the files that are to be synced on each device. (As you'll see, it's OK if one or more of the devices doesn't contain copies of the files at all.)
  2. The application then determines which of the copies of each file among the various devices have been modified most recently.
  3. It pushes the latest version of each file to those devices that either don't have that file, or that have an older version of the file (in which case, the older version is overwritten).

See? No database required: just the basic date and time information associated with each file on each device.

However, even though Tiger's sync engine doesn't handle whole files, Tiger does provide other methods and applications to sync files; see, for example, Sync Your Files with iDisk in the full e-book. And, in many of those cases where Tiger's file-syncing capabilities fall short, third-party developers have stepped in with their own syncing tools.

Syncing Versus Backups

Some people confuse syncing with backing up, which is not surprising because each employs similar techniques and strategies. Both syncing and backing up take the information you have stored in one place and copy it to another place, and both usually allow copying of only those items that have changed since the last sync or backup.

The difference lies in this point: syncs are about space, and backups are about time.

When you sync, you make your data available for use in two or more different places. You can then work with that data in those places independently of each other, and later use the syncing process again to resolve any changes and conflicts that occur.

When you back up your data, on the other hand, you store your data as it existed at that particular time so that, if necessary, you can revert to it: you never actually use the stored data in a backup—or, at least, you really, really, shouldn't use it—other than to copy it back to your working environment.

Although you can use syncing as an informal way of backing up some of your information, you can't rely on it to get your information back to the way it was at any specific time, nor can you guarantee that a subsequent sync won't alter or eliminate something you may need later. That's what backups are for: to let you go back in time.

Tip: I recommend Take Control of Mac OS X Backups by Joe Kissell (http://www.takecontrolbooks.com/backup-macosx.html) for an in-depth look at backing up your Mac.

Michael E. Cohen has worked as a teacher, a programmer, a Web designer, a multimedia producer, and a certified usability analyst. He's the author or co-author of several books; his latest is Take Control of Syncing in Tiger (TidBITS Electronic Publishing, 2006).


Return to the Mac DevCenter.

Copyright © 2009 O'Reilly Media, Inc.