MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


Sync Services Framework (How It Works and What You Can Do)

by Mary Norbury-Glaser
11/04/2005

The Old

With Tiger, Apple introduced a new extensible Sync Services framework embedded into the OS X 10.4 operating system. With a .Mac membership, Tiger users can take advantage of new functionality and features across many applications and across multiple Macs.

In Panther, syncing was handled by the iSync application that, together with an optional .Mac membership, managed Safari bookmarks, iCal, and Address Book over many networked Macs. In Tiger, iSync.app still exists, but now takes the limited role of synchronizing iPod, cell phones, and Palm devices with your Mac.

figure 1
Figure 1. /Applications/iSync.app

figure 2
Figure 2. iSync.app message

figure 3
Figure 3. Adding devices in iSync.app

And The New

Tiger's expanded Sync Services are now accessed through the .Mac pref pane in System Preferences.

figure 4
Figure 4. .Mac in Systems Preferences

figure 5
Figure 5. Sync tab in .Mac pref pane

The new framework includes synchronizing Mail account settings (including Smart Mailboxes, signatures, and rules) and Keychain items between Macs.

figure 6
Figure 6. Computers registered with .Mac

Sync Services is also available to any application, not just Apple programs like iCal and Address Book. Developers have the ability to embed the Sync Services framework into their third-party applications to offer iSync integration and functionality. Syncing preferences can therefore be set within individual applications. In addition to looking at how some Apple apps handle syncing, I'll also include one third-party program that has taken advantage of this: Mark/Space's Missing Sync for hiptop. First, let's get a better feel for the Sync Services architecture.

Mac OS X: The Missing Manual

Related Reading

Mac OS X: The Missing Manual
By David Pogue

A Closer Look at Sync Services

The Sync Services framework is based on a client (applications)/server (sync engine) model. The client(s) pushes data to the sync engine, the sync engine applies those changes to a truth database (a master copy of all of the information for the clients that use the Sync Services framework), and then the client(s) pulls the changes back.

The truth database and client processes reside on one computer and each user account has one sync engine and one truth database. The sync engine organizes and merges the data received from the many sync-enabled clients and stores this data in the truth database. The sync engine uses field-differencing to process changes made to records, as well as to discrete fields in a record.

There are several possible types of clients: an end-user application like iCal or a Cocoa app, a server application like .Mac, or an application that syncs a device like a phone or PDA. When a client registers with the sync engine, a property list file is created.

The client description, which includes schemas (Attributes, Data Class, etc.) and properties (Type, DisplayName, Entities, etc.), is stored in this file. The three Tiger sync schemas can be found in the /System/Library/SyncServices/Schemas directory.

Clients communicate directly with the sync engine; clients do not sync with other clients or through another client's processes. Multiple clients can make push/pull requests simultaneously and faster devices can synchronize independently of slower ones. Sync Services supports a diverse range of clients and is meant to enable efficient syncing of user data: fast, frequent syncing of small data sets. Whole records are not pushed or pulled when only a few property values are changed.

This is called "trickle syncing." The process should be automatic, transparent to the user, and not require user intervention. Sync operations can be cancelled or postponed by the user, without data loss.

Developers can build sync-enabled applications in several ways:

  1. Adopt a schema that uses an existing data type. For example, Address Book or iCal.
  2. Extend existing schemas. For example, add fields to the system contacts schema in order to sync these special attributes across multiple Macs.
  3. Create a new schema for new types (sync custom objects).

The sync process involves five steps:

  1. Initiate a sync. Occurs when a client wants to push or pull data from the sync engine.
  2. Manage and negotiate the sync mode. A client can sync with the sync engine using several modes: slow, fast, refresh, push the truth, and pull the truth. A slow sync occurs the first time a client syncs. The client pushes all of its data to the sync engine, not just the changes. The sync engine creates a snapshot to compare future syncs against. Fast syncs will occur after this first time, with only changes passing back and forth between the client and the sync engine. A refresh sync happens, for example, when a client device is reset and all data is lost. A refresh request tells the sync engine to delete the client snapshot. The client then pushes no records and pulls all of the records still residing in the truth database. Push the truth describes the situation when a client wants to replace all of the data in the sync engine with its current records. Conversely, a client may want to replace all of its current data with the records residing in the truth database. This is called pull the truth.
  3. Push changes. A client can push either the whole record or just the properties where changes have been made.
  4. Resolve changes. The sync engine determines which changes belong to which record by using specific properties defined by the client(s) taking part in the sync. Unresolved conflicts are passed back to the user.
  5. Pull changes. Each client participating in the sync session gets only the data it needs from the sync engine.

You can view sync log files in ~/Library/Logs/Sync/ using either the default Console.app or, my favorite, BBEdit.

Sync Services in Action

Let's take a look at sharing an Address Book using .Mac and sync services. I keep track of not only my contacts but also my son Jack's contacts, because he's only eight and not a great speller (and I want to know who his friends are!). I enter his friends in a new group called Jack.

Once that's done, I can share out my Address Book.

1. Open Address Book Preferences and select the Sharing icon:

figure 7
Figure 7. Address Book sharing

2. Check the "Share your Address Book" option:

figure 8
Figure 8. Share Address Book

3. Add the .Mac user you'd like to share your calendar(s) with.

figure 9
Figure 9. Add .Mac member

4. Use the Allow Editing option to give write access to your calendar(s).

figure 10
Figure 10. Allow or deny editing

5. Send an invitation to the .Mac member.

figure 11
Figure 11. Invite .Mac member

Back on the Mac mini, Jack confirms that his sync services are running, checks his email, and sees my invitation.

figure 12
Figure 12. Email invitation to share my Address Book

He clicks on the link to subscribe to my Address Book and his iCal.app opens, asking if he wants to subscribe.

figure 13
Figure 13. Calendar subscription confirmation

And ta-da! He can now look up his friends' email addresses and phone numbers through my published Address Book.

figure 14
Figure 14. My Address Book in Jack's Address Book

Jack has a pretty busy schedule. I keep track of his activities, but want him to learn to be responsible for knowing where he has to be. To keep us both current, I set up a "Jack" calendar inside my iCal app, populate it with his calendar events and then publish it:

1. Open iCal. Highlight the Calendar you want to publish, go to Calendar in the menu bar, and select Publish.

figure 15
Figure 15. Select a calendar to publish

2. The default calendar name shows up with several options to choose from. Click the Publish button to finish.

figure 16
Figure 16. Publish the calendar

3. You'll get some information on how to subscribe to the calendar and how to just view it via a browser.

figure 17
Figure 17. Subscribe and view information

4. There is also a button called Send Mail, which will open your Mail.app and prepare an email with this information. You only need to add the recipient email address(es).

figure 18
Figure 18. Email announcement

When Jack receives the subscription announcement via email, he can choose to either view the calendar on the Web or subscribe to it.

figure 19
Figure 19. View the calendar or subscribe

figure 20
Figure 20. View the web version

figure 21
Figure 21. Subscribe to the calendar

After he clicks on the Subscribe button, he can rename the calendar or leave it as the default. He can also choose to refresh the calendar at intervals and remove alarms and To Do items.

figure 22
Figure 22. Calendar options

Jack now has a calendar that I keep current for him.

figure 23
Figure 23. Jack's subscribed calendar

Missing Sync for hiptop/Sidekick from Mark/Space

Sync Services is a low-level API that offers developers great control and flexibility. My favorite example of a success story is Mark/Space's Missing Sync for hiptop, also known as the T-Mobile Sidekick. For several years, I've been a Sidekick owner, but haven't been able to synchronize the data from my Mac to the device. I used it anyway and hand-entered my contact and calendar information. I fielded a lot of hungry queries from Mac owners every time I pulled it out at MacWorld, WWDC, or at O'Reilly conferences.

I hated disappointing potential buyers but while I raved about the device, I had to confess that I couldn't as yet sync it with my Mac. Mac owners are notorious for loving beautifully designed and cool devices, so it's no surprise that many of us bought and used the hiptop and trusted that we'd get it working with our Macs eventually. I haunted the Mark/Space hiptop list, patiently waiting for news.

In spring of 2005, Mark/Space finally released beta software, and then version 1.0 release software for Panther. Of course, with Tiger imminently due to arrive and with the new Sync Services framework integrated into the OS, the waiting game began again for a new version of Missing Sync for version 10.4 of OS X. Patience is a virtue--and in late August, Sidekick users with Tiger installed on their Macs could finally breath a sigh of relief. The new Tiger version of Missing Sync for hiptop is a great illustration of an application that has been retooled to take advantage of Tiger's Sync Services framework. In the Panther version of the software, you added the hiptop device through a plugin using iSync.app. The Tiger upgrade includes a standalone Missing Sync application:

figure 24
Figure 24. Missing Sync for hiptop.app

1. As a precaution, back up your iCal and Address book databases by selecting the appropriate backup option under File in the menu bar of the particular application (iCal -> File -> Backup Database... and Address Book -> File -> Back Up Address Book).
2. Go to Mark/Space's website and download the software ($29.95, free upgrade for version 1.0/Panther registered users) and double-click the Missing Sync for hiptop installer package.

figure 25
Figure 25. Begin installation

3. Enter your contact information and the registration code (you will have received a registration code via email with your paid download).

figure 26
Figure 26. Enter contact information and registration code

4. The application will initiate a connection through the internet and verify your registration. This effectively registers your computer with Mark/Space. You are allowed to install the software on two computers and Mark/Space keeps track!

figure 27
Figure 27. Register

figure 28
Figure 28. Registration confirmation

5. Go fetch your hiptop/Sidekick and turn it on. Double-click the Missing Sync for hiptop.app and you'll need to fill in your cellular provider account information. This may actually be a username instead of your account number. To check for your username, turn on your hiptop/Sidekick and go to Menu -> Settings -> System Info. Select all of your calendars or check the specific calendars you want to sync to your hiptop. You'll also need to designate which one of your contact groups you want to sync (if you have more than one) and one calendar name you want your events to show up under, on your hiptop.

figure 29
Figure 29. Initial Missing Sync for hiptop preferences

6. Click OK and you're ready to begin the first sync. Remember, the first sync under Sync Services will take a longer time than subsequent syncs, because the client is registering with the sync engine and the truth database is being populated (slow sync).

figure 30
Figure 30. Initial slow sync

7. You'll be asked to verify a sync with your Contacts with Address Book and other devices (note that my nano is included). You have the option to delete all your Contacts from your hiptop before syncing.

figure 31
Figure 31. Contacts sync

You'll see a progress bar.

figure 32
Figure 32. Syncing progress bar

8. Then, you'll be asked to verify a sync with your Calendar with iCal and other applications (again, my nano is listed) and you'll also be able to zero your hiptop calendar items first, if you've hand-entered some.

figure 33
Figure 33. Calendar sync

When the syncing completes, you'll be notified.

figure 34
Figure 34. Missing Sync for hiptop initial sync completed

9. Check out your hiptop or Sidekick, and you'll see your address book entries and calendar items!

Some caveats to be aware of:

Some current (temporary!) Missing Sync issues:

figure 35
Figure 35. Debug options

Enable Debug Logging and you can view the Missing Sync for hiptop.log file in ~/Library/Logs/.

figure 36
Figure 36. Using Console.app to view logs

Usually, after a "Failed" entry, you will find the record that caused the problem.

Final Thoughts

Apple is the innovator in integrated devices, applications and services. The Sync Services framework opens up a previously insular API to third-party development and extension. Taking advantage of the opportunity to incorporate non-Apple clients into the Sync Services framework is an exciting challenge for developers. What Sync Services doesn't supply at this time, however, is a way to sync files and global preferences. It would be a natural extension of the framework to enable file and preference synchronization across multiple Macs. Also, Apple hasn't extended network services outside of .Mac for obvious reasons, so that functionality isn't open to third-party providers. But no doubt the future will show Sync Services evolving into a broader framework and Apple will provide a wider range of integrated solutions involving this new service.

Mary Norbury-Glaser is the IT director at a University of Colorado affiliate center. She has over 15 years of experience in cross-platform systems administration in the education sector. She loves fast cars and geocaching.


Return to the Mac DevCenter

Copyright © 2009 O'Reilly Media, Inc.