MacDevCenter    
 Published on MacDevCenter (http://www.macdevcenter.com/)
 See this if you're having trouble printing code examples


Creating Spotlight Plugins

by Matthew Russell
07/15/2005

Last time, we used Spotlight to query and manipulate metadata for existing files. This is great, but what if you want Spotlight to index a file custom file format, perhaps some marvelous format that you created yourself? Don't fret. Spotlight was designed to be extensible and to accommodate any file format. In this piece, you'll learn how to index a custom file format by creating a Spotlight plugin.

An Overview

A Spotlight plugin simply parses and returns the useful metadata from a file to the Spotlight server; the server handles the rest. Apple has provided a nice example of creating a Spotlight plugin that you should skim over before proceeding. It's located in /Developer/Examples/Metadata, includes a simple application that creates a custom file format, and supplies a Spotlight plugin for that custom file format. In particular, take note of these highlights:

Creating a Spotlight Plugin for Stickies

If you haven't noticed by now, Spotlight does not index the notes that Stickies creates. This is at least in part because Spotlight indexes files by their extension, and each file extension should map to a unique Uniform Type Identifiers (UTI). Unfortunately, Stickies stores all of its information in ~/Library/StickiesDatabase: a single file encapsulating multiple notes that has no extension. This situation is basically the worst possible scenario, but we can have some good fun working with it. The ideal situation would have been for Stickies to create a separate file on disk for each note, and for each of these files to have an extension. This is the new approach used by Mail as of Tiger's release. (Previously, Mail had stored the information in a single Unix-style "mbox" file.)

There are only a few basic steps to creating a Spotlight plugin, and Xcode really streamlines the process. All that you really have to do is use the template Xcode provides, define a method body, and make a few other minor changes. Since Stickies poses such an interesting situation, we'll have to do some clever hacking. The basic procedure we use will work in virtually any other situation, so this makes for a very practical learning experience.

Prepping Stickies

The ~/Library/StickiesDatabase file does not have an extension. Creating a soft link with an extension can't help us any and, incidentally, neither can creating a hard link. (It appears that, Stickies rewrites the file each time it processes an update, so the file's inode changes, breaking the hard link.) Changing the applicable occurrences of the "StickiesDatabase" string in the bundled Mach-O executable with a hex editor would be the easiest approach to fix the dilemma, but would have violated Apple's End User License (specifically section 2.C), so we'll stay away from that. Instead, we'll simply have cron periodically copy and rename "StickiesDatabase" to a file with an extension. (Check here for a cron tutorial, or check the man pages.) In Terminal, type crontab -e to open up the table for editing, paste in the following line, and save it.


*   *   *   *   *   cp ~/Library/StickiesDatabase ~/Library/StickiesData.aaa

This line says to copy and rename "StickiesDatabase" to "StickiesData.aaa every minute. The "aaa" is an arbitrary extension that I made up and is highly unlikely to create a conflict with existing applications. This approach saves us from violating Apple's End User License, and still creates the desired effect (with a slight delay involved).

Now that we'll always have a "fresh" StickiesDatabase file with an extension in place, we can proceed. Here's a quick overview of what we'll do:

Create a New Project in Xcode

When you create your new project, choose "Metadata Importer" from under "Standard Apple Plugins," and name the project, "StickiesImporter." Take a moment to peruse "GetMetadataForFile.c." The file "main.c" is interesting, but mostly cookie cutter stuff we don't necessarily have to be concerned with at an application programming level.

create metadata importer
Create a "Metadata Importer" using Xcode.

Set the UTI Types the Importer Supports

Our contrived UTI type is specifically for the StickiesDatabase format and designated by an "aaa" extension, which is arbitrarily picked and unlikely to cause any conflicts. In "Info.plist," find the block of code that's commented out (shown below), and remove the comments. Add our UTI type designated by "aaa" and customize the file where applicable. Here's the first relevant code snippet with the interesting lines in bold:


<key>UTExportedTypeDeclarations</key>
<array>
    <dict>
        <key>UTTypeIdentifier</key>
        <string>com.apple.stickies.aaa</string>
        <key>UTTypeReferenceURL</key>
        <string>http://www.company.com/yourproduct</string>
        <key>UTTypeDescription</key>
        <string>The StickiesDatabase Format</string>
        <key>UTTypeConformsTo</key>
        <array>
            <string>public.data</string>
            <string>public.composite-content</string>
        </array>
        <key>UTTypeTagSpecification</key>
        <dict>
            <key>com.apple.ostype</key>
            <string>T78q</string>
            <key>public.filename-extension</key>
            <array>
                <string>aaa</string>
            </array>
        </dict>
    </dict>
</array>

At least two other changes need to happen. One of these changes is another UTI-type placeholder:


<key>LSItemContentTypes</key>
<array>
    <string>com.apple.stickies.aaa</string>
</array>

The other change is with regard to our bundle identifier:


<key>CFBundleIdentifier</key>
<string>com.apple.stickies</string>

It's crucial to understand the format of the "Info.plist" file. A single typo in the wrong spot or swapping "com.foo.whatever.aaa" with just "aaa" will cause your plugin to fail, and these types of errors can be difficult to track down.

Update the schema.xml File

While the "Info.plist" file primarily handles information that pertains to a file's UTI, the "schema.xml" file handles the file's actual metadata attributes—including custom ones that are specific to our application. In our case, this flexibility is crucial to getting our importer up and running. Since Stickies can store an arbitrary number of notes in a single file, we'll use a multivalued string attribute, where each value of the string will correspond to the text of a note. Without the availability of multivalued attributes, we'd be stuck placing an upper limit on the number of notes that could be indexed.

Here's an updated "schema.xml" file. The only notable changes are that the reverse DNS suffix placeholders were updated, and we identified our multivalued attribute.


<attributes>
    <attribute name="com_apple_stickies_aaa_noteTitle" 
               multivalued="true" 
               type="CFString"/>
</attributes>

<types>
    <type name="com.apple.stickies.aaa">
        <note>
        The keys that this metadata importer handles.
        </note>
        <allattrs>
            com_apple_stickies_aaa_noteTitle
        </allattrs>
        <displayattrs>
            com_apple_stickies_aaa_noteTitle
        </displayattrs>
    </type>
</types>

The "schema.strings" file (created automatically as part of a "Spotlight Plugin" project) facilitates localization by mapping attribute names to various languages. This kind of support keeps people all over the globe happy. If you want to support more than one language, this is the place to go.

Copy in Supporting Project Files

In order to read the StickiesDatabase format, you'll either need to reverse engineer the format yourself (you can read about the process here) or you can benefit from someone else's hard work and use open source code released under the GNU GPL that's available here. Simply save these files to disk, navigate to them in Finder, and copy them into your project by dragging them into your "Classes" folder in Xcode. Make sure the box is checked that specifies to copy them into the project. (Actually, only a small portion of the "Document" class is really needed, but the others are left there for convenience in case you decide to be creative and use them for other things.)

Implement the GetMetadataForFile Function

Our last big step is to implement the GetMetadataForFile function; but as you'll see, it's not much of a chore at all. Here's the function's body:


Boolean GetMetadataForFile(
    void* thisInterface, 
    CFMutableDictionaryRef attributes, 
    CFStringRef contentTypeUTI,
    CFStringRef pathToFile)
{
    Boolean success=NO;
    NSAutoreleasePool *pool;

    // Don't assume that there is an autorelease pool 
    //around the calling of this function.
    pool = [[NSAutoreleasePool alloc] init];
    
    /* Pull any available metadata from the file */
    
    //The Document class provides the methods necessary to 
    //manipulate the StickiesDatabase file
    NSMutableArray *array = [[NSMutableArray alloc] initWithArray:
        [NSUnarchiver unarchiveObjectWithFile:
            (NSString*)pathToFile]];

    [array autorelease];
    
    NSEnumerator *enumerator = [array objectEnumerator];
    Document *d;
    NSMutableArray* tempArray = [NSMutableArray array];

    while (d = [enumerator nextObject]) { 
        NSString *t = [d stringValue];
        [tempArray addObject:t];
    }
    
    [(NSMutableDictionary *)attributes setObject:tempArray
        forKey:@"com_apple_stickies_aaa_noteTitle"];

    // return YES so that the attributes are imported and
    // add more resilient error handling at your own discretion
    success=YES;

    //memory management
    [pool release];
    
    return success;
}

I'm sure you'll agree that this method is trivial. This plugin will only process files ending in "aaa," which we're assuming to be in a specific format. The text of each note is simply read and stored into an array. The array is loaded into a dictionary, mapped to the key specified in our schema.xml file, and passed to the Spotlight server, where the rest of the work is handled. It's noteworthy to clarify that our "metadata" is the complete text of each note. Recall that the intention of this plugin is to make the text of each note searchable in Spotlight.

Make Minor Changes to Reflect our Use of Cocoa

At this point, all of the conceptual Spotlight work is done. Now we just need to take care of a few details to ensure our project compiles and links properly:

At this point, your code should build just fine. Making sure you've modified cron's schedule to copy and rename the "StickiesDatabase" file, you can now go into your project's "build" folder and copy your importer into place with a sudo cp -r StickiesImporter.mdimporter /Library/Spotlight from Terminal. If you want it, the complete XCode project file is available here.

xcode 2.0 bug
Side note: In Xcode 2.0, there's a bug that doesn't allow you to change out of a "Development" build style when double clicking on the blue application icon and trying to do so from the "Styles" tab. For changes to build style to stick, you need to make them from the "Project" menu.

With your plugin in place, you should now be able to find text in Stickies notes from Spotlight. (If things don't turn out as expected, there are some troubleshooting references below.) Type some unique practice text such as "ABCDEFG" into a note to check it out, keeping in mind there will be at most a one-minute lag because cron has to do its thing.

our plugin in action
Our plugin at work!

A Few Words on Debugging

Spotlight plugin development can be frustrating because the debugging process isn't always that easy. You have to actually install the plugin and monitor its behavior to determine if it's working, which means extra time copying files around, extra time on the command line, etc. Recall from last time the following tools

Of these tools, stick mdimport inside of "code" tags to be absolutely vital. For example, mdimport -d1 ~/Library/StickiesData.aaa provides feedback that tells you whether or not the proper Spotlight plugin can be located by the server—a fair question to ask. Apple has a very good document on debugging Spotlight plugins available here that you should definitely read.

In Apple’s debugging document, you'll see references to a command called lsregister, which can force the Spotlight server to recognize a bundled Spotlight plugin (like the one just given to you in the "OpenStickies.app"). By default, however, this command is not in your path. Type the following into Terminal to place a soft link to lsregister in your path.


sudo ln -s /System/Library/Frameworks/ApplicationServices.framework/\
Versions/A/Frameworks/LaunchServices.framework/Versions/A/\
Support/lsregister /usr/bin/lsregister

A Freebie for All of your Hard Work

You may have already noticed an inconvenience. You can't click on the "StickiesDatabase.aaa" file that appears in Spotlight's results and have it open up Stickies, as would be the case for most any other application. This is truly annoying, but nothing a good old-fashioned hack can't fix.

By choosing to "Get Info" on a file, you can set its default application that should open it. The problem then is that there is no default application to do such a thing for type "aaa" files—well not yet. To get Stickies to come front and center when we click on the "StickiesDatabase.aaa" file in Spotlight's search window, we simply need to run a simple AppleScript from within an application bundle. The AppleScript is a one-liner:


tell application "Stickies" to activate

This script opens Stickies if it's not already open, and brings all of its windows to the front, giving us the desired effect even though it's not really "opening" the file at all. All that's left now is to create and associate a bundled utility application that executes the AppleScript with the ".aaa" file type. It's that simple.

Embedding an AppleScript into a Cocoa app is really simple, and you can read about it here. I've used this very approach to create a project called "OpenStickies.app." The project's source is available here.

One thing that I didn't mention is that you can embed Spotlight plugins into application bundles. If you'd like a compiled version of "OpenStickies.app" with the "StickiesImporter" plugin already embedded, you can get it here. You'll need copy it to your "Applications" folder and manually run it once to ensure it is "trusted." Once it's in place, go to your ~/Library/StickiesDatabase.aaa file (which cron should be updating every minute), choose to "Get Info," and set "OpenStickies.app" for its default application. Make sure to check the box that says "Always Open With" to fully automate the process. (Try clicking on “StickiesDatabase.aaa” in Finder to test). Assuming you've made your one-line update with cron, you're now good to go.

set default app 1

set default app 2
Set the utility application "OpenStickies.app" to open files of type "aaa" (top) so that Stickies will come front and center when you click on "StickiesData.aaa" in the Spotlight search window (bottom).

Congratulate Yourself

Let's face it, if you've followed this tutorial, you're up to speed on using Spotlight in your Cocoa apps, you can create custom Spotlight plugins (even for the most contrived situations) and you've enjoyed some good hacking along the way. Bravo.

Building Cocoa Applications: A Step by Step Guide

Related Reading

Building Cocoa Applications: A Step by Step Guide
By Simson Garfinkel, Michael Mahoney

Matthew Russell is a computer scientist from middle Tennessee; and serves Digital Reasoning Systems as the Director of Advanced Technology. Hacking and writing are two activities essential to his renaissance man regimen.


Return to the Mac DevCenter

Copyright © 2009 O'Reilly Media, Inc.