What's New For Developers in QuickTime

by Chris Adamson
06/28/2005

QuickTime 7 has made a splash since premiering as part of Mac OS X 10.4 (Tiger) and as a separate download for users still on 10.3 (Panther). In Part 1 of this series, I covered the features of QT7 that end users will see, specifically:

• Expiration of QuickTime Pro 6 keys (maybe not a "feature", per se...)
• One-touch audio and video recording menu items in QuickTime Player
• New properties and controls windows in QuickTime Player
• The H.264 video codec
• Support for authoring multichannel audio
• A new, web-based QuickTime Component Download Program

In this second article, I'll cover some of the most prominent changes from a developer point of view. These include:

• The QTKit API for Cocoa
• The new QuickTime MetaData API
• Support for frame-reordering codecs
• Core Audio-based sound enhancements
• Miscellany: JavaScript and QuickTime for Java

Programming with QTKit

If, like me, you've picked up Cocoa and Objective-C after spending years in other object-oriented languages like Java, you probably don't mind the differences in syntax—you say square brace, I say dot operator—because the ideas are fundamentally similar. And to be sure, developing Cocoa applications is quite pleasant when you get to use tools like Interface Builder, more pleasant than writing hundreds of lines of GridBagLayout code, like us J2SE guys are used to.

Unfortunately, Cocoa access to QuickTime functionality has been limited up to now. Two Application Kit classes, NSMovie and NSMovieView, provided a bare-bones ability to open movies from the clipboard or a URL and to send them a few basic messages: start and stop; cut, copy, and paste; setEditable and isEditable, etc. Deeper needs than this required taking a pointer to the QuickTime Movie and heading off to the straight-C QuickTime API.

QuickTime 7's SDK—installed by Tiger or via Software Update if you have XCode Tools installed on Panther—includes a new framework called QTKit, which significantly increases the number of things you can do in Cocoa and Objective-C, without needing to resort to more difficult C calls to QuickTime itself.

QTKit is still fairly small, consisting of just five classes:

• QTMovie: a collection of media resources and their relative arrangement.
• QTMovieView: a visual component for displaying movie contents and (optionally) a controller widget.
• QTTrack: an organization of one kind of media, describing its timing, spatial, layering, volume, etc.
• QTMedia: a reference to actual samples of some kind of media—frames of video, clips of text, samples of audio, etc. Each track has exactly one media.
• QTDataReference: a specification of the location of some media data, potentially in memory, on disk, or on the network.

You might be tempted to think that QTMovie and QTMovieView would be simple replacements for NSMovie and NSMovieView, but the docs reveal a surprise. While NSMovie offered just 11 methods, QTMovie offers 77, including some shockingly powerful calls. For example, addImage:forDuration:withAttributes: effectively allows you to build video media, frame by frame, without even looking at the compression sequence API.

QTKit has its limits, of course. There's no object to represent the SequenceGrabber for media capture, nor a wrapper to the Presentation for broadcasting. Furthermore, the objects that do exist don't wrap every possible QuickTime call with which each object might be used. Think of it as an 80-20 proposition: what's provided so far is a small subset of the QuickTime API, the most popular calls that will allow many developers to meet their needs without delving into the straight-C API.

A programming example of QTKit appears later in this article, after covering another new feature.

Meeting Metadata

Related Reading QuickTime for Java: A Developer's Notebook By Chris Adamson

The ability to "tag" media with descriptive data is becoming essential for supporting search, as well as providing a better end-user experience. However, schemes for tagging media are all over the map: the ID3 standard for MP3's, EXIF for JPEG's, and an undocumented and proprietary format for iTunes-encoded AACs (that I investigated and provided a parser for in the ONJava article QTJ Audio and in the book QuickTime for Java: A Developer's Notebook). To top it off, QuickTime movies allow for a user data container, into which you can place pretty much any data you like.

Implementing support for all of these formats in your application would be a headache.

Realizing this, QuickTime 7 provides a new metadata API that abstracts away the differences between the many schemes, exposing all of them through a common interface. In its first implementation, QuickTime supports three "containers":

• Classic QuickTime user data: along with supporting genuine QuickTime user data, this also provides access to MP3s' ID3 tags and other forms of metadata that are parsed into user data items when files and streams are imported into QuickTime.
• iTunes Metadata: album, artist, song, user rating, cover art, etc., from files encoded by the user or purchased from the iTunes Music Store. Note that according to Movies.h, "The list of keys for iTunes metadata is TBA."
• New QuickTime metadata storage format: the docs advertise this as simply "a richer QuickTime metadata container format." Among its most interesting traits is the abandonment of the four-character-code scheme of user data keys in favor of a reverse-DNS format, similar to package names in Java (e.g., com.apple.something.somethingelse). This has the advantage of avoiding name collisions, not requiring Apple to act as a universal registrar of key names, and being infinitely extensible.

The idea is one of simple key-value coding: you supply a known key (like "description" or "album name"), and you get back 0 or more values. In practice, using the Metadata API has significant similarities to getting Components from QuickTime: you search for them by providing criteria to match against, you call the search method with the last match (null on the first call) to find the next match, and there can be several matches even when asking for fairly specific criteria. To clarify the last point, you can ask for a given key in a given container format and get multiple hits. After all, there might be multiple "author" or "artist" tags.

Demo: QTKit and Metadata in Practice

By way of a demo, here's a short Cocoa application to show:

• How to use QTKit's QTMovieView in a nib
• How to get a QTMovie and put it into the QTMovieView
• How to move from QTKit to QuickTime, by getting the underlying Movie pointer from a QTMovie object
• How to get metadata from the movie with the new metadata API

I'll walk through all the Xcode steps, but this is going to be lightning fast; for a proper introduction to Cocoa development, check out James Duncan Davidson's Learning Cocoa with Objective-C, 2nd Edition. The complete Xcode project is available in the resources at the end of the article.

Building the QTKit GUI

First, kick off a new Cocoa-application project in Xcode. Double click the MainMenu.nib to launch Interface Builder (IB). The first thing you need to do in IB is to add the QTKit palette, so that you'll be able to add a QTMovieView to your GUI. So, do a "Show Palettes" (if the Palettes window isn't already up) and go to "Palette Preferences..." Click the "Add..." button and navigate to /Developer/Extras/Palettes/QTKit.palette to add the QTKit palette. You should see the QTKit palette in your Palettes window, as seen in Figure 1.

Figure 1. QTKit Palette in Interface Builder

The big "Q" is a QTMovieView. Drag it to the main window to add a QTMovieView to your GUI. It will originally appear as a very small component; I used the Inspector to set its size to 320 x 256 (the common 320 x 240 movie size, plus the usual 16 pixels for the scrubber bar) and resized the window to align it to the bounds suggested by IB's friendly blue lines. The resulting window should look like Figure 2.

Figure 2. QTMovieView in an IB window

Finish off the window by adding an NSTextView at the bottom and a label above it that says "Metadata".

Now you need to start wiring things up, by creating a controller object to handle UI events. In the MainMenu.nib window, click the Classes tab, and then click NSObject. Go to the Classes menu and do "Subclass NSObject". Rename the new class MyController.

Now tell the controller about its actions and outlets: with MyController selected, bring up the inspector. Add two outlets, naming them metadataView and movieView. Also add an action, which you can call handleOpenMovie.

You need an instance of the controller, which you'll wire up to the GUI components. With MyController selected, pull down "Instantiate MyController" from the Classes menu. This will create a "MyController" object, represented as a blue cube in the Instances tab of the MainMenu.nib window. To wire up the outlets, control drag from the object to the QTMovieView, as illustrated in Figure 3. This will bring up the inspector, and allow you to select which of MyController's outlets you're connecting. Choose movieView. Now repeat the process to wire up the metadataView to the NSTextView.

Figure 3. Wiring up a QTMovieView as an outlet

In the MainMenu.nib window, open the MainMenu object and trash all the menus except "File", and delete all of its menu items except for open. Control drag from the open menu item to the MyController object to wire up the menu item to the handleOpenMovie action.

Everything is now wired up. In the MainMenu.nib window, switch to the "Classes" tab and select "MyController". Go up to the "Classes" menu and select "Create classes for MyController". The default location should be good, and you should see that "insert into targets" says "QT7MiniDemo" (or whatever you named your project). Save up and quit IB.

QTKit Coding

Back in Project Builder, your next job is to add in the QTKit and QuickTime frameworks, so you can compile code against them. Control click on "Linked Frameworks" and choose "Add -> Existing Frameworks...". Navigate to /System/Library/Frameworks and choose QTKit.framework. Then do the same thing to add QuickTime.framework. You'll also need to import these frameworks in your header file, so open MyController.h and add the lines:

    #import <QuickTime/Movies.h>
    #import <QTKit/QTKit.h>

Next, you should strongly type the outlets listed in Movies.h so the compiler can better check your code. Your outlets should look like this:

    IBOutlet NSTextView *metadataTextView;
    IBOutlet QTMovieView *movieView;

All right, now you're ready to code! In the handleOpenMovie method, your job is to present a file dialog, open the selected file as a QTMovie, and put the movie in the QTMovieView. The first part is pretty easy... OK, it's all pretty easy, but this part you might have seen before:

NSArray* fileTypes = [NSArray arrayWithObjects:@"mov", @"mp4",
                        @"m4a", @"mp3", @"m4p",
                        @"jpg", @"jpeg",nil];
NSString* moviesDir = [NSHomeDirectory() stringByAppendingString:
                        [NSString stringWithString: @"/Movies"]];
// ...
    NSOpenPanel* panel = [NSOpenPanel openPanel];
    [panel runModalForDirectory:moviesDir file:nil types:fileTypes ];
    NSURL* url = [[panel URLs] objectAtIndex: 0];

This just shows an NSOpenPanel for several known QuickTime-friendly file types. When the panel is dismissed, you get the selection as a URL.

Next job: converting this file into a QuickTime movie. Hold on, this is going to be tough...

    NSError* openError = nil;
    qtMovie = [QTMovie movieWithURL: url error:&openError];

...or not. Getting a QTMovie from a URL is a one-line call. OK, a few more lines if you want to inspect the error if it isn't nil; this is in the downloadable code, but I've omitted it here. Assume for now that it worked and that you have a valid QTMovie object. Now you need to put it in the view:

    [movieView setControllerVisible: YES];
    [movieView setMovie: qtMovie];

And... we're done. Seriously. This is all you need to do to open and show a QuickTime movie with QTKit. The control bar is live, so you don't even need to provide menu items or buttons to start and stop movie playback, though it's not like a [qtMovie start]; would exactly kill you, now would it?

This is an extremely simple example of QTKit, practically the "Hello World" of using the framework. For a far more ambitious example, check out Apple's tutorial QuickTime Kit Programming Guide.

One little note here: I've noticed that when I do a development build, the QTMovieView doesn't become active until I switch to another application, and then back to QT7MiniDemo. This doesn't happen when I build the application via a deployment build. Just so you don't think you made a mistake...

Pages: 1, 2

Next Page