What's New For Developers in QuickTime
Pages: 1, 2

Moving on to Metadata

As if to remind you how pleasant QTKit is, this next section on metadata requires using the straight-C QuickTime API. First, get a pointer to the QuickTime movie:

    Movie movie;
    movie = [qtMovie quickTimeMovie];

The first thing to do with metadata is to get a reference to a QTMetaDataRef. You can get this from a movie, any of its tracks, or any of their media; obviously, this parallels the use of QuickTime user data, which exists at each of those levels. For the purposes of this demo, you're only interested in Movie-level metadata:

    QTMetaDataRef movieMetaData = malloc (sizeof (QTMetaDataRef)); 
    QTCopyMovieMetaData (movie, &movieMetaData); 

This object will give you metadata one of two ways: you can either ask for specific metadata items by known keys, or you can use wildcards to tour the available metadata. You need to specify which kind of container you want to use (user data, iTunes, new QT metadata), but there's a wildcard for that too. You retrieve a QTMetaDataItem by way of the QTMetaDataGetNextItem function. As you might have guessed from the use of "Next" in the method name and other QuickTime discovery conventions (like FindNextComponent), you iterate through the metadata items by passing in the last one you found, using kQTMetaDataItemUninitialized for your first trip through the loop. This makes for a loop that looks like the following:

    QTMetaDataItem item = kQTMetaDataItemUninitialized;
    while (noErr == QTMetaDataGetNextItem (movieMetaData,
                                kQTMetaDataStorageFormatWildcard,
                                item,
                                kQTMetaDataKeyFormatCommon,
                                nil,
                                0,
                                &item)) {

        // do stuff with item

    }

This call is documented in the QT API reference, but to quickly summarize the parameters:

1. The QTMetaDataRef you obtained earlier
2. A constant for the container (user data, iTunes, new QT metadata) that you want to search in
3. The last QTMetaDataItem you found
4. A constant for the format of the key you're querying with
5. A pointer to the key you want to search by. Pass nil for a wildcard
6. The size of the key
7. A pointer to receive the address of the next matching metadata item.

Once you have a QTMetaDataItem, there are various things you can do with it. By using the QTMetaDataGetItemProperty method, you can get the key's type, size, and value. For this example, it's nice to show the keys that go with each value, so you'll know what to search for in the future. Here's how to get a key from an item discovered via wildcards:

    UInt8 buffy[100];
    // ...
    QTPropertyValuePtr key = &buffy;
    ByteCount keySize = 0;
    QTMetaDataGetItemProperty (movieMetaData,
                               item,
                               kPropertyClass_MetaDataItem,
                               kQTMetaDataItemPropertyID_Key, 
                               sizeof (buffy),
                               key,
                               &keySize);
    NSString* keyString =
        [NSString stringWithCString: key length: keySize];

As you can see QTMetaDataGetItemProperty takes the QTMetaDataRef and the QTMetaDataItem, along with a "class" and "ID" indicating what you want to get from the item. The last two parameters receive a pointer to the data and its size, from which you can easily make an NSString for later use with the GUI.

So you have a key. Next, you want a value. This can be potentially huge (think cover art in iTunes), so you precede the "get value" call with a "get size of value" call, in order to allocate a sufficiently large buffer:

    // get value size
    ByteCount valueSize = 0;
    QTMetaDataGetItemValue (movieMetaData,
                            item,
                            NULL,
                            0, 
                            &valueSize);
    printf ("  got value, size=%d\n", valueSize);
    // get value
    char valueBuf[valueSize];
    QTMetaDataGetItemValue (movieMetaData,
                            item,
                            &valueBuf,
                            valueSize, 
                            NULL);

As you can see, these are the same function, except that the first time, you pass in NULL for the second parameter (outValuePtr) to have the size returned in the last parameter. On the second call, you pass a pointer to a sufficiently large buffer in the second argument and ignore the last argument.

One problem left: what did you get back? If you query by a known key, you might know how the returned buffer is organized, but for a wildcard tour like this, you don't know what you got back. Fortunately, there's a call for this too:

    QTPropertyValuePtr* valueType = buffy;
    ByteCount valueTypeSize = 0;
    QTMetaDataGetItemProperty (movieMetaData,
                               item,
                               kPropertyClass_MetaDataItem,
                               kQTMetaDataItemPropertyID_DataType, 
                               sizeof (buffy),
                               valueType,
                               &valueTypeSize);

The returned QTPropertyValuePtr can be one of the following types defined in Movies.h:

  kQTMetaDataTypeBinary         = 0,
  kQTMetaDataTypeUTF8           = 1,
  kQTMetaDataTypeUTF16BE        = 2,
  kQTMetaDataTypeMacEncodedText = 3,
  kQTMetaDataTypeSignedIntegerBE = 21, 
  kQTMetaDataTypeUnsignedIntegerBE = 22,
  kQTMetaDataTypeFloat32BE      = 23,
  kQTMetaDataTypeFloat64BE      = 24

In the example, these values are used to determine whether the value can be converted into a string (ultimately into an NSAttributedString) and displayed in the NSTextView. If not, an array of type names allows the view to show a description of the discovered value.

That's pretty much it. If you have QuickTime Pro, you can test it by using the Properties viewer to add some annotations to a Movie. Figure 4 shows an example of a movie I've annotated this way.

Figure 4. QT7MiniDemo showing Movie-level annotations

If you don't have QuickTime Pro, don't panic. Lots of QuickTime files have interesting metadata that can be read with this application. For example, the metadata for an iTunes Music Store song is shown in Figure 5.

Figure 5. QT7MiniDemo showing iTunes Music Store metadata

One last talking point before moving on... do you suppose that Apple added this feature to QuickTime just to be nice? I don't. Most recent QuickTime features have been added to service strategic goals, like how AAC and Apple Lossless bolstered the iPod. Let me show you what I think is up here. If I use Spotlight to query for "Elk Rapids", a fairly unique term in my annotation, Figure 6 shows what I get:

Figure 6. Finding QuickTime metadata with spotlight

Notice that "Elk Rapids" isn't in this file name, only in its metadata. On the other hand, don't just assume that Spotlight searches the entire file. That would be incredibly pointless on a multi-gigabyte video file. Moreover, it's demonstrably not true: every QuickTime movie contains an atom called moov, and every tagged MP3 starts with the string ID3, yet searching for those strings doesn't pull up all your movies or all your MP3s. I think this suggests that Spotlight uses the QuickTime metadata API when indexing your movies, and I wonder if that's the reason the metadata API was developed to begin with. (Join me in the talkbacks below for further idle speculation.)

More New Stuff

As I'm already over the MacDevCenter article word limit, let me briefly discuss a few other points that will be of interest to specific classes of QuickTime developers.

Support for Frame-reordering Codecs

To support H.264, and other modern codecs, the image decompression API has seen significant changes to support "out of order" or "frame-reordering" codecs. These kinds of codecs allow for three kinds of video frames:

• Intra-frame (I-frame): a self-contained frame, i.e., one that does not depend on any others for decoding
• Predicted frame (P-frame): a frame that depends on one or more previous frames (an I-frame and zero or more other P-frames)
• Bi-directional frame (B-frame): a frame that depends on one or more previous frames and a future frame

Being able to depend on a future frame increases compression efficiency, but increases cost and complexity. On this latter point, imagine the set of frames illustrated in Figure 7:

Figure 7. Conceptual layout of I-, P-, and B-frames

In this example, the letters equal the frame type and the numbers indicate their order. So, frames 1 and 5 are independent, 2 depends on 1, 3 depends on 2 (and thus on 1), and 4 depends on 3 and on 5. That means to display frame 4, frame 5 has to be decoded first, even though it will be rendered after 4. This can result in an encoding that actually puts frame 5's data before 4's, as seen in Figure 8.

Figure 8. Example of a reordered B-frame

What this introduces in the API is a distinction between decode time and display time since some frames will be decoded but not used right away (as is the case with frame I5 above). Notes on CompressSequenceBegin and DecompressSequenceBegin in ImageCompression.h now advocate use of the alternatives ICMCompressionSessionCreate and ICMDecompressionSessionCreate. These require a change in coding style too, because instead of providing a frame in a return value or parameter, they use a callback function, and can return multiple frames.

Most application-level developers don't need to worry about these issues, but applications that compress or decompress media at the sample level, or that count samples, need to understand the concept and look for new functions that are frame reordering aware.

Audio Enhancements

QuickTime 7 makes a number of changes to improve audio support. A new SoundDescriptionV2 opens the way to multichannel audio (exploiting Core Audio on Mac OS X to support it... thus indicating it's not meaningfully supported on QuickTime for Windows), but you aren't responsible for actually dealing with this structure. Instead, a higher-level sound description API allows you to create the description with an AudioStreamBasicDescription, a channel layout, and (optionally), a "magic cookie." And yes, I said, "channel layout." QuickTime now goes beyond left-right stereo to support arbitrarily placed sound sources, such as 5.1 surround-sound arrangements.

Other small but novel improvements include the ability to change playback rate without pitch-shifting audio, and level and frequency metering API's. On this latter point, previous versions of QuickTime offered some level-metering functions at the media level, but they were undocumented.

Safari Support for JavaScript Access to the QuickTime Plug-in

Many popular browsers have been able to use JavaScript to control an embedded QuickTime movie for some time, but this feature now works on Safari in Tiger. This means that you can do some simple tricks, like providing script-based start and stop buttons as seen in this example from an ONJava QuickTime for Java article.

New Fixes, New Bugs in QuickTime for Java

Speaking of QuickTime for Java, it got a little love in QuickTime 7, with a number of outstanding bugs fixed for it. Unfortunately, QuickTime 7 does not address many long-standing requests from QTJ developers, like the ability to get an AWT or Swing preview component for the SequenceGrabber. Also, QT7 seems to have introduced some new QTJ bugs of its own: displaying the settings dialog for a streaming Presentation, something I discussed in the article Streaming QuickTime with Java, is now a 100% crasher. Sigh. QTJ giveth, and QTJ taketh away.

Big Finish

This article only broadly introduces the major changes in QuickTime 7; when I diff'ed the header files between it and QuickTime 6, I had over 9,000 new lines just of constants, functions, and comments. A more in-depth tour of QT7's major new features is available in Apple's QuickTime 7 Update Guide. Still, I hope it will get you thinking about the new things you can do with QuickTime in Tiger.

Resources

• Sample code for the QTKit/metadata application
• QuickTime 7 Update Guide
• QTKit Programming Guide

Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.

---

Return to the Mac DevCenter