Modding Apple Sample CodebyErica Sadun
Apple develops sample code to demonstrate and showcase its developer technology. Often the sample code provides end users with applications that offer unexpected utility. Sometimes you can leverage this technology directly. Sometimes you need to modify the source code to create the results you want.
This article focuses on MovieVideoChart, a typical Apple sample app that's meant to demonstrate how to access video frames in QuickTime 7. But that's just where things begin to get interesting. The scrolling layout and built-in video frame displays make for a perfect starting point, allowing you to re-aim the application toward a different end point.
In this article you'll discover how to modify the MovieVideoChart sample code to create a comic-book-like video layout tool. This tool will be able to scroll through video frame by frame, letting the end user compare successive frames in the same window. And along the way, you'll pick up some important techniques that will help you modify any Apple sample app you encounter.
Tip: You'll need to be at least slightly familiar with both Xcode and C programming to work through this article.
Finding Useful Code
Apple's generous library of sample applications is chock full of gems like HackTV, which you can use to monitor and record Firewire video, and SimpleVideoOutX, which creates a DV output stream from QuickTime videos. These utilities can be used exactly as produced, with terrific results and without any further modification needed on your part.
Apple's sample apps aren't intended to provide libraries of free software, even though in some cases they do. They're meant to showcase a specific Apple technology and provide the source code that demonstrates how to use that technology in your own applications.
Some sample applications simply provide a jumping-off point; many of these apps lend themselves to adaptation. You can modify the source code to build your desired utility by grabbing the preprogrammed chunks at a macro level and making relatively minor tweaks to create the results you want.
MovieVideoChart is one such application. By itself, the sample app is fascinating. It is not, however, useful in any general way. This isn't to say that the information provided isn't helpful or important, but that the utility isn't general-purpose. Yet with a little tweaking the app easily lends itself to modification. Here's a quick overview of the intended purpose and function of MovieVideoChart. Feel free to skip ahead to the next section if you start to nod off.
More than You Wanted to Know About MovieVideoChart
As designed, MovieVideoChart demonstrates how to access MPEG frames in QuickTime 7. Because MPEG compression uses three frame types, QuickTime's underlying technology uses slightly different techniques to access each kind of frame. As the name indicates, key frames (or I-frames) store complete frame shots. Predictive frames (or P-frames) store the difference between one frame and the next. Bidirectional frames (or B-frames) look for changes in both directions--forward and backward--providing the highest level of compression. B-frames introduce the most difficulty with respect to recovering full-data frames because of the interframe dependencies.
Since B-frames rely on data both before and after them, they're tricky to handle when you're trying to retrieve an entire image from the movie data. New QuickTime 7 support allows you to pull B-frames at will, accessing both previous and future frames to create the entire image. This is very convenient and much easier to handle than in the past.
B-frames are the frames that can be dropped most readily during playback on slower connections and machines. As Figure 1 shows, in the MovieVideoChart display, blue-dotted B-frames are marked "droppable" for this very reason. The red-dotted "Sync" frames are I-frames, and the green dots indicate P-frames.
In addition to identifying frames, MovieVideoChart gives you insight into the frame order of your movies. It demonstrates how the new APIs take care of all interframe dependencies, so as a programmer, you can grab data on a frame-by-frame basis. Notice how the actual media frame order in the top row is 1, 3, 2, 5, 4, 7, 6 and so forth? Via QuickTime, this decodes to the expected display order of 1, 2, 3, 4, 5, 6, 7 shown in the bottom row.
Tip: If you want to create a movie that takes advantage of B-frame compression, export from QuickTime using a highly compressed format like MPEG-4 using Broadband Low settings.
Figure 1: MovieVideoChart in its original form as distributed by Apple