The release of Mac OS X Tiger brings with it a new release of QuickTime, the backbone of Mac OS X media applications such as iTunes, iMovie, and more. QuickTime 7 is included in Tiger and is also available as an update to Mac OS X 10.3 (Panther).
This scenario is similar to the way QuickTime 6.4 accompanied the release of Panther in 2003 (and had a companion release for those still on Jaguar), but while that release included some "nice to have" features--VideoCD support, the Pixlet codec, and a QuickTime for Java that worked with Java 1.4--QuickTime 7.0 includes "gotta have" features, in addition to major changes to the user experience, the developers' API's, and the underlying architecture.
Covering all these changes requires a two-part series. In this first article, I'll describe the user-visible features and changes in QuickTime 7, including QuickTime Pro, changes to the QuickTime Player application, and the use of the powerful new H.264 video codec. In the second installment, I'll cover the developer-oriented changes, including new functions in the straight-C API and the all-new, Cocoa-friendly QTKit framework.
First, let the bells ring out, and cheers rise from towns and villages: the much-hated QuickTime Pro "nag dialog" is gone. This dialog, shown in Figure 1, would typically come up every time you played a QuickTime movie and hadn't paid to upgrade to QuickTime Pro. It was an annoyance to many, particularly to sysadmins overseeing large installments who didn't want to (or couldn't afford to) upgrade dozens or hundreds of machines.
Of course, I wouldn't be noting its absence if it weren't for the fact that Pro registration is an issue again in QuickTime 7. The new version requires the purchase of a new Pro key to get access to the Pro features: all the editing- and authoring-related features like copy-and-pasting movie data, exporting to different formats, saving movies from web pages to your hard drive, as well as some non-editing functionality, like the ability to view a movie in full-screen mode. Until you upgrade QT7 to Pro, menu items for these features are disabled and show a little "PRO" icon. Figure 2 shows these menu items in QuickTime Player, and Figure 3 shows them for a movie in a web page.
The QuickTime Player has received a major overhaul, gaining more than just the "live resize" feature often cited in Apple's PR. Besides, how often do you resize a playing movie anyway? There's better stuff to check out.
If you've upgraded to QuickTime Pro, you'll be able to use the File menu's "New Movie Recording" and "New Audio Recording" features. These items allow quick access to your capture devices--microphones, webcams, etc.--bringing up a simple preview window with a red dot button to start the recording, as seen in Figure 4. The feature allows you to dash off a quick video or audio clip, which is then opened as a new movie that you can edit and save.
To save your movie, the Player has a new simplified "Share..." menu item, which replaces the confusing "Import..." of earlier versions (since "Open..." does an implicit import, the separate menu item was little-used). "Share..." is like a dramatically simplified "Export...", reducing the user's choices to "small", "medium", "large" or "actual size". This feature, shown in Figure 5, allows you to save the captured movie--or any QuickTime movie you've opened--with a minimum of fuss, and with an estimate of how large the file will be. As you can see in the figure, it exports to H.264 video and AAC audio, and thus assumes that whomever you send this to also has QT7. Of course, the traditional export dialog is still available, so you can specify formats, codecs, bitrates, and more, down to whatever level of detail suits you.
Another major improvement in the QuickTime Player is the presentation and use of controls. QuickTime 6 offered controls that made use of a rather tortured analogy to the controls you'd see on a television or other consumer electronics product, overlaid on top of the video as seen in Figure 6. This had many disadvantages, not the least of which was the fact that many users didn't even notice the small up-and-down triangles that would allow them to advance to controls other than "Brightness".
In QuickTime 7, these overlay controls are replaced by a single controls window, shown in Figure 7, which offers more familiar slider widgets to adjust the video and audio presentation. It also contains two new "playback" controls, a "jog shuttle" that adjusts the playback rate only so long as the slider is held in a non-default position, and a "playback speed" slider that allows you to change the playback from half-speed all the way to 3x speed.
Similarly, the Movie Properties window has undergone a major overhaul. This Pro-only feature allows you to inspect and change various settings for the movie and all its various tracks. But a problem with the old presentation, as shown in Figure 8, is that the GUI gave no sense of relationship or importance. Little-used properties were as prominent as commonly-used ones, neither visual nor sound properties were grouped together, and some properties were patently mislabeled (for example, the balance control was part of the sound track's "Volume" property GUI).
In QuickTime 7, the Movie Properties window is a multi-paned editor. At the top is a short table showing the various tracks in the movie. Clicking any of these brings up a properties overview in the bottom of the window. In the case of a video track, as seen in Figure 9, the properties have four major groupings: annotations, resources, visual settings, and other settings. The visual settings tab brings together many of the properties that had previously been spread across multiple properties, forcing you to view and edit them in isolation.
There are other little conveniences to be had throughout QuickTime Player, such as the fact that Pro users will now only see the editing selection points when they're actually mousing over the scrubber bar, the idea being that if you're not actively selecting in- and out-points, then you don't need to see these selection pointers. It's a nice idea, but it takes some getting used to. As you work more with QuickTime Player, I'm sure you'll find other points of interest. But on to the main attraction...
The most-touted feature of QuickTime 7 is the inclusion of the H.264 video codec, also known as Advanced Video Codec (AVC) or MPEG-4 Part 10. This is the latest video standard to emerge from the MPEG standards group. It has been widely seen, for years, as the successor to various formats: to MPEG-2 in digital television broadcasting, to the "simple" and "advanced simple" MPEG-4 video codecs on the desktop, etc. Both of the competing standards to replace the DVD--Blu-Ray and HD-DVD--specify H.264 as a video codec that players must support (both also require MPEG-2 for backwards compatibility with existing DVD's, as well as the Microsoft-derived VC-1, H.264's main rival). In television, DirecTV is using H.264 for its HDTV versions of U.S. local stations, while the U.K.'s Video Networks, Ltd., is encoding the "Toonami" cartoon channel with H.264, with other channels to follow.
H.264 is already big news, and it's only going to get better as QuickTime 7 adds millions of players (and a non-trivial amount of Pro-paying encoders) to the mix.
But does H.264 live up to its advanced billing? To find out, I used a DV camcorder to create a short movie in Final Cut Express and encoded it with H.264 and other codecs to see what difference, if any, was visible. I decided to target three use-cases and their associated bitrates:
Standard-definition broadcast--approximately the quality of existing television, this is typically represented as 640x480 with 30 frames per second (fps). Since the H.264 backers have claimed that it would support SDTV at DSL speeds, I've taken them at their word and encoded at 1.5 Mbps.
Internet video--this is a broad categorization, but it's fairly typical to see a 320x240 movie in the range of 300-500 Kbps: an easy download for a broadband user, but a long haul over dial-up.
Mobile--H.264 is a supported codec for 3GPP mobile phones (and is being considered by the 3GPP2), so it's worth looking at how it performs at tiny sizes and bitrates.
As for the test media itself, it's a 45 second video of my son, Keagan, climbing up and then sliding down a slide. This was just recent DV footage that I had handy and wasn't chosen for any particular reason, though it turns out to have a few traits that challenge the various compressors. But suffice to say that I'm no Ben Waggoner. The audio is encoded in AAC (except in the mobile case, where I've stripped it out entirely for space reasons) and is just the natural sound picked up from the camera's mic. The power tools you hear in the background represent a house being built about 500 feet away from the scene. In the first two tests, I've used regular QuickTime files as containers (as opposed to exporting to actual
.mp4 files), while in the mobile case I did use the proper 3GPP container.
The 640x480 movie is meant to simulate the size and demands of standard definition television. The NTSC television standard, used in North America and Japan, has 480 visible lines, and 30fps (technically 60 interlaced fields for an effective 30 fps). I encoded it with H.264, with the regular MPEG-4 visual codec, and with Sorenson Video 3 for old time's sake:
H.264 version - 8.6 MB - requires QuickTime 7
MPEG-4 simple version - 8.1 MB -requires QuickTime 6
Sorenson Video 3 version - 7.9 MB -requires QuickTime 5
Subjectively speaking, the H.264 version is watchable, and the others aren't. The MPEG-4 version starts OK, but as Keagan walks into the dark playhouse, the colors are significantly misrepresented: in some frames, his details average with the wood of the structure and neither is right. Moreover, the colors are corrected every second (with the keyframe) only to get more wrong over the course of the next 29 frames. Figure 10 shows a detail of the same frame of the movie (00:00:19.01), with the H.264 version on the left and the MPEG-4 version on the right. Notice the exceptional error on the right, where MPEG-4 has copied the brown color of the wood post most of the way into Keagan's elbow.
If it sounds like I'm being too hard on MPEG-4, that is fair to a point. Apple's MPEG-4 encoder has always been kept simple, both to make life easier for end users (try explaining "artifacting" and "macroblocking" to your mom sometime), and as a means of leaving open a market for third-party tools aimed at professionals (hello, Sorenson Squeeze for MPEG-4). However, MPEG-4's basic visual codecs can offer more efficient coding than is supported by QuickTime 6.
Apple chose to only support the "Simple" profile of MPEG-4, not the "Advanced Simple" profile, which would have looked better. You can see what Advanced Simple looks like by installing the 3ivx MPEG-4 codec for QuickTime, which encodes and decodes Advanced Simple Profile MPEG-4 video. Beware though: this replaces Apple's MPEG-4 implementation, and it's easy to create files that can only be viewed by other 3ivx users. Maybe it's best to just say that supporting Advanced Simple Profile on the Mac was possible, but Apple apparently chose to move directly from MPEG-4 simple to H.264.
Another advantage enjoyed by the H.264 and 3ivx encoders is that they support multi-pass encodes, meaning that the encoder compresses the data once, then takes another pass through the video looking for further optimization opportunities. Apple's MPEG-4 encoder didn't support multi-pass encode, but its H.264 encoder defaults to multi-pass.
Speaking of encoding, you know that all this super-efficient encoding doesn't come free, right? Encoding the 45-second H.264 movie with multi-pass took between 15 and 20 minutes on a dual 1.8 GHz G5 PowerMac. Working on this article, I got into a workflow of: kick off encode, walk into other room, play a period of ESPN NHL 2K5, come back and check the exported movie, repeat.
One other catch: older Macs may not be able to handle H.264 decoding. I tried the H.264 file on a 450 MHz G4 Cube and it couldn't keep up, dropping down to a frame rate of 15 fps. Supposedly, QuickTime can play H.264 even on G3's, but certainly not at this high of a bitrate.
For distributing video over the internet, it's pretty typical to use a size of 320x240 and a bitrate between 300 and 500 Kbps. This is a size that fits nicely in its own window or on a web page, and doesn't devour an extreme amount of bandwidth. You can double the size of your window to get it up to 640x480 and, depending on the codec, you may or may not be happy with the results.
H.264 performs well, of course, but what's surprising is that if you double the size of the player, the medium bitrate H.264 file looks as good if not better than the MPEG-4 simple file from the last section. Perhaps we're so used to compression artifacting that doubling the size of all the pixels isn't any harder on the eye, and may even be more tolerable.
At this 450 Kbps bitrate, MPEG-4 Simple shows some strong macroblocking that is absent in the H.264 version. Figures 11 and 12 show the same frame (00:00:08.04) from the movies, in MPEG-4 and H.264 respectively. The MPEG-4 Simple version has big, jagged, green macroblocks of different colors on the slide, and they're really distracting when the picture is moving and you get different macroblocks every few frames. The H.264 version is much smoother and more accurately detailed.
If you want to send 48 Kbps video to your friends with mobile phones or dial-up...good luck. This remains at the outer edges of what's reasonably possible with video compression.
At least for this particular piece of video, there isn't a clear winner. As seen in Figure 13, H.264 showed some unattractive smearing early on (it gets corrected by a keyframe after a second), but MPEG-4 Simple drops its framerate when the camera moves from the side of the playhouse to the slide around 00:00:22.0.
In the end, H.264 offers a nice bargain to QuickTime 7 users, as well as all the video professionals who are moving to it: a radically more efficient video codec means either much better picture quality at the same bitrate, or more video over the same pipe by using much lower bitrates.
A few curious little items are worth mentioning. The first is the much-requested support for multi-channel audio, more than just stereo. The news is that it's there... but it's not there there. Specifically, there is now some API-level support for formats like 5.1--the brave can check out the comments for the
SoundDescriptionV2 struct in the Movies.h file (see /System/Library/Frameworks/QuickTime.framework/Headers)--but even formats that support multi-channel audio won't actually emit 5.1 audio at this time, even from hardware like the PowerMac G5 that has optical audio out and would thus seem to be capable of delivering a suitable signal.
The problem is that the optical audio out isn't as plug-and-play as you might think. TOSLINK, as this connector is actually called, has a standard whose only option for uncompressed PCM output is two-channel stereo. It does support compressed multi-channel formats such as Dolby Digital and DTS, and Apple's DVD Player supports these...but only because it's passing through the audio that was encoded when the DVD was authored. To turn multi-channel QuickTime audio into multi-channel output, Apple would have to license Dolby Digital and/or DTS (yuck) and do the encoding/compression in real-time (double yuck).
Another change of note is the reform of the QuickTime Component Download Program. In previous versions of QuickTime, if you opened a movie that required a third-party component that you didn't already have, such as the On2 VP3 video codec, QuickTime would be able to download and install it on the fly. However, the program has languished and hasn't been updated in a while. In QuickTime 7, the new behavior is that when an uninstalled third-party component is encountered in a movie, the user is taken to a new page on Apple's QuickTime site, which leads the user to the vendor's page, from which they can presumably download the needed software.
QuickTime 7 offers some substantial improvements in the QuickTime Player user experience and a kick-ass new video codec in the form of H.264. There are also some major changes for developers of QuickTime applications, which will be the topic of a follow-up article.
Author's Note: Thanks to fellow Kicktammers Nate Caplin and Ben Waggoner for clarifying the issues related to multi-channel digital audio output.
Chris Adamson is an author, editor, and developer specializing in iPhone and Mac.
Return to MacDevCenter.com.
Copyright © 2009 O'Reilly Media, Inc.