Editor's note -- Google recently added a "download to video iPod" as this article was in production. I wanted to post this piece anyway because I think Google Video has lots to offer iPod users, and I still find this article interesting. Just know that there's now an easier way to grab the video.
Roughly speaking, most things forwarded to my email inbox are garbage. The latest political humor. The best pictures from Hubble. Another useless internet petition. Yawn, delete, delete. Sometimes, though, someone forwards me a link that really tickles my fancy.
A friend recently pointed me to a Google Video link. In it, two Chinese students lip-synched to the Backstreet Boys. It was cute. It was cool. I wanted it on my iPod.
So here's what I did. In this article, you'll discover how I downloaded the video, converted it to an iPod-friendly format, and loaded it onto my new 5G video iPod.
Figure 1: Google Video iPod fun.
Please take a second to note that there's a big difference between copying a video to your iPod for personal viewing, and broadcasting it as a torrent to others. Use your Google Video powers for good not evil. Google Terms of Service states "[Y]ou agree that you will not copy, reproduce, alter, modify, create derivative works, or publicly display any content (except for your own personal, non-commercial use) from the Site".
To start, navigate to the Google Video web page that hosts the video you want to download. Copy the URL of the page and paste it somewhere safe, where you can access it again. For the Chinese Backstreet Boys video, this was:
http://video.google.com/videoplay?docid=-6739710473912337648Behind this URL lurks the web page source code. The source code knows where to find the actual video file, and with a little hunting around, you can find it too. Begin by opening the page source:
The source code opens in a new window. Here, you need to search for videoURL=. Just after this phrase is an important sequence, the slightly encoded version of the video URL. It ends with autoPlay=true.
For the Chinese BSB video, it looks like this:
http%3A%2F%2Fvp.video.google.com%2Fvideoplayback%3Fid%
3D2efa74be06d7001%26begin%3D0%26len%3D215666%26itag%3D5%2
6urlcreated%3D1135291639%26docid%3D-6739710473912337648%26
urlcreated%3D1135291639%26sigh%3DgP3sGiMoQJ5gjCqZtq1ds8rwOg8
&autoPlay=true(Please note that I've added line breaks to make this a little more readable.)
Figure 2: Make a note of the video's URL.
When I say "slightly encoded," I mean that it uses escape sequences for many of the normal URL characters. Instead of http://, for example, you get http%3A%2F%2F. This encoding prevents you from using simple tools like the Unix curl command-line download utility.
For those strong of heart, you can unescape this URL in Firefox or Explorer. Enter javascript:unescape("URL") into the address bar and press Return. This produces a clean URL you can use for downloading.
Of course, it's always better to automate and take the guesswork out of the equation. So I put together a couple of utilities to get the job done. These utilities consist of a shell script that automates the URL capture and download process, and an unescape program that converts an escaped URL into a normal unescaped version. And yes, these are for the Mac.
getGoogle Shell ScriptMy getGoogle shell script grabs the page source directly from a main Google Video URL, like the one you just copied down. It uses sed, the Unix stream editor, to cut off the text before videoURL= and after autoPlay=true to recover the escaped URL.
Then it uses my unescape utility (described below) to convert to an unescaped URL. It calls curl to fetch the data, which is saved to your desktop as GoogleOut.flv. (Google uses the FLV format for its videos.)
#!/bin/csh # Grab the escape-coded video URL from Google Video set escBAR = `curl -s "$1" | sed "s/autoPlay=true.*/autoPlay=true/" | sed "s/.*v ideoUrl=//" | grep google.com` # Unescape the URL set unBAR = `./unescape "$escBAR"` # Perform the download curl "$unBAR" > ~/Desktop/GoogleOut.flv echo "File has been output to ~/Desktop/GoogleOut.flv" exit 0
|
unescape UtilityUnfortunately, there's no "unescape URL" utility built into the Macintosh (or at least not one that I could find). There is, however, an unescape routine distributed as part of the standard developer library. The following utility takes one argument, an escaped URL (in quotes, please!) and returns it unescaped.
// cc unescape.c -o unescape -lcurl
#include <curl/curl.h>
#include <stdio.h>
main(argc, argv)
int argc;
char **argv;
{
if (argc < 2)
{
fprintf(stderr, "Usage: %s URL\n", argv[0]);
exit(2);
}
printf("%s\n", curl_unescape(argv[1], strlen(argv[1])));
return(0);
}
If you don't have the developer package installed, you'll find a pre-compiled version of this utility here.
You'll need to create both the getGoogle shell script and unescape.c source as plain text files in a new folder on your Desktop. Launch Terminal and change directories to that folder.
$ cd ~/Desktop/Escapestuff
$ ls
getGoogle unescape.c
$
Compile the unescape utility. (Feel free to ignore the warnings.)
$ cc unescape.c -o unescape -lcurl
unescape.c: In function 'main':
unescape.c:14: warning: incompatible implicit declaration of built-in function 'exit'
unescape.c:17: warning: incompatible implicit declaration of built-in function 'strlen'
$
Make the getGoogle shell script executable.
$ chmod 755 getGoogle
$
Now you're ready to perform the download. The getGoogle utility takes one argument, the main Google Video URL in double quotes. curl provides live updates as the download progresses.
$ ./getGoogle "http://video.google.com/videoplay?docid=-6739710473912337648"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 12.3M 0 12.3M 0 0 156k 0 --:--:-- 0:01:20 --:--:-- 160k
File has been output to ~/Desktop/GoogleOut.flv
$
After the download ends, you'll have a brand new file on your desktop: GoogleOut.flv.
Figure 3: Drop the FLV file onto the From field to start setting up your encoding.
It should come as no surprise that the FLV video you download will not play back on your iPod. Or in QuickTime. Or pretty much anything else. Even VLC won't play it--they dropped support as of 0.8.2. (Sure, if you've got $30 to blow, you could download the Wimpy AV player, which offers FLV support for both Windows and OS X, but let's get back to the whole iPod thing, right?)
Here's where the fabulous ffmpegX comes in. Or at least it does for the Mac. Windows users? You'll have use it the ffmpeg command-line utility or search for a Windows-based ffmpeg GUI like PSPEnc. With ffmpegX, converting your videos couldn't be simpler.
After installing ffmpegX (make sure to install all those extra components, namely mplayer, mencoder and mpeg2enc), just follow these steps:
An ffmpegX progress window opens, letting you keep track of your encoding job. When finished, it chimes. The new MP4 file appears on your desktop (unless you chose to save it somewhere else) and is ready to play in QuickTime and iTunes.
All that's left now is to add your movie to your iPod. Drag the new MP4 file into your iPod playlist in iTunes. (You can also drop it directly onto the iPod icon there if you prefer.) iTunes copies the video and places it into your iPod's Videos-->Movies collection. Simply select it, sit back, watch, and enjoy.
Editor's note: Just in case you were wondering, it's OK to download this content for your own personal use. Here's the word from Google: "Accordingly, you agree that you will not copy, reproduce, alter, modify, create derivative works, or publicly display any content (except for your own personal, non-commercial use) from the Site."
Erica Sadun has written, co-written, and contributed to almost two dozen books about technology, particularly in the areas of programming, digital video, and digital photography.
Return to the Mac DevCenter
Copyright © 2009 O'Reilly Media, Inc.