O'Reilly Book Excerpts: BSD Hacks
Hacking BSD, Part 2by Dru Lavigne
Editor's note: Last week's excerpts from O'Reilly's BSD Hacks offered a few administrator tricks -- finding files easily and managing disk space without reinstalling. This week's excerpt provides administrators and users alike with a hack on how to get the most out of FTP using macros and scripts.
Hack #51. Get the Most Out of FTP
Get the most out of stock ftp with macros and scripts.
In this age of GUIs and feature-rich browsers, it's easy to forget how quick
and efficient command-line
ftp can be. That is, until you're logged
into a system that doesn't have X installed, nor a browser, nor any fancy FTP
programs. If it's really your lucky day, it won't even have any manpages. And,
of course, you'll need to download something.
Perhaps you find yourself using
ftp all the time, always going
to the same FTP servers and downloading from or uploading to the same directories.
Clearly, it's time for some FTP automation.
Have you ever noticed how easy it is to use FTP from a modern browser? Simply click on a hyperlink to start a download. At the command line, though, you can't even browse the FTP directory structure until you successfully log into the FTP server. Well, guess what: you always have to log into an FTP server. It's just that your web browser hides this little detail by doing it for you in the background.
You can achieve the same transparency for command-line
creating a file called .netrc in your home directory and placing the
following line in that file:
% more ~/.netrc default login anonymous password firstname.lastname@example.org
This line will work for any FTP server on the Internet that accepts anonymous logins. (Most do, unless it's a private server.) When creating your own file, use your own email address as the password.
Test your change with this command:
% ftp ftp.freebsd.org
Compare your results to the FTP output in [Hack #71] . You should receive the same banner shown there without having to first type in a username and password.
If you're a webmaster who uses FTP to upload your new files, you do have to log in first. After all, you don't want just anyone uploading files, so you require a username and password. To automate that process, add a section to your ~.netrc that reflects the name of your server and your username and password:
machine ftp.myserver.com login myusername password mypassword
Since you've just inserted your password into a plain text file, it's important to change the permissions on this file so that only you can read it:
chmod 600 ~/.netrc
If you forget to change the permissions and try to access an FTP server that requires a username and password, your login attempt should fail and result in this error message:
ftp: Error: .netrc file is readable by others. ftp: Remove password or make file unreadable by others.
To be extra safe, exclude the
password line completely. When you
connect to the FTP server, your username will be provided for you, but you will
still be prompted for the password.
Now, let's say that you visit ftp.freebsd.org on a regular basis and always access its pub/FreeBSD/releases/i386 directory. Rather than
cding every time, you can automate that process by creating an FTP macro. Add these lines to ~/.netrc:
macdef fbsd bin cd /pub/FreeBSD/releases/i386
Macros are defined by
macdef, and the name of the macro follows.
Keep the name short but useful, as a macro is supposed to be a timesaver. Once
you've declared the macro, add the FTP commands you want to execute, one line
at a time. This particular macro contains the
command. That command is useful when downloading because it ensures all files,
including non-ASCII files such as applications, will download correctly. I also
cd command to automatically take me to my usual working
TIP: It's important that a macro always ends with a blank line.
There are two ways to use your macro. If you're already connected to the FTP
$macroname at the
$ fbsdbin 200 Type set to I. cd /pub/FreeBSD/releases/i386 250 "/pub/FreeBSD/releases/i386" is new cwd.
Note that each command in the macro will be executed, followed by its results.
The second way to run the macro is when you first invoke the
echo "$ fbsd" | ftp ftp.freebsd.org
Now, if you try that one, you'll notice that all of your commands will succeed.
Then, your FTP session will abruptly end, and you'll receive your regular prompt
back! Rather disappointing if you were planning on typing some more commands
ftp prompt, but absolutely perfect if your intention is
to script an entire FTP session.
Scripting an Entire Session
If you already know what you want to do, and especially if you need to do it
more than once, why type in everything at the
ftp prompt? Suppose
you want to download the latest XFree86 distribution directly from ftp://ftp.xfree86.org/.
Consider placing this macro in ~/.netrc:
macdef X bin bell prompt cd /pub/XFree86/4.3.0/source mget * bye
This macro assumes that this ~/.netrc file already contains the line that allows anonymous logins.
bellcommand, which is optional, should produce a sound after
each successful file transfer. The
promptcommand is very important,
though. By default, the FTP server expects interaction from the user. That is,
when you ask to download multiple files with
mget, the FTP server
will wait for you to confirm every transfer by typing
we want to disable that behavior when we're scripting a download.
To run this macro:
% echo "$ X" | ftp ftp.xfree86.org
ftp will save the downloaded files in your current
working directory. If you prefer, you can specify an alternate location in your
macro with the
lcd (local change directory) command. For example:
will save the downloaded files to the /usr/local/Xsource directory. Make sure your directory exists and put the
lcd line before your
A Better FTP?
No matter how hard you try to make the default FTP client user-friendly, it
is still a very basic command, and you may find a little too primitive, especially
if you use
ftp often. If you would like to try a more convenient
and user-friendly command-line tool, try
ncftp, which is available
as a port or package for FreeBSD, NetBSD, and OpenBSD.
ncftpweb site (http://www.ncftp.com/ncftp/)
Dru Lavigne is a network and systems administrator, IT instructor, author and international speaker. She has over a decade of experience administering and teaching Netware, Microsoft, Cisco, Checkpoint, SCO, Solaris, Linux, and BSD systems. A prolific author, she pens the popular FreeBSD Basics column for O'Reilly and is author of BSD Hacks and The Best of FreeBSD Basics.
Return to the BSD DevCenter.