IMAP or POP? Which is the best mail protocol to use on your Macintosh? This IMAP primer will help you understand its benefits and limitations, especially when using the Mail.app client on Mac OS X.
Most modern email clients, including Mac OS X applications such as Apple's own Mail.app, Qualcomm's Eudora, or Microsoft's Entrourage, all default to using the same mail transfer protocol: the venerable Post Office Protocol, more commonly known as POP. POP defines a simple set of instructions that lets users connect to a mailhost account, download new mail, and then disconnect. Nearly every ISP's mailhost supports POP, so it's a safe choice for mail-fetching client applications to support as well.
More recently, an alternative protocol known as the Internet Message Access Protocol (IMAP) has been steadily gaining popularity and support from Internet providers. IMAP contains a more sophisticated command set that allows users to store and organize mail on the server, instead of simply downloading and deleting it.
IMAP gives the most benefit to people who connect to a mailhost using more than one computer, since an IMAP-enabled mail account will look the same on all these machines, from overall mailbox structure down to the status of individual messages, and any changes made to a mailbox on one machine become visible to the rest.
This article provides an IMAP primer, focusing on receiving email though IMAP on computers running Mac OS X. It assumes you have some familiarity using Apple's mail application, or can at least fake it.
Before getting into Mac specifics, we'll look at IMAP's primary features, comparing them to those of the more familiar POP.
POP has been with us in its present form since the early 1990s. This protocol defines very few commands; just enough for a client to authenticate a user with a mail server, download all new mail from the server to the user's computer, and then bid the server adieu.
Most of the time, a client will clean up after itself by having the server delete all its new messages once they've been safely downloaded to a local hard disk. In other words, a POP client usually treats the server like a physical mailbox, letting it hold mail for only as long as it takes for the recipient to arrive and empty it out.
|
Related Reading 
Managing IMAP
Table of Contents
Index Sample Chapter Author's Article Read Online--Safari Search this book on Safari: |
In this way, POP lets you read mail with minimal connection time. Because a POP client always downloads mail messages in their entirety to a user's local machine, one can then disconnect from the Internet and browse new mail offline, going back online only when it's time to send replies and check for more new mail.
POP3's simplicity won it fast adoption across the Internet, but its disadvantages also lie in this simplicity. For one thing, it makes fetching mail an all-or-nothing proposition. If someone mails you a 50-megabyte attachment with an email, for example, you can't choose to download it later (or not at all); you must swallow it with the rest of the mail you're downloading during this session.
Furthermore, its download-and-delete usage model presents problems for users who access their mail accounts from multiple machines -- perhaps using one PC at work, another at home, and a laptop for traveling. Every time one machine's mail client downloads new messages, they then exist only on that machine, and remain inaccessible to the others unless manually transferred.
You can solve this problem to an extent by directing your client to keep mail stored on the mail server after downloading, but this isn't a perfect solution since you're left dealing with multiple copies whose structure and status can get out of synch with one another. For example, if you move or delete a message on one machine, it will retain its previous location on the others that have already downloaded it.
IMAP gets around POP's weaknesses simply by replacing the download-and-delete model with what we could call a keep-and-reveal one. An IMAP-aware mail server expects its users to permanently store all mail, both new and old, on the server. IMAP contains many commands for organizing all this server-side mail, including creating and editing mailboxes and folders (see the section called "Organizing Mailboxes"), and variously flagging individual messages (see the section called "Organizing Messages").
By keeping mail on the server, your view of your mailbox remains consistent, no matter where you travel, since all your email remains in only one place, instead of scattered across all the computers you use to read mail. Delete a message, and when you connect to your IMAP mailbox from a different machine, it's still gone. Reply to a message, and all your other computers can also see that you've replied to it.
(On the other hand, you can only store as much server-side mail as your mailhost account allows. You'll have to either delete or download some mail if you start running out of space. Fortunately, clients that support automatic caching, such as Apple's mail application, can assist with the latter option.)
IMAP also contains commands for fetching only immediately relevant parts of an email. When it checks for new mail, an IMAP client can request only the new messages' headers and bodies, and not immediately grab any large attached files, leaving their download up to the user's discretion.
  Are you using IMAP with the Mail.app client on Mac OS X? If so, let's here what you've learned. If you haven't tried it yet, ask your questions here. | |
While IMAP uses its own mail-fetching model, it doesn't totally abandon all the nice ideas that POP introduced. You can, if you wish, use IMAP in the same download-and-delete style as POP, and thus browse your mail while offline; the commands for this are all in the protocol. It's just up to your client application how deeply it supports these commands.
With that in mind, we'll use Apple's mail application (which I'll henceforth call by its formal name of "Mail.app" to prevent confusion between it and the small-m mail it handles), the one it bundles with every Mac OS X distribution, for the examples in the remainder of this article, because it does quite a good job in supporting IMAP's major features.
However other mail clients, including the aforementioned Eudora and Entourage, have IMAP modes of their own. You should feel free to explore the functionality of your favorite clients, even on ones outside Mac OS X. While the examples in this article are specific to Mail.app, the concepts apply to all IMAP-supporting mail applications.
I'll now guide you through setting up and using an IMAP-enabled mail account through the Apple mail application, covering IMAP-specific concepts as terminology as they come up.
Note: Of course, before you can start having fun with IMAP, your mailhost must support it! You can find out simply by asking your ISP or network administrator about it, and you can also poke your mailhost machine directly to find out, by launching the Terminal application and running a command like this:
% telnet
my-mailhost.net
143
If the host runs an IMAP service at port 143 (where IMAP usually lives, if it's anywhere), then you'll see a response that looks something like this:
* OK [CAPABILITY IMAP4REV1 X-NETSCAPE
LOGIN-REFERRALS STARTTLS AUTH=LOGIN] mail.my-mailhost.net IMAP4rev1
2001.315 at Sun, 5 May 2002 10:35:13 -0400
(EDT)
If so, you're good to go. If you instead receive a blunt
Connection refused or
something altogether different, ask the mailhost's masters
about its IMAP support.
By the way, this is how you can connect to an IMAP
server in the raw, without any friendly client program in
between. If you're feeling ambitious, you can type in IMAP
commands (as defined in the IMAP RFC -- see the section called "References") here and watch what happens. This
is probably only of interest to people who want to create
their own mail clients, or are otherwise curious about what a
mail-fetching program really does. For now, though, you can
send the server the LOGOUT command (just type
foo LOGOUT followed by
Return), which ends the IMAP session you manually started.
|
First, create a new account. Call up Mail.app's Preferences dialog through Mail-->Preferences..., and then click the Create Account button, as seen in Figure 1. As shown in Figure 2, select IMAP Account from the Account Type menu on the resulting sheet, and fill in the textfields with information appropriate to this account.
Click the same sheet's Account
Options tab to see Mail.app's IMAP-specific
options, shown in Figure 3. (The controls that appear under this tab depend upon
the type of account you've selected under the Account
Information tab.) Mail.app will fill the
Connect to server using port: textfield
with 143, the usual TCP port of a
mailhost's IMAP service. Change this value only if you know that
your host runs its IMAP service on a different port. Activate the
Use SSL checkbox if your host supports
secure IMAP (a.k.a. IMAPS) and you wish to take advantage of it;
see the section called "Using IMAP Securely".
Activate the Compact mailboxes when closing checkbox if you want Mail.app to purge your mailboxes of deleted messages when you quit the application; if left unchecked, deleted emails will remain within mailboxes, invisible to Mail.app but still accessible by other means. See the section called "Deleting Messages" for more details on how Mail.app handles IMAP mail deletion.
The sheet's Account
Directory textfield lets you specify the location of
this account's cache folder on your local file system. Unless
you have good reason to do otherwise, leave this field
blank; Mail.app will use its default location of
Library/Mail/IMAP/,
which should work just fine. See the section called "Message Caching" for more details.account-name
Once you've started to use the account, this field becomes grayed-out and uneditable.
The Account Path Prefix textfield
specifies the path that Mail.app will prepend to all mailbox names,
when it's trying to locate them on the mail server. If you keep
all your mailboxes in a directory called
my_mail within your home directory, then
you should put ~/my_mail here. (Where the
~ character is, of course, the Unix shorthand
for "my home directory".)
On the other hand, if you never log into your mailhost machine directly, or are otherwise not sure what to put here, then you're probably safe putting nothing here at all, and letting the server figure it out for you.
To help keep things
efficient, Mail.app keeps local caches of your IMAP
accounts' content, even though the "real" messages reside on
the server. By default, an account's cache lives in
Library/Mail/IMAP within your Home
folder, unless you specified a different folder when you
created the account (see the section called "Account Directory"). Every account gets its
own folder there, named IMAP/. Note the similarity between
the structure of this folder on my Mac, and that of my
actual, server-side IMAP mailboxes seen in Figure 4.account
name
Note:The mailbox files with imapmbox extensions are actually packaged
directories. If you're curious, you can explore their
contents by control-clicking the mailbox icons and selecting
Show Package Contents from the
resulting pull-down menu, or just cd into
them using the Terminal. You'll find some index and
preference files, as well as a
CachedMessages folder that contains one
file for every cached message's MIME section.
Through the Message caching: pull-down menu, you can specify how much of your incoming email Mail.app should cache, and when it should cache it:
This will direct Mail.app to download the entirety of every new message upon connection. This will allow you to read these messages and their attachments when offline, much as you can do through a POP account.
This is the default selection for a new Mail.app IMAP account.
When selected, Mail.app will cache all new messages' text bodies, as well as a list of any attachments for each, but not the attachments themselves (unless they're relatively small). If you specifically request to see a message's attachment (by clicking on the attachment's icon in the message view window), Mail.app will fetch a fresh copy from the server for you.
This is a good choice if you like the convenience that having all your textual email stored locally (which allows nice features like indexing and searching), but would like to avoid downloading large attachments you might not always want.
This directs Mail.app to hold off on any message caching when fetching new mail. It will display new mail in the message list as usual, but doesn't actually fetch a message's content until you select one for reading. Once it loads a message, Mail.app places its body into the cache. Subsequent visits to this message will read from the cached copy (unless the server's version of the message changes).
Like the previous menu choice, this does not cache large attachments.
Finally, this selection tells Mail.app to forget about caching entirely. Every time you access a message, Mail.app will fetch its contents from the server anew, regardless of whether you've read it before.
As testimony to IMAP's usefulness, the Mac.com accounts that Apple provides to its customers as part of its iTools package use IMAP as their protocol. This lets you consistently access and organize your Mac.com mail from any machine with an IMAP client —, Macintosh or otherwise.
Setting up a Mac.com-flavored IMAP account is easy; just select Mac.com Account from the Account Type pull-down menu, instead of IMAP Account. It's really just a shortcut that cues Mail.app to fill in the account configuration textfields to point to Apple's mail servers.
|
In the mailbox list underneath an IMAP account's header, you'll find a list containing an inbox, any mailboxes you create, and any mailboxes Mail.app creates in order to support some of its own special features.
An IMAP server abstracts all a user's new and otherwise unsorted mail into a single mailbox called inbox, so you'll always have at least this mailbox available to you.
Mail.app's commands for creating and organizing mailboxes and folders, found under the Mailbox menu, remains consistent across all its account types, IMAP included. When you create, rename, and delete mailboxes through the commands in this menu, or move mailboxes around by dragging their icons in the mailbox list drawer, Mail.app echoes these actions on your IMAP account's structure. Thus, all the changes you make in one session with Mail.app will carry across to any future connections you make to this IMAP account with any mail client.
While Mail.app takes full advantage of IMAP's ability to let you create and organize mailboxes any way you like, the application also has the ability to map its own functionality onto some special server-side mailboxes, if you let it. In all of the following cases, Mail.app will create these mailboxes on the server as necessary.
By default, messages that you save as a draft (through
File->Save As Draft (
-S), or the message window's Save As
Draft toolbar button) stay in Mail.app's special
Outbox mailbox, stored only on your
Mac. If you wish, you can instead store unfinished messages
on an IMAP mailbox, so they'll be available to choose and complete from other machines.
To do this, call up the Composing tab of Mail->Preferences as seen in Figure 5, and select one of your IMAP mailboxes from the Save unsent mail in: pull-down menu.
As you can see in Figure 4, I gave my drafts mailbox a
leading exclamation point in its name, for the simple reason
that !drafts will always appear at the
top of the alphabetically-sorted list of server-side
mailboxes, and I didn't want it mixed in with that list. (Of
course, I did that before I got around to sorting all my
other mailboxes into their own folders, which makes them all
stand apart anyway. Oh well. It made sense at the
time.)
It's worth noting that Mail.app does not keep its Sent Messages mailbox on the IMAP server; it's only on your Mac. Mail.app stores copies of all the mail you send through all your accounts, IMAP and otherwise, here.
If you want to keep server-side copies of sent mail, choose an IMAP mailbox from the Save sent mail in: pull-down menu found under the Preference panel's Composing tab (Figure 5).
If you have the Move deleted mail to a folder named: checkbox set under the Viewing tab of Mail-->Preferences..., then Mail.app will create a folder on the server to serve as a trashcan, where deleted mail will move itself. It will name this folder either Deleted Messages, Deleted Items, or Trash, depending on your selection from that checkbox's attached pull-down menu (Figure 6).
You don't need to have a special folder for maintaining deleted messages since IMAP lets you store deleted mail in any mailbox, as described in the section called "Deleting Messages." However, Mail.app doesn't let you see any deleted messages except for ones in this special mailbox, and only if you have this checkbox activated.
An email message sitting in an IMAP mailbox can have some number of message flags set on it, recording the actions performed on this message, such as the user's reading, replying to, or deleting it. When you reply to a message on your office PC, for example, and then later connect to your mailbox at home, that message will "remember" the fact you already replied to it, and be able to report this to you.
Mail.app works with most flags in a fairly straightforward
fashion, but gets a little squirrelly when it comes to IMAP's
Deleted flag, as we shall see.
Note: There's nothing particularly magic about how message flags work; they exist simply as headers the IMAP server adds to the messages on its end as their status changes.
Mail.app displays IMAP flags through symbols in the Flag and Status columns of a mailbox's message list, as shown in Figure 7. The following list uses the flag names as they're defined by the IMAP standard, and explains how Mail.app represents them.
A message gets a Recent flag if the
current IMAP connection is the first to have seen it. Mail.app
places a blue dot in this message's status column, marking
it as a new, unread message. Mail.app unsets this flag once
the user reads a message.
A message with a Seen flag has
been read.
Mail.app reacts to the absence of this flag; a message lacking a
Seen flag (which all
Recent messages do, by definition)
gets a blue dot. Mail.app does not distinguish between
unread mail that arrived since the current session started
(and has a Recent flag) and unread mail carried over from a previous IMAP session (and
therefore has no message flags).
|
Related Reading 
|
Selecting
Message-->Mark As Unread (Option--M) removes this flag from selected messages,
and Message-->Mark As Read (Option--M) sets it. (One of these two commands
appears in the Message menu,
depending upon the status of the selected messages.)
Replying to an IMAP message prompts Mail.app to set its
Answered flag. Mail.app displays such
messages with a little U-turn arrow in its status column,
unless the message lacks a Seen flag.
The Flagged flag can mean
whatever you want. Generally, it's meant to signal that a
message requires urgent attention.
In Mail.app, you can toggle this flag for the selected
message(s) through Message-->Mark As (Un)Flagged (Option--G). Flagged messages receive
little flag icons in the message list's flag column (see
Figure 7).
Mail.app sets a message's Draft flag if
it's an unfinished, unsent message you're storing in an
IMAP mailbox, as described in the section called "Drafts".
Mail.app gives messages
Delete flags when you delete them
(through the Delete key, or
Message-->Delete, or dragging them into your dock's Trash
icon). This seems fairly straightforward, and it does more
or less what you want, but this flag's actual
implications are convoluted enough that this article
dedicates a whole section to the topic: see the section called "Deleting Messages".
IMAP uses a two-step process for deleting messages. Any
message can set a Deleted flag on itself,
which marks it as susceptible to actual deletion, but doesn't
actually get rid of it or even move it out of its original
mailbox. A separate IMAP command purges a mailbox of all the
deleted messages it contains.
Different mail clients have different ways of representing deleted (but not yet erased) messages to the user. Mail.app chooses to simply not show deleted mail at all, unless it's inside the designated "trash" mailbox, as described in the section called "Trash Mailbox".
Mail.app's IMAP response to deleting mail changes
depending upon how you've set the Move deleted mail to
a folder named: checkbox seen in Figure 6. If you've checked it, then
deleting a piece of mail will cause Mail.app to move it to your
chosen trash mailbox, rather than setting its
Delete flag.
If you've instead left that checkbox
unchecked, then Mail.app will set the message's
Deleted flags, but otherwise leave them
be. Since Mail.app refuses to display deleted mail in mailboxes
other than the trash mailbox, this action will also make the
message vanish from sight, even though it continues to exist on
the server (and perhaps remain visible to other mail
clients).
That same checkbox also dictates Mail.app's behavior with
actually erasing deleted messages. If checked, then Mail.app gives
you a Mailbox-->Empty
Trash
Mailbox (-K) command. This will have Mail.app send the IMAP EXPUNGE
command to its trash mailbox, and since it contains only
messages with the Deleted flags set,
they'll all go away (unless you've been weird and snuck other
mail in there through sneaky means; those would stick
around). Deleted messages in other mailboxes, however, simply
remain present and invisible to you, at least so long as you use
Mail.app as your client.
If you leave this box unchecked, then Mail.app instead offers
the Mailbox-->Compact Mailbox (-K) command. This will simply expunge the selected
mailbox, permanently erasing all its unseen deleted messages,
and seeming to shrink the mailbox's size without affecting any
of its visible messages. ("Compact", in this case, is Mail.app's
positive way of spinning the fact that it doesn't have a way
of dealing with deleted mail in arbitrary IMAP mailboxes, and
so they appear as so much dead weight.)
Note that both these commands share the -K keybinding, so hitting this combo will always
erase deleted mail, one way or another.
Like POP, IMAP offers no built-in security mechanisms, meaning that network eavesdroppers may be able to obtain your account password when you fetch mail. Depending upon the services your mailhost provides, though, you may be able to avoid this attack by using IMAP over a secure, encrypted network connection. See my earlier article, Secure Mail Reading on Mac OS X, for details.
The IETF maintains the RFC documents that define the standards of POP and IMAP (and most every other technology the Internet uses).
Version 3 of the Post Office Protocol is defined in IETF RFC 1939.
Version 4 of the Internet Message Access Protocol is defined in IETF RFC 2060.
The IMAP Connection at the University of Washington (IMAP's alma mater, where the protocol was first drafted and implemented by Mark Crispin) serves as a comprehensive information repository regarding all things IMAP.
Thanks to Jason Lavoie for assistance with this article.
Jason McIntosh lives and works in and around Boston. He has co-authored two O'Reilly books, Mac OS X in a Nutshell and Perl & XML.
Return to MacDevCenter.com.
Copyright © 2007 O'Reilly Media, Inc.
