Today I just want to show off a fun little project I'm integrating into the Tidal dashboard. This article is applicable to anyone who wants to process a bunch of incoming emails -- potentially addressed to different recipients -- and route them either into user dashboards or support buckets or something similar.

Running your own email servers is pretty annoying. Outgoing email is a nightmare if you want to show up anywhere outside of the spam folder. Incoming email is a nightmare if you want a highly-available and reliable inbox.

Fortunately, the outbound email problem is solved by providers like SendGrid (my personal favorite). And incoming email: why not just use Gmail? I can't tell you the number of people I've interviewed with this very question, and none of them have ever suggested using IMAP to interface with Gmail to receive emails.

And my specific setup goes a step further. I don't want just grab incoming emails, I want to grab incoming emails that are intended for an infinite number of different people. I happen to be routing the emails to different dashboards, but that's not necessarily what you're doing. You may want to forward emails or do some algorithmic processing on them or something.

There are two aspects of Gmail you can leverage to use one account to handle multiple recipients. Choose which one works best for you.

  • I use a Gmail catch-all account for my project. The account is "support@domain.com" but any email with a non-existant recipient will end up there. That means that "client-name@domain.com" and "other-client@domain.com" will both end up in the support catch-all account. This is a great approach if you have clients that require something a little more branded and whitelabeled.
  • If you have thousands of casual users, you can always use the plus sign trick. If you have an account "incoming@domain.com", you can just tack on some kind of user identifier to the mailbox name: "incoming+2912390@domain.com", "incoming+bobjones@domain.com", or whatever floats your boat.
Either technique will give you the desired result of being able to handle multiple destination addresses with one mailbox. Which is great, because we'll now use the PHP IMAP extension to grab those emails!

I should note here that you can always just make separate Gmail inboxes to handle your various accounts. I like the approaches above because you don't have to create a new inbox every time you get a new client, but if your client base is slow-growing or you can't use the methods above, making a separate inbox will certainly work.

The IMAP extension is stupidly simple to use, and I'm surprised I had never toyed with it before. I'm happy this project came up, because it was super fun to learn.

The one caveat is that the IMAP extension generally doesn't ship with PHP. On Linux systems you can use your package manager to install it pretty easily, and on Mac all you have to do is compile the IMAP module separately (no need to recompile all of PHP) and install it into your modules directory.

$imap = imap_open(
	IMAP_SERVER,
	IMAP_LOGIN,
	IMAP_PASSWORD
);

Server strings are the trickiest part of the above, and they look something like this:

{imap.gmail.com:993/ssl}INBOX

Check out the PHP imap_open documentation for details on the server string. It includes the server address, port, and options (the /ssl is not a path). You can also specify the default mailbox to connect to.

$emails = imap_search($imap, 'ALL', SE_UID);

The imap_search function is the best way to loop through emails in a mailbox. The function returns a simple array of email IDs. The third parameter above, SE_UID (part of a bitmask), specifies that I want the unique email ID -- otherwise it'll return the regular mail ID. The distinction is this: every email in the account has a unique UID, but they also have a "mail ID" which is the same as "email # X of 1,200" (where X is the mail ID). I feel safer using the UID, since you can store the ID value in your database and rest assured that it will always reference a single email.

The second parameter above is amazing! Values can be things like 'ALL', 'UNSEEN', 'UNANSWERED', 'UNSEEN FROM mom@website.com', and all sorts of great, human-readable filters like that. Again, check out the imap_search documentation for details.

The $emails variable now holds an array of UIDs. Keep in mind that IMAP uses the plain-ol' ID by default, so if you decide you want to UIDs you need to specify that in every imap_* function call.

foreach ($emails as $uid)
{
	$overview  = imap_fetch_overview($imap, $uid, FT_UID);
	$headers   = imap_fetchbody($imap, $uid, 0, FT_UID);
	$plaintext = imap_fetchbody($imap, $uid, 1, FT_UID);
	$html      = imap_fetchbody($imap, $uid, 2, FT_UID);
}

Some important things to note:

The overview function can actually take a comma-separated list of IDs for the second argument. If you give it a list, it will return the overviews for that entire list of emails. This is great if you just want to get a quick glimpse at the subjects and from fields for the whole inbox -- no need to download the whole message. This is one of the core philosophies of IMAP.

Note that when you call imap_fetchbody, the message will be marked as seen (or "unread" in Gmail terminology). To avoid this, use the FT_PEEK constant in the bitmask:

$plaintext = imap_fetchbody($imap, $uid, 1, FT_UID | FT_PEEK);

Finally, $headers, $plaintext, and $html all use the imap_fetchbody function with the third parameter as either 0, 1, or 2. I don't believe that any of these are guaranteed to exist, but I'm not sure there so be safe and double check.

If your application should sort emails by "to" address, use the overview to check out the "to" field. If your application makes use of CCs, however, you'll need to look at the entire header section. The documentation says that the overview function is supposed to return multiple "to" and "cc" recipients, but my PHP 5.3.16 extension doesn't do that. It may be a bug, or it may just be a 5.4 feature -- I haven't looked into it that deeply. While it's slightly annoying that the overview function isn't doing that properly, it's also not that hard to parse the headers.

I'm very glad I got to play with this, and I'm happy to say that this will be appearing in the next release of Tidal's dashboard in production. <3 IMAP.