Trav Stone

Note: Sad to say, but it looks like openspf.org is no longer around. I've updated this post to point to the Wikipedia page for SPF instead.

Hard to believe, but I've finally gotten around to setting up SPF and DKIM for my Mailserver. In the process, I managed to clean up a few other things too. If you haven't set up SPF/DKIM yet, it's probably worth it–but maybe do it smarter than I did.

The Prelude

For a few years I've been ignoring some issues with my Mailserver. I have a small-ish VPS that I run a few sites from–It's probably more or less your standard WHM/cPanel on Linux. I've been hosting sites for more than 10 years now, through a succession of VPSes. Over the years I've built up more than a few domain forwards, unused email addresses, and random debris. All of this added up to fair amount of noise in my system email and logs.

Thinking I was clever, about two years ago I switched the system email address to a Gmail one; I was thinking that I'd use the features to help sort and categorize mail, and benefit from Google's robust SPAM filtering. That last part is what eventually brought the whole thing to a standstill.

Initially it worked ok, but as time went on more and more of the mail I got consisted of failure messages: my server's message was rejected by Gmail (so it never reached the inbox). Oddly, or obviously, enough, the failure message itself did reach my inbox.

While that issue was unfolding, a similar thing was happening on the couple of forms that appeared within the handful of sites on my server. The problem forms were sending (via my server's SMTP) submissions to outside addresses at Gmail and elsewhere. Within the last couple months, they eventually were being rejected almost every time–to the point that there were user complaints. Bad Webmaster.

Start the Repairs

Having let things languish for a couple years, it was now time to fix things ASAP. I'd noticed that there was a link in each email for more information, so I followed it and read through the (brief) info there. I also checked out a couple of the links within that page, where I learned about the importance of SPF/DKIM. Google dings you without them.

Fair enough, so off I went trying to set it all up. As it turns out, WHM/cPanel makes it fairly easy to add SPF/DKIM to your DNS records. Easy is a relative term though; I don't really know anything about this stuff, so it took a bit of reading and trying things before it started working. It wasn't initially apparent that I only needed part of the generated DKIM string to go into my DNS record.

Adding the email authentication stuff had a more or less immediate impact on deliverability. There were still some failures, but these were essentially actual SPAM messages being routed through the system (did I mention Bad Webmaster).

While I was doing that, I also registered for the Google Postmaster Tools by adding some TXT records. Ultimately this account hasn't really helped me: my server just doesn't send enough email to matter. The dashboard in GPT is still completely empty after several weeks. Similarly, my server doesn't really have a 'low' reputation... it just doesn't have a reputation at all.

Fortunately, the server IP doesn't appear to be on any blacklists at this point.

Getting Squared Away

The SPF/DKIM stuff was a start, but this revealed another problem: my server was not being very smart about routing email. It was essentially blindly relaying messages without doing much vetting aside from asking if the recipient existed. This is dumb, since there are some apparently reasonable tools for dealing with these problems included in WHM/cPanel.

After setting up global SPAM filtering and email authentication, there was a huge drop in the number of messages being passed along. It appears as though most of the messages that fail the basic SPF/DKIM tests are simply rejected on receipt. This is good. At this point form-based messages appear to be getting delivered 100% of the time. This is great.

Upon experiencing this success, I did what anybody would do: I kept pushing things too far.

Go Ahead and Blow Things Up

Feeling very content with myself for achieving teh awesome, I decided I'd start fiddling around with more advanced options, and run some other security checks. Don't be like me. Or, do be like me, but go about it differently. Read through things a few times first before you go a-wacking on the server. I on the other hand just set an option that prevents nobody from sending mail. Doesn't that sound like a good idea?

It is a good idea- but if you don't have your PHP configured right, then it's a bad idea. As soon as I made the change, the form failures started rolling in. PHP likes to send mail as nobody, unless you're using suPHP, in which case it'll send as the account name (which remote email servers like better). I switched to using suPHP (heedless of the warnings), and succeeded in breaking entire sites. Turns out, when they say that you'll need to check owners and permissions, they actually mean it.

Any Minor World that Breaks Apart Falls Together Again

Long story short: when I finally got the owner/permission issues sorted out (and a little hack for a PHP weirdness), I finally had forms sending properly header-ed email with appropriate SPF/DKIM records.

A nice side-benefit of all this is that I was actually able to correct a bit of a problem I was having with posts in this very blog.

So, to recap, if you want to get your email server straightened out, you just need to have a little knowledge of the following:

• Apache
• Linux
• WHM/cPanel
• PHP
• .htaccess
• SMTP/POP3/IMAP
• SPF/DKIM
• Encryption-based authentication
• The whims of Gmail
• DNS
• How to maintain your cool when you dig a big hole for yourself

...and that's it!

I guess the moral of this story is if you're going to have a server, you have to take care of it. It's like a kid... er, pet.... er, houseplant?