Eventual Consistency

Getting E-Mail right with Django and SES

Sending E-Mail is an important part of almost every web or mobile product. And while it may seem like a trivial thing to do at first, getting it right and finding the optimal setup is often not that easy. Spam filters, bounce rates and throttling are all things we must take into account, and sooner or later they WILL fail us.

Here's a quick run-down on our current setup at Swayy. While it definitely isn't perfect, it held up pretty well so far, sending tens of thousands of emails, with a relatively high acceptance (and conversion) rate. I'll go over setting up Django with Amazon's Simple Email Service, while covering some best practices and things to avoid.

Step #1 - setting up the Domain

Before we dive into code, we first need to prepare our DNS records. There are basically two things that we need to do.

The first step is to add a DNS SPF record - a TXT record that lets other mail servers know we allow email originating from SES to use our domain name in outgoing mail.

$ dig swayy.co TXT

; <<>> DiG 9.8.3-P1 <<>> swayy.co TXT
...
swayy.co.       3600    IN  TXT "v=spf1 include:_spf.google.com include:amazonses.com ~all"
...

As you can see, our SPF record contains two domains from which outgoing email can originate - the first is our Google Apps account that sends email from our personal username@swayy.co accounts, and the later is SES, used by our app servers for automated emails.

The second part in setting up the domain, is to enable DKIM signing. If you are also using Amazon's DNS solution - Route53, you can follow this document to easily enable automatic DKIM signing for your domain. Otherwise, it's simply a matter of assigning a few CNAME records to the domain.

Step #2 - Setting up Django to use SES

So our domain is properly configured, which means we can now use SES to send email on our behalf. Django makes sending email rather easy, with pluggable Email backends.

With SES, you have 2 options: either use SES as an SMTP server, or call it via the AWS web service. Since both are mature by now, I don't think there is any noticeable difference between the two. We are currently using the AWS web service and are pretty happy with it.

SMTP backend

To use SES as an SMTP backend, we need to wrap Django's default SMTP backend with an SSL capable one, since SES requires SSL. There is a ready made implementation called django-smtp-ssl that we can use. Once installed, add the following to your settings module:

EMAIL_BACKEND = 'django_smtp_ssl.SSLEmailBackend'
EMAIL_HOST = 'email-smtp.us-east-1.amazonaws.com'
EMAIL_PORT = 465
EMAIL_HOST_USER = 'ses-smtp-username'
EMAIL_HOST_PASSWORD = 'ses-smtp-password'
API backend

Using the web service is just as easy. This time we'll need to use django-ses, which is an email backend wrapping the python AWS API library - boto. After installing the package, add the following to your settings module:

EMAIL_BACKEND = 'django_ses.SESBackend'
AWS_ACCESS_KEY_ID = 'aws-access-key'
AWS_SECRET_ACCESS_KEY = 'aws-secret-access-key'
AWS_SES_REGION_NAME = 'us-east-1'
AWS_SES_REGION_ENDPOINT = 'email.us-east-1.amazonaws.com'

For more information on how to configure both backends, see their official documentation - but for most implementation this will probably do.

Step #3 - Using Django's templates in Email

Let's get it out of the way now: HTML email is a PITA. It is hard, if not impossible to create good looking HTML email that will work consistently across devices. and forget every best practice you picked up over the last +5 years regarding HTML and CSS. It's mostly inline styles and tables for layout. Now that we are mentally prepared, lets see how it is done.

Django comes with a class called EmailMultiAlternatives that will allow us to send an email that contains both an HTML and a plain text representation of our email body. Make sure to always include a text version! Not all mail clients support HTML.

Since we don't plan on passing static HTML strings, rendering the email body (for both the text and the HTML versions) using Django's template system would be nice. I use a wrapper around the EmailMultiAlternatives class to simplify the process. It looks something like this:

from django.conf import settings
from django.core.mail import EmailMultiAlternatives
from django.template.loader import render_to_string

class EMail(object):
    """
    A wrapper around Django's EmailMultiAlternatives
    that renders txt and html templates.
    Example Usage:
    >>> email = Email(to='oz@example.com', subject='A great non-spammy email!')
    >>> ctx = {'username': 'Oz Katz'}
    >>> email.text('templates/email.txt', ctx)
    >>> email.html('templates/email.html', ctx)  # Optional
    >>> email.send()
    >>>
    """
    def __init__(self, to, subject):
        self.to = to
        self.subject = subject
        self._html = None
        self._text = None

    def _render(self, template, context):
        return render_to_string(template, context)

    def html(self, template, context):
        self._html = self._render(template, context)

    def text(self, template, context):
        self._text = self._render(template, context)

    def send(self, from_addr=None, fail_silently=False):
        if isinstance(self.to, basestring):
            self.to = [self.to]
        if not from_addr:
            from_addr = getattr(settings, 'EMAIL_FROM_ADDR')
        msg = EmailMultiAlternatives(
            self.subject,
            self._text,
            from_addr,
            self.to
        )
        if self._html:
            msg.attach_alternative(self._html, 'text/html')
        msg.send(fail_silently)

All it does is call render_to_string to render a Django template into a string, and pass that on to the EmailMultiAlternatives class.

Important notes on HTML Email

As I said, HTML email is a pain. Here are a few tips that will make life a little more tolerable:

  • Use a ready made template as base. You can use the awesome HTML Email Boilerplate project, or one of Mailchimp's ready-made templates. With django, you can use these as a base template and extend them.

  • Avoid images when possible. While others might suggest the opposite (styling the entire email using images since they do not depend on the client's rendering quirks), many clients nowadays block images by default for incoming mail. This is at least true for Gmail and Google Apps, which make up a pretty sizable chunk of the market.

  • Personalize the body text - If you are sending out notifications or digests, do your best to make the email as personal as possible. Use the user's name, country, sign-up date and other identifying features in the text. This will both lower your chances of getting marked as spam (again mostly by Gmail - they tend to block messages that look exactly the same but go out to many users during a short period), and more importantly it also builds trust with that user, which will usually lead to better conversion.

  • Add a prominent "Unsubscribe" link. Again to avoid spam filters, and to help build trust. Some countries actually mandate such a link by law, to appear in any commercial email.

Step #4 - Monitor those bounce and complaint rates

Good email delivery relies heavily on reputation. Use the AWS SES console to monitor your bounce and complaint rates. Both should be kept well below 1%. If this is not the case for you, there are effective methods you can use further lower these:

  1. Validate email addresses by sending an activation link - While there is a small trade-off in terms of user experience with these emails, it's not that bad and most users kind of expect them by now. If you are using a social login feature such as Facebook Connect, you can usually safely use the email provided by the social network as they have already validated it.

  2. Reduce typing errors with mailcheck - it's an ingenious little library that binds to the email input field and checks for common spelling errors such as hotnail.com or yajoo.com. It will help lower the bounce rates for activation email.


As always, I'd love to hear your tips for better email delivery, and to discuss the ones I mentioned. Feel free to leave a comment or come talk to me on Twitter.

comments powered by Disqus

Hi, I'm Oz Katz

I am a co-founder and CTO over at Swayy.

I usually write about software development using Python, JavaScript and other awesome, open source tools.

Feel free to reach out on Twitter, or contact me using the links at the bottom of the page.