Eventual Consistency

Backbone.js with Django 1.5

Backbone.js and other client-side MV* frameworks are a relatively new paradigm in designing and building rich web applications. Since this transition is still ongoing, most serious server-side frameworks are just now catching up to this trend. Django included.

This is not a critique on Django - most sites and apps are still built in the "traditional" way. Backbone and friends are not a one-size-fits-all solution. They have their respective use cases, but they do introduce a new level of complication that is simply not needed in many real world project.

That being said, while there are lots of tools around to make the transition to single page apps smoother, a big part of Django is still about rendering templates, and the prevalent serialization format is still HTML.

These are some lessons I've learned by experience, trying to reduce the friction between the two frameworks.

Models != Resources

Backbone.js has two classes for working with data: Models and Collections. Intuitively, this is reminiscent of Django's own Models and QuerySets, but this is generally not an accurate comparison. Backbone relies on RESTful APIs, in which a model is actually a RESTful resource.

In real life, such resources are rarely easily mapped into a row in a table (represented by a Django model). A resource could be a composite of several such models, or a only a subset of the fields from a single model. In some cases, we may have resources that don't map to anything in our database (or other persistence layer) at all.

Realizing this early can help us avoid common pitfalls when designing our API. It is good practice to maintain a loose coupling between the persistence layer and the presentation one, if possible.

Using an API framework (or not)

Django has a large ecosystem of open source reusable apps created by 3rd parties. Their job is to help us solve problems not tackled by Django's core. Creating RESTful APIs is one of those problems.

The two most popular apps for this are Django-Tastypie, and the newer Django REST framework. While they both offer great tools for common problems when desiging APIs (serialization, authentication/authorization, versioning to name a few) - they both really shine when our resources are in fact mappable into DB Models. For many cases this is exactly what we need - which makes both frameworks very feasible options.

More complex setups or APIs that rely heavily on other sources besides Django's model layer are better-off with other alternatives.

In many cases I find that simply working with views that serialize to JSON can provide a good start:

import json
from django.http import HttpResponse

class JSONResponse(HttpResponse):
    """
    Return a JSON serialized HTTP resonse
    """
    def __init__(self, request, data, status=200):
        serialized = json.dumps(data)
        super(JSONResponse, self).__init__(
            content=serialized,
            content_type='application/json'
            status=status
        )

class JSONViewMixin(object):
    """
    Add this mixin to a Django CBV subclass to easily return JSON data.
    """
    def json_response(self, data, status=200):
        return JSONResponse(self.request, data, status=200)

Handling CSRF protection

Django comes with built-in CSRF protection, which is turned on by default.
This is a security best-practice, and I highly advise to leave it in place.

One important thing to note about it, is that while it makes working with HTML forms easy, using it in conjunction with AJAX and Backbone's Sync module takes some adjustments.

The easiest solution I've found for this was embed the CSRF token in a meta tag in the HTML header, and to modify Sync to take the token from the DOM and add it to the AJAX request's HTTP headers, like so:

<!-- inside our page's <head> tag -->
<meta name="csrf-token" content="{{csrf_token}}">

And in our javascript app add this:

(function() {
  var _sync = Backbone.sync;
  Backbone.sync = function(method, model, options){
    options.beforeSend = function(xhr){
      var token = $('meta[name="csrf-token"]').attr('content');
      xhr.setRequestHeader('X-CSRFToken', token);
    };
    return _sync(method, model, options);
  };
})();

Adapted from this gist.

Working with client side templates

Backbone apps generally rely on a client-side templating engine to render HTML. However, many of the more popular ones (namely Mustache and Handlebars) have a syntax similar to Django's own template system.

This can cause collisions between what Django's template engine tries to render, and client-side templates that we want to embed on the page served by Django.

New in Django 1.5 is the verbatim tag. Data placed between a {% verbatim %} tag and a {% endverbatim %} tag will not be interpreted by Django's template system.

Users with older versions of Django can use something like this to achieve a similar result.

Making our app "real-time"

One of the main reasons to build single page applications with Backbone and the likes is speed. We don't want the user to wait for the entire page to reload every time we change the underlying data. In a similar manner, we also want the client to reflect changes that happen on the server that might have been triggered by someone else.

Generally today, there are two ways to achieve such "real-time" behavior:

  • Pulling - This means that the client-side application will ask the server every once in a while if anything had changed. the most common implementation is called XHR Polling, which is just a fancy name for doing an AJAX GET request every few seconds. This requires no changes to your Django code, just be sure to set the interval to something that makes sense to prevent yourself from DDoSing your own servers.

  • Pushing - Meaning that the server pushes updates to the clients whenever something changes. This is usually done with either WebSockets, SSE (Server Sent Events) or Comet (Mainly Long Polling). Sockets and SSE are both relatively new and only supported by modern browsers (websockets are also supported by IE10. To my knowledge, No version of IE supports SSE).

One thing to note about "push" style connections is that they require a persistent connection between client and server. This means that for each user currently visiting our app, we need to keep an open socket to the server. This is a non-trivial thing to do once you have a sizable amount of concurrent connections (see the infamous c10k problem).

The thing about Django is, that it is predominantly a synchronous framework: a request goes in, server/thread/worker blocks while we compute stuff, a response goes out. The notion of persistent connections is somewhat removed from Django's scope. If we really MUST setup a persistent connection, something like Tornado, Twisted, or even Socket.IO are better suited for that specific problem.

For most cases though, a quasi-realtime connection using something like XHR polling is usually good enough. It's easy to do with Backbone, it keeps our stack simple, and has the added benefit of being arguably more robust in the sense that there is no connection that could break - which tends to happen a lot, especially with websockets over a mobile connection.


As always, I'd love to keep the discussion going. Please leave a comment bellow,
Or Follow me on Twitter where I talk a lot about Backbone, Django and other web development related things.

comments powered by Disqus

Hi, I'm Oz Katz

I am a co-founder and CTO over at Swayy.

I usually write about software development using Python, JavaScript and other awesome, open source tools.

Feel free to reach out on Twitter, or contact me using the links at the bottom of the page.