MongoDB in Django

Traditional relational databases (mySQL PostgreSQL etc) and noSQL systems are not mutually exclusive. I have several Django applications that are happily using mySQL. If your site is not scaling due to your database, you are doing it wrong! noSQL will not help you until you start caching some of those expensive queries using something like Memcached.

I use MongoDB alongside mySQL for all the dirty work like storing stats for later processing. There’s no point in polluting mySQL with this sort of data, especially when you’re dealing with millions of entries.

This post is intended for absolute beginners who use Django tranditionally and are curious about how they can integrate a secondary storage service into their apps. I’m assuming you have already installed MongoDB on your dev environment. You will also need to install the MongoEngine library for Python.

Let’s start.

You already know how to create data models in Django, but let’s say we want to store an activity feed for your users everytime they do something on your site. We begin by creating a data model similar to Django’s ORM using MongoEngine but the difference here is that you don’t need to run “syncdb” to create your tables. Mongo’s collections (similar to SQL tables) are schemaless so these models can be manipulated and you won’t need to worry about running migration scripts.

Let’s create a simple collection for storing user activities. Create a file where you normally keep your Django models and call it mongomodel.py

1
2
3
4
5
6
7
8
9
10
11
12

from mongoengine import *
 
# connect to a db (no need to create this - it will be created automagically)
connect('useractivity')
 
class Author(Document):
    pk = IntField()
    name = StringField(max_length=200, required=True)
 
class Activity(Document):
    message = StringField(max_length=200)
    author = ReferenceField(Author, reverse_delete_rule=CASCADE)

“What’s this??!! Django already has a User model, why do I need another in Mongo?” Well, you don’t, but say you want your activity to say something like: “Joe uploaded a photo” and you want Joe’s name to be linked to his profile page. We keep a reference to his mySQL id in case we need to look up other info or construct a URL.

You’ll also notice in the Activity model we are referencing the Author model. This is like a foreign key that will allow us to create relationships, similar to SQL. The CASCADE option will make sure if the user is deleted, all activities are also cleared out.

Ok, let’s start using this puppy! Using the example above we want to create an activity for Joe next time he uploads a photo. First, import mongomodel.py whenever you’re planning to interact with Mongo. In my photo upload view function I will create an activity like so:

1
2
3
4
5
6
7
8
9
10
11

# After photo upload is complete
 
from main.mongomodel import *
 
# first create a user object - you can grab data from request object
the_author = Author(pk=request.user.id, name=request.user.first_name)
the_author.save()
 
# now create the activity
activity = Activity(message='uploaded a new photo', author=the_author)
activity.save()

That’s it. If you decide later you also want to add the name of the file uploaded you can simply add a new field to your Activity model and it will just work, plus it will be backwards compatible, i.e. older records without this field will not complain. Lovely.

Displaying the activity is just as simple. In your view function pull out the record and push down to your template:

1
2
3
4
5
6
7

from main.mongomodel import *
 
# get all activities
activities = Activity.objects
 
# push down to template
return render_to_response('activities.html', {'activities':activites})

Now in your template loop and output like any other model:

1
2
3
4
5

<ul>
{% for a in activites %}
    <li><a href="{% url main.views.profile a.author.pk %}">{{ a.author.name }}</a> {{ a.message }}</li>
{% endfor %}
</ul>

I’ve used the user’s mySQL primary key to construct his profile URL.

This is a very basic example but hopefully you can see the advantage of offloading some of the data storage to Mongo. You may ask “but what if the user changes his name? won’t the data in the activity remain out of sync?”. Yes, it will, but you can very easily add a simple method in your Django user model to update Mongo records whenever the user’s details are updated.

Good luck.