tushman.io

The musings of an insecure technologist

A Better Way to Manage Indexes With Mongoengine

I was unsatisfied with out-of-the-box solution that mongoengine gave me for managing indexes. What they offer is adding some meta information to your model, and then it litters ensure_indexes when making calls. There approach is shown here

I find that I need more finegrained control of defining indexes, and more importantly when to kick off there creation.

So I added an ensure_indexes() method to my manage.py, and then dropped down to pymongo to get the control that I want

It looks something like this …

manage.py 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

from pymongo import ASCENDING, DESCENDING
from mongoengine.connection import get_db
import database

@manager.command
def ensure_indexes():
    database.connect()
    db = get_db()

    vt_collection = db['video_test']
    vt_collection.ensure_index([('video_batch', ASCENDING), ('video_total_views', DESCENDING)], background=True)
    vt_collection.ensure_index([('video_batch', ASCENDING), ('video_name', ASCENDING)], background=True)


    # show index info:

    for collection_name in db.collection_names():
        print "Collection: [{}]".format(collection_name)
        col = db[collection_name]
        print col.index_information()

I like using mongoengine’s get_db() method to get a handle on a pymongo database object, it centralizes the connection logic which I keep in my database.py file, which for completeness is here:

database.py 
1
2
3
4
5
6

from settings import *
import mongoengine
  
def connect():
    mongoengine.connect(MONGO_DATABASE_NAME, host=MONGO_HOST, port=MONGO_PORT,
                        username=MONGO_USERNAME, password=MONGO_PASSWORD)

While figuring this out the following two docs to be useful:

Comments