tl;dr: I forked the lettuce package to use multiprocessing, tests run more then 4x faster on my MBP

I am a fan of Gabriel Falcão’s lettuce Behavior-Driven Development (BDD) tool. We have been using it on my team for 6+ months now. Recently our test suite completion time has crossed the 10 minute line, which had a bunch of negative effects, as you can imagine:

• people writing less tests
• people running the test suite less frequently
• people spending more time watching a test suite run, then coding, …

We all are using relatively modern MBP with 4 cores, and we might as well make the most of them. Here is my fork of lettuce that allows you to take advantage of all of your cores:

https://github.com/jtushman/lettuce

I have made two main modifications (You will find the lion share of my modifications in this file):

• I created a ParallelRunner (I have left the main runner alone), which kicks off processes to pull the scenarios off a queue
• After each run I store the run times of each test in a .scenarios file, so in subsequent runs I can sort them longest to shortest

My test suite used to take 12 minutes, now its takes 2 minutes — REJOICE!

Usage

lettuce tests -p 4 -v 2

-p: stands for parallel. You can set it to how many processes you like, I find that the number of cores should be your default

-v is the same verbosity parameter, but I recommend setting it to 2 when using parallelization, otherwise the steps will interlace and not make much sense

in your terrain.py file, there are two new callbacks:

@before.batch and @after.batch

which you should use to set up and tear down each process. I use main to fire up flask, selenium and mongo. Also note that I set a port_number attribute on world which you can use set up processes specific servers. For example:

1
2
3
4
5
6

@before.batch
def batch_setup():
    settings.MONGO_DATABASE_NAME = 'testing__{}'.format(world.port_number)
    mongoengine.connect(settings.MONGO_DATABASE_NAME, host=settings.MONGO_HOST, port=settings.MONGO_PORT,
                        username=settings.MONGO_USERNAME, password=settings.MONGO_PASSWORD)
    clear_database()

Caveats

For this to work all of your tests need to be isolated, they can not depend on each other (which I think is best practice anyways). This means in your tests you should not use world at all Use scenario instead:

To do this, in your terrain file add the following:

1
2
3
4
5
6
7
8

class ScenarioState(object): pass

@before.each_scenario
def setup_senario(senario):
    world.scenario = ScenarioState()

def scenario():
    return world.scenario

And I use this all the time in my steps to refer to state from previous steps

1
2
3
4
5
6
7
8

@step(u'Given a user exists with one account')
def given_a_user_exists(step):
    scenario.current_user = UserFactory.create()

@step(u'And the user has a dog')
def user_has_a_dog(step):
    scenario.current_user.dog = DogFactory.create()

Hope you guys find this useful!