Behavior-Driven Development is an excellent process to follow in software development. With testing often a practice that is pushed aside to the last minute (or ignored entirely), baking the process into your daily workflow can prove to be hugely beneficial to the quality of your code. The structure and design of the tests, coupled with the Gherkin syntax, makes tests easy to read—even for team members with non-technical backgrounds.
All code should be tested thoroughly, meaning that defects should ideally never reach production. If they do, then a thorough test suite, focused on the behavior of your application as a whole, ensures that they are easy to both detect and fix. This speed, clarity, focus and quality in your code is why you need to be adopting this process... now.
Behavior-Driven Development (which we will now refer to as "BDD") follows on from the ideas and principles introduced in Test-Driven Development. The key points of writing tests before code really apply to BDD as well. The idea is to not only test your code at the granular level with unit tests, but also test your application end to end, using acceptance tests. We will introduce this style of testing with the use of the Lettuce testing framework.
Behavior-Driven Development (BDD) is a subset of Test-Driven Development (TDD).
The process can be simply defined as:
Rinse and repeat for every feature, as is necessary.
BDD really comes into its own when used with agile development.
Tip: Refer to The Principles of Agile Development for more information on agile development methods.
With new features and requirements coming in every one, two or four weeks, depending on your team, you need to be able to test and write code for these demands quickly. Acceptance and unit testing in Python allows you to meet these goals.
Acceptance tests famously make use of an English (or possibly alternative) language format "feature" file, describing what the test is covering and the individual tests themselves. This can engage everyone in your team—not just the developers, but also management and business analysts who otherwise would play no part in the testing process. This can help to breed confidence across the whole team in what they are striving to achieve.
The feature files allow for tests to be described in a language this is/can be accessible to all levels of the business, and ensures that the features being delivered are articulated and tested in the way the business requires and expects. Unit testing alone can not ensure the application being delivered actually provides the full functionality that is required. Therefore acceptance testing adds another layer of confidence in your code to ensure that those individual 'units' fit together to deliver the full package required. The great thing about acceptance testing is that it can be applied to any project you are working on, either big or small scale.
Acceptance tests usually make use of the Gherkin Syntax, introduced by the Cucumber Framework, written for Ruby. The syntax is quite easy to understand, and, in the Lettuce Python package, makes use of the following eight keywords to define your features and tests:
Below, you can review these keywords in action, and how they can be used to structure your acceptance tests.
The installation of the
Lettuce package is straightforward, following the usual
pip install pattern that most Python developers will be familiar with.
Perform the following steps to begin using
$ pip install lettuce
$ lettuce /path/to/example.featureto run your tests. You can either run just one feature file, or, if you pass a directory of feature files, you can run all of them.
You should also install
nosetests (if you don't already have it installed) as you will be making use of some of the assertions that
nosetests provides to make your tests easier to write and use.
$ pip install nose
Feature files are written in plain English, and specify the area of the application that the tests cover. They also provide some setup tasks for the tests. This means that you are not only writing your tests, but are actually forcing yourself to write good documentation for all aspects of your application. So you can clearly define what each piece of code is doing and what it is handling. This documentation aspect of the tests can be great as the size of your application grows, and you wish to review how a certain aspect of the application works, or you wish to remind yourself of how to interact with a part of the API for example.
Let's create a Feature file which will test an application that was written for my Test-Driven Development in Python article for Tuts+. The application is just a simple calculator written in Python but will show us the basic of writing acceptance tests. You should structure your application with an
app and a
tests folder. Within the
tests folder, add a
features folder also. Place the following code in a file named
calculator.py under the
class Calculator(object): def add(self, x, y): number_types = (int, long, float, complex) if isinstance(x, number_types) and isinstance(y, number_types): return x + y else: raise ValueError
Now add the following code to a file named
calculator.feature under the
Feature: As a writer for NetTuts I wish to demonstrate How easy writing Acceptance Tests In Python really is. Background: Given I am using the calculator Scenario: Calculate 2 plus 2 on our calculator Given I input "2" add "2" Then I should see "4"
From this simple example, you can see how straightforward it is to describe your tests and share them across the various people involved in your team.
There are three key areas of note in the feature file:
SetUp()method and allows you to perform necessary setup code, such as making sure you are on some page, or have certain conditions in place.
Following on from the Feature file, we need to have the steps file underneath. This is where the 'magic' happens. Obviously, the Feature file itself will not do anything; it requires the steps to actually map each line to execute Python code underneath. This is achieved through the use of regular expressions.
"Regular Expressions? Too complex to bother with in testing" can often be a response to RegEx's in these tests. However, in the BDD world, they are used to capture the whole string or use very simple RegEx's to pick out variables from a line. Therefore you shouldn't be put off by their use here.
Regular Expressions? Too complex to bother with in testing? Not in Lettuce. Simple and Easy!
If we review an example. you'll see how easily the Steps file follows on from the Feature.
from lettuce import * from nose.tools import assert_equals from app.calculator import Calculator @step(u'I am using the calculator') def select_calc(step): print ('Attempting to use calculator...') world.calc = Calculator() @step(u'I input "([^"]*)" add "([^"]*)"') def given_i_input_group1_add_group1(step, x, y): world.result = world.calc.add(int(x), int(y)) @step(u'I should see "([^"]+)"') def result(step, expected_result): actual_result = world.result assert_equals(int(expected_result), actual_result)
The first thing worth noting is the standard imports at the top of the file. So we need access to our
Calculator class and, of course, the tools provided by Lettuce. You also import some handy methods from the
nosetest package such as
assert_equals to allow for easy assertions in the steps. You can then begin to define the steps for each line in the Feature file. We can see that, as explained earlier, the regular expressions are mostly just picking up the whole string, except where we want access to the variable within the line.
If we use the
@step(u'I input "([^"]*)" add "([^"]*)"') line as our example, you can see that the line is first picked up using the
@step decorator. Then you use the
'u' character at the start to indicate a unicode string for Lettuce to perform regular expressions upon. Following that, it's just the line itself and a very simple regular expression to match anything within the quotes—the numbers to add in this case.
You should then see that the Python method follows directly after this, with the variables passed into the method with whatever name you wish. Here, I have called them
y to indicate the two numbers to be passed to the calculator
Another item of note here is the use of the
world variable. This is a globally scoped container, and allows variables to be used across steps within a scenario. If we didn't, all variables would be local to their method, but, here, we create an instance of
Calculator() once, and then access that in each step. You also use the same technique to store the result of the
add method in one step and then assert on the result in another step.
With the feature file and steps in place, you can now execute the tests and see if they pass. As mentioned earlier, executing the tests is simple and Lettuce provides a built-in test runner, available to you from the command line after installation. Try executing
lettuce test/features/calculator.feature in your preferred command line application.
$ lettuce tests/features/calculator.feature Feature: As a writer for NetTuts # tests/features/calculator.feature:1 I wish to demonstrate # tests/features/calculator.feature:2 How easy writing Acceptance Tests # tests/features/calculator.feature:3 In Python really is. # tests/features/calculator.feature:4 Background: Given I am using the calculator # tests/features/steps.py:6 Given I am using the calculator # tests/features/steps.py:6 Scenario: Calculate 2 plus 2 on our calculator # tests/features/calculator.feature:9 Given I input "2" add "2" # tests/features/steps.py:11 Then I should see "4" # tests/features/steps.py:16 1 feature (1 passed) 1 scenario (1 passed) 2 steps (2 passed)
Lettuce's output is really nice, as it shows you each line of the feature file that has been executed and highlights in green to show that it has passed the line successfully. It also shows which feature file it is running and the line number, which comes in handy once you have built up a larger test suite of numerous features and need to find an offending line of a feature such as when a test fails. Finally, the last part of the output provides you with stats about the number of features, scenarios and steps that have been executed, and how many passed. In our example, all the tests were good, but let's take a look at how Lettuce shows you test failures and how you can debug and fix them.
Make a change to the code of
calculator.py, so that the test will fail, such as changing the add method to actually subtract the two numbers passed in.
class Calculator(object): def add(self, x, y): number_types = (int, long, float, complex) if isinstance(x, number_types) and isinstance(y, number_types): return x - y else: raise ValueError
Now, when you run the feature file using Lettuce, you will see how it clearly indicates what has gone wrong in the test and which part of the code has failed.
$ lettuce tests/features/calculator.feature Feature: As a writer for NetTuts # tests/features/calculator.feature:1 I wish to demonstrate # tests/features/calculator.feature:2 How easy writing Acceptance Tests # tests/features/calculator.feature:3 In Python really is. # tests/features/calculator.feature:4 Background: Given I am using the calculator # tests/features/steps.py:6 Given I am using the calculator # tests/features/steps.py:6 Scenario: Calculate 2 plus 2 on our calculator # tests/features/calculator.feature:9 Given I input "2" add "2" # tests/features/steps.py:11 Then I should see "4" # tests/features/steps.py:16 Traceback (most recent call last): File "/Users/user/.virtualenvs/bdd-in-python/lib/python2.7/site-packages/lettuce/core.py", line 144, in __call__ ret = self.function(self.step, *args, **kw) File "/Users/user/Documents/Articles - NetTuts/BDD_in_Python/tests/features/steps.py", line 18, in result assert_equals(int(expected_result), actual_result) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 515, in assertEqual assertion_func(first, second, msg=msg) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual raise self.failureException(msg) AssertionError: 4 != 0 1 feature (0 passed) 1 scenario (0 passed) 2 steps (1 failed, 1 passed) List of failed scenarios: Scenario: Calculate 2 plus 2 on our calculator # tests/features/calculator.feature:9
Clearly, the expected value of
4 now does not match the actual return value of
0. Lettuce has clearly shown you this issue and you could then debug your code to find out what has gone wrong, apply a fix, and make the test pass again.
There are plenty of alternative options within Python to do this form of testing. We have examples, such as Behave, Lettuce and also Cucumber, which, as mentioned, defined this structure. The other tools are essentially clones/ports of Cucumber. Cucumber can be used with Python code, via the use of a Ruby-Python interpreter, but that is beyond the scope of this tutorial.
The key point, with all of these tools, is that they are all more or less the same. Once you have mastered one, you'll quickly pick up on the others, should you choose to switch. A quick review of the documentation should be enough for most developers who are proficient in Python.
You can dive into refactoring confidently.
There are significant advantages to using a thorough test suite. One of the major ones revolves around the refactoring of code. With a robust test suite in place, you can dive into refactoring confidently, knowing that you have not broken any previous behavior in your application.
This grows in importance the more your application develops and increase in size. When you have more and more legacy code, it becomes very hard to go back and make changes with confidence and know that you have definitely not broken any existing behaviour. If you have a full suite of acceptance tests written for every feature being developed, you know that you have not broken that existing functionality as long as when you make your changes you run a full build of your tests before pushing the changes live. You check that your code has not "regressed" due to your changes and restrings.
Another great benefit of building acceptance testing into your daily workflow is the ability to have a clarification session before starting the development of a feature.
You could, for example, have the developers who will code the solution of a feature, the testers (quality assurance/QA's) who test the code once complete, and the business/technical analyst all sit down and clarify the requirements of a feature, and then document this as the feature files that the developers will work towards.
Essentially, you can have a set of failing feature files that the developers can run and make pass one by one, so that they will know they are done with the feature once all are passing. This gives developers the focus they need to deliver exactly to the requirements and not expand upon the code with features and functionality not necessarily required (also known as "gold plating"). The testers can then review the features files to see if everything is covered appropriately. The process can then be undertaken for the next feature.
Having worked in a team using the process and tools outlined above, I have personally experienced the huge advantages of working in this manner. BDD provides your team with clarity, focus, and the confidence to deliver great code, while keeping any potential bugs to a minimum.
If this article has whetted your appetite for the world of testing in Python, why not check out my book "Testing Python", released on Amazon and other good retailers recently. Visit this page to purchase your copy of the book today, and support one of your Tuts+ contributors.