Hands On Guide to Unit Testing with Pytest and Django

14 min readJan 4, 2021

In the initial steps of my career, when I just started writing software, only the senior developers of my company felt responsible to write tests for their code. Over the years, this has become mainstream. But “mainstream” does not mean “universal”. Plenty of developers still do not have comfort with or even exposure to the unit testing practice. And yet, a form of peer pressure causes them to play that close to the vest.

So I reach out to these folks to say “Hey, no worries. You can learn and you don’t even have to climb too steep of a hill.” I’d like to revisit that approach here, today, in the form of a blog post.

Let’s get started with unit testing in Python using pytest, assuming that you know absolutely nothing about it. Although examples are beginner-friendly, this article is not going to be a tutorial on how to write tests with pytest. Its docs already make this. I will be more focused on the idea of testing in itself.

Intro to testing

Automated testing is an extremely useful bug-killing tool for the modern Web developer. (Django docs)

As a definition, automated testing or test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. In other words, it is a piece of software completely separate from your source code, responsible to make sure that the source code is working as expected.

What kind of problems do you solve or avoid when you write tests?

When you are writing new code, you can use tests to validate that your code works as expected.
When you are refactoring or modifying old code, you can use tests to ensure your changes haven’t affected your application’s behavior unexpectedly.

Why do we write code to test other code? Why aren’t we testing manually?

Let’s say you are done with the development of a new feature and ready to deliver it to your Quality Assurance team. Your QA team will spend some time to make sure that all the edge cases are covered and the feature is ready for production. After deploying to production, you are asked to modify or add something new on top of that feature. Then you make all the required changes to fulfill the new requirements and deliver them to your QA team.

Can you spot the problem above?

Now, the QA team is going to make the same manual tests as before, to make sure that modifications to the code didn’t break anything and all edge cases are still covered. It means time, lost time for the same tests, again and again.

What is the solution?

The solution is automating those tests. When you have automated tests, you can run them anytime you want (in the middle of the night), in any environment you want (on your development machine or a CI server) and be sure that you haven’t broken anything in your code, or even if you broke something, then you have a clear idea of what it is. In addition, automated tests are fast, faster than any QA team.

So, yes, we should make some automation on tests, but how? Unit tests can be a good start. They are easy to write and easy to maintain. So what is a unit test?

Unit tests

Unit testing is a software testing method by which individual units of source code are tested to determine whether they are fit for use. Unit tests take small pieces of source code and check if those pieces are operating as expected under given conditions. Unit tests find problems early in the development cycle. Finding bugs in the source code before the release is critical. The cost of finding a bug when the code is written is way lower than finding it later. Finding a bug is just the beginning: after you find a bug, you have to find what is causing it and how to solve it. Making all of those operations on a module that is already in production is very expensive in terms of time and very stressful for the developer. You need to be real quick, the code is in production so the bug is too and your users are facing this bug. You need to be sure that after fixing the problem you haven’t created some new ones. Unit tests are a perfect way to find those problems and tell you exactly where the problem occurs so you can skip the detection and identification of the bug and jump directly to solving it. Unit tests have a clear result. They are either passing (which means there is nothing to show but a success message) or they are failing (in which case the failure message will contain the exact check performed by the test that has failed). The result of a failing unit test contains a lot of information that can help you have a better idea of what is going wrong.

Real-world example with a demo pizza app

Now let’s jump into a real-world example and show how unit testing works. We are going to work on a demo pizza application. Below is a simplified diagram that describes the application.

So the diagram says that:

— Each topping and each size has its price

— Each pizza has some default toppings

— For each pizza order, we have to select a pizza, a size and extra toppings optionally

— Each pizza order is linked with a single order

— Finally, each order holds some general information about the order

We are going to focus on writing some unit tests for the price calculation.

How is the price calculated?

Each pizza has a base price, which is the sum of the prices of the default toppings
Each pizza order has a price, which is the sum of the following:
1. price of pizza
2. price of size
3. sum of the extra toppings
Finally, each order has a price, which is the sum of the pizza order prices

Let’s have a look at the Pizza model class which we are going to test.

Pizza Model

So we have some toppings, each of them with its price, and the sum of those prices gives us the total price of a Pizza instance. Let’s write some unit tests for this.

In the gist above we have two tests for the price calculation of a Pizza instance. They check the price calculation of a pizza with and without toppings. Let’s analyze them.

Django DB mark

So unit tests in pytest are not supposed to access the DB by default. To be able to access the DB, we use this mark from the pytest_django package.

2. Preparation phase

Here we create and link all required instances with each other, pizzas and toppings. We assign a value to all of the required attributes, especially to price.

3. Calculating results

We get the actual price by calling the total_price property of the Pizza instance. This property is going to make its calculation, and it is supposed to return a value at the end.

We get the expected price by summing up two toppings of the pizza instance.

4. Assertion

It’s time to compare the values: if the price we get from total_price property is the same as the expected_price the test will pass, otherwise it will fail.

We can run the tests by using the command pytest in a terminal window and here are the results:

So pytest started to search for all tests inside the project, collected 2 tests and ran them. The configuration for pytest is done by using an ini file called pytest.ini located at the root level of the project. Let's have a look at the configuration file.

DJANGO_SETTINGS_MODULE = pizza_app.test_settings

We have defined a different settings file for Django settings which will be used by pytest when getting Django up. This is a practice to speed up the testing process. So for example, if your model fields are supported by SQLite you can use it to make the DB a lot faster while testing. You can use a different password hasher and remove password validators to speed up the process of creating users. You can also change the DEBUG setting which is False by default when running tests to execute the tests in an environment as close as possible to production. You can view my test_settings file.

python_files = tests.py test_*.py *_tests.py

Here we have defined the naming convention of test files. So every file name that starts with test_ or ends with _tests or is exactly tests will be considered as a test file while discovering the tests by the discoverer.

addopts = — reuse-db — nomigrations

Here we are saying that after finishing with test execution, the test DB that was created should not be destroyed, in order to be reused. Furthermore, when creating the DB, pytest should not inspect the Django migrations, but instead go and check the models directly: this option can be faster when there are several migrations to apply for each app. Running pytest in the terminal will use the configurations provided in the .ini file. These configurations can be overridden when running the command, in the form

pytest --create-db

The reason you may want to use this command is that, when you have any changes to your database schema, you can run the tests once with this command and a new database will be created. After that, you can continue running your tests only with the pytest command so all ini configs will be used again.

Write clean tests

Some developers think it is OK for the testing code to be dirty and not compliant with all the conventions of your language, unlike your source code. I do not agree with that.

Let me describe to you the cost of writing dirty tests. At first, it will be all fine, you wrote some tests super fast, the code is dirty, but that doesn’t matter because they are just tests. After some time, the requirements change, so you have to change your tests too. Now you have two options. The first one is to spend the required time modifying the “dirty testing” code. The second one is to place a skip mark to those tests, because you are almost sure that your code is working and plan to deal with them later on. Even if you want to spend time now to fix broken tests, there will always be a tight deadline that’ll make you choose the second option. And trust me, if you skip those tests, there will never ever be free time to fix them.

So what is the cost? Time. Lost time.

Remember the reason of writing tests? By writing dirty tests, we actually wasted our time writing them and will waste time again by fixing or skipping them and facing bugs in production.

So, my suggestion is to always consider your testing code the same way you consider your source code. Write it as clean as you can.

Now let’s review our testing code. Is it clean? No.

Preparation phase

We are not interested at all in what values will any of the attributes have. Even price. We just want to create a pizza with some toppings. Imagine having to create an instance of a class with dozens of required fields. There must be an easier way.

Comes in: model-bakery. This is a must-have package if you want to write hundreds of tests in Django. This package is capable of creating instances of a class by providing only the class name. So it goes as follows:

Using model-bakery

It is as simple as that but can we go further? We are doing almost the same thing but without toppings in the second test too. So we can move the creation of a pizza in another module, let’s name it test_utils.py:

And the testing code:

Parametrization

The testing code is a little clearer now, but as you may have noticed, we have two tests that test the same thing: the price of a pizza. The only thing that changes is the number of toppings. To make it more simple, we can use one of the best features of pytest, which is parametrization. What we want to parametrize is the number of toppings, it will be given to the testing function as a parameter and pytest will run the test more than once by giving the next number of toppings every time.

@pytest.mark.parametrize(‘number_of_toppings’, list(range(0, 10)))

This line of code makes this test run 10 times, each time with more toppings. So because 0 number_of_toppings means a pizza without toppings, we do not need the second test anymore. This is the output of pytest -v command (-v: verbose, display more information):

Mocking certain operations

Now we have a general understanding of how to approach tests and how to make them simple and clean. Another problem we will be facing while testing our web application is costly operations. By costly operations, I mean some side effects of source code that are out of the testing scope but will be executed while running the tests. For example, let’s suppose that for every order created we need to print out a receipt. Furthermore, let’s suppose that the actual printing is done by a microservice, which we contact via an API, providing the order details. Naturally, we don’t want to print out anything while we are running our tests. This means that, while testing order creation, we need to find a way to skip this API call. The mechanism that provides what we want is called mocking.

There is a pytest way to achieve this. It’s called the monkeypatch: this fixture provides some helper methods for safely patching and mocking functionalities in tests.

Let’s assume we have the following code to save and print out the receipt.

What we want is to mock the post method of the requests module to do nothing instead of posting to the receipt microservice. We just want to test if the price is set while saving an order instance. The testing code:

Monkeypatch is a pytest fixture, so we can directly include it as a parameter in our testing function. After including it, this line does the work:

monkeypatch.setattr(requests, 'post', mock_post)

This command replaces the post method of the requests module with our mock_post function, in the scope of the test. Now, whenever (inside the test) requests.post is called, our mock function will be executed instead. If for some reason, I need the response of the post method, I can write a mock class, which will behave as the original response, i.e. implement its methods that we need.

Mock response class:

Test:

We can play around with the MockResponse class as we like to fulfill the requirements of the code that we are mocking.

Debugging

Our tests will not always pass. Sometimes they will fail, which is precisely the reason why we write them: to fail when they need to and tell us what is going wrong. After a test fails, it’s time to verify if we are doing something wrong in the testing code or if something is actually broken in our source code. To be able to do that, we need some more details about what is going on inside the application while the test is running. The source code that we are testing can be hard to understand and even tests can sometimes be hard to understand (although they shouldn’t be). To go deep and understand all the circumstances of a failing test, we need to debug it. Here are two ways to debug a test.

Using Pycharm

The first one is using a modern Integrated Development Environment, like PyCharm. First of all, to be able to use pytest support in PyCharm, you need to set the Default test runner setting to pytest. You can find this setting under Settings > Tools > Python Integrated Tools.

After indexing your project, PyCharm will show a green run icon besides each test:

When you click on this icon you see the following:

By clicking Debug you can run this single test in a debug session and the execution of the program will stop in the breakpoints you have placed around the code.

Dropping to Pdb

If you are already on a terminal window and want to see some variables and other things very quickly, you can also drop into Pdb by using the built-in function breakpoint. In your code:

In your terminal:

If you press TAB two times you can view all available options:

Some of the options are as follows:

next: will execute the next line
cont: will continue until the next breakpoint
list: will display the code that will be executed

Isolation

Test isolation is a topic worth writing an entire article about. I will treat it really briefly here, because if we are going to write some tests, we definitely need to keep this concept in mind.

Basically, test isolation says that all tests must be independent from each other and also from other modules of our project. A test function must not make any assumption of the application’s state. It must not depend on the actual state to be able to run. It shouldn’t matter for a test if it is running first or last; or if it is running alone or in a group of tests.

What is the point of all of that?

If a test depends on other tests, you are not guaranteed that it will always pass. If you are not sure about that, then you cannot trust the value this test produces: in other words, the test becomes unusable. For example, for our order_create test, we have created an order using the following util function:

Here we have not assumed that a pizza order is already created in the DB by some other tests. That’s why we haven’t retrieved any PizzaOrder instance, but we have directly created it. Every single thing that a test needs to be able to run must be created inside its body or through a database fixture. What the test needs to run must be obvious. If it is not, it will be much harder to debug it when failing, because we don't even know what the test is receiving as input. When you don't even know the exact input, you can't be checking for the output.

Final worlds on isolation, always make obvious what the tests need to run; tests should not depend on running order; pay attention to the assertion timing.

Flaky tests

A flaky test is a test that both passes and fails periodically without any code changes. Flaky tests are annoying but they can also be quite costly since they often require engineers to retrigger entire builds on CI and often waste a lot of time waiting for new builds to complete successfully. A flaky test is probably a result of a not well-isolated test. This means that it depends on some global variables which change over time. This kind of test provides you some false negatives. It is better to not have tests at all instead of having some tests that eventually fail without any code changes. This type of test will make you and your organization lose confidence in tests and turn back into manual verification. Here is an article about them, check out how do the Spotify engineers deal with them.

In conclusion

We have discussed some ways of writing clean and maintainable tests. Key topics treated:

Pytest, as an advanced testing framework that has countless features compared to the standard built-in unittest.
Ways to debug tests.
Isolation of tests and their importance.
Flaky tests, as the result of bad-isolated tests.

If you are going to work on complex applications, bugs will always be present. Testing is a bug-killing tool that can save you a lot of time and headache.

Thanks for reading!