What is Unit Testing?

Unit test­ing is a fun­da­mental aspect of soft­ware devel­op­ment, crit­ical for veri­fy­ing the integ­rity of an application’s indi­vidual com­pon­ents. It con­sists of isol­at­ing units, such as func­tions, meth­ods, or classes, and sub­ject­ing them to auto­mated tests to ensure they func­tion inde­pend­ently. In this test­ing pro­cess, units are eval­u­ated sep­ar­ately, often with the sup­port of stubs, mocks, or fakes to mimic depend­en­cies. This prac­tice ensures that each unit per­forms as expec­ted, unaf­fected by external ele­ments of the application.

Aims and benefits

The primary goal of unit test­ing is to check that each unit oper­ates cor­rectly accord­ing to the spe­cific­a­tions. It is cru­cial for code modi­fic­a­tion, catch­ing regres­sions so that exist­ing func­tion­al­ity remains unaf­fected by changes. It also helps to reduce the num­ber of bugs dur­ing the integ­ra­tion of the units and can provide examples of how the code is meant to be used.

Unit test­ing provides imme­di­ate feed­back on design issues, sig­nal­ing when a piece of code is poten­tially dif­fi­cult to test and may need refact­or­ing, i.e. using depend­ency injec­tion. Bugs detec­ted dur­ing unit test­ing are gen­er­ally less costly to fix than those found in later stages. It is also a crit­ical com­pon­ent of Con­tinu­ous Integ­ra­tion (CI) and Con­tinu­ous Deliv­ery (CD) practices.

Unit­test vs. Pytest

When developers face the decision of select­ing a unit test­ing frame­work in Python, key play­ers are unittest and pytest. The choice hinges on vari­ous factors includ­ing the desired test struc­ture, asser­tion style, setup pro­ced­ures, and dis­cov­ery meth­ods. Also the avail­ab­il­ity of external com­pon­ents in the tar­get envir­on­ment comes into play, espe­cially when work­ing with leg­acy code in closed systems.

unittest is the tra­di­tional choice, embed­ded within Python’s stand­ard lib­rary, offer­ing a famil­iar object-ori­ented approach. Tests are neatly organ­ized into meth­ods that reside within classes derived from unittest.TestCase. This frame­work provides a suite of asser­tion meth­ods tailored for vari­ous scen­arios, along­side setUp and tearDown meth­ods that are invoked before and after each test, respect­ively. How­ever, this comes at the cost of verb­os­ity and a some­what rigid test dis­cov­ery pro­cess. Since ver­sion 3.3, the mock pack­age was integ­rated into the stand­ard lib­rary as unittest.mock. This offers rather com­plex, but very power­ful mock­ing and stub­bing functionalities.

In con­trast, pytest presents a mod­ern altern­at­ive, known for its sim­pli­city and eleg­ance. It breaks free from the class-based struc­ture, enabling developers to write tests as plain func­tions. Asser­tions are sim­pli­fied, lever­aging the stand­ard assert state­ment, mak­ing the tests easier to write and under­stand. The setup and cleanup are handled by fix­tures that offer more flex­ible and power­ful scop­ing options. While pytest is not part of the stand­ard lib­rary and thus requires an addi­tional install­a­tion, it com­pensates with a highly intu­it­ive test dis­cov­ery sys­tem and extens­ive plu­gin sup­port, fos­ter­ing a dynamic and extens­ible test­ing envir­on­ment. Mock­ing is typ­ic­ally done with mon­keypatch­ing, a very straight-for­ward mech­an­ism built into pytest (see below).

Why pytest is the bet­ter option

Typ­ic­ally, a pytest test case will be sig­ni­fic­antly shorter in terms of code amount than a unit­test test case with the same func­tion­al­ity. There can be cases, when the stand­ard pytest tool­box will be insuf­fi­ciently in func­tion­al­ity and advanced meth­ods such as unittest.mock is needed. Luck­ily, pytest provides a plu­gin to util­ize these dir­ectly from within pytest. Fur­ther­more, pytest will run tests on exist­ing unit­test-style test sets. This makes a trans­ition from exist­ing unittest code to pytest very straight-forward.

Each frame­work has its own set of trade-offs, and the selec­tion often boils down to the spe­cific require­ments of the pro­ject and per­sonal pref­er­ence. unittest appeals to those seek­ing the con­sist­ency and avail­ab­il­ity of a stand­ard lib­rary tool, while pytest attracts those look­ing for more con­cise code and advanced features.

Cov­er­age: Assess­ing Test Effectiveness

Cov­er­age meas­ures how much of the code­base the tests cover. It shines a light on untested parts of the code, encour­ages com­plete test­ing, and sets goals for cov­er­age met­rics. By high­light­ing what code is not tested, it indic­ates where more test­ing might be needed and helps main­tain the health of the code by point­ing out unused or redund­ant seg­ments. Though a 100 % code cov­er­age should be the goal of unit tests, one must be care­ful with the inter­pret­a­tion. Full code cov­er­age does not mean a full cov­er­age of all advis­able test cases.

Cov­er­age reports are pro­duced with coverage python pack­age. Luck­ily, there is a pytest plu­gin pytest-cov, that provides the full func­tion­al­ity of the pack­age from within pytest. This way, developers can test and determ­ine the cov­er­age in one go. There are dif­fer­ent report­ing styles, from simple com­mand line inter­face out­put to detailed visu­al­iz­a­tion as html doc­u­ments. An example of the lat­ter can be seen below, show­ing the covered and uncovered lines in green and red, respect­ively, as well as the total cov­er­age of the file.

Shows the html represenation of a coverage report
HTML based visu­al­ized cevo­er­age report of code.

Enhan­cing Test­ing with Fixtures

Pytest intro­duces fix­tures for effi­cient setup and tear­down in tests. Fix­tures pro­mote reusab­il­ity and allow for dif­fer­ent scopes to con­trol how often they are executed. They also sup­port para­met­er­iz­a­tion to test with vari­ous inputs eas­ily and offer lazy eval­u­ation, only run­ning when required. Depend­ency injec­tion through fix­tures sim­pli­fies tests by provid­ing resources dir­ectly, and the frame­work includes a mix of built-in and cus­tom fix­tures for broad applicability.

Fix­ture may invoke other fix­tures and by defin­ing fix­tures with the yield keyword (rather than return), one can imple­ment tear­down code after the fix­ture has done its job. Here is an example of a fix­ture that returns a data­base con­nec­tion and requests ses­sion, clos­ing the ses­sion afterwards:

Code ecaple of a pytest fixture.
The fix­ture provides a db con­nec­tion and a requests ses­sion, as well as a tear­down mechanism

The fix­ture can be used in a test simply by provid­ing it as a para­meter to the test function.

Mon­keypatch­ing: Tem­por­ary Code Adjustment

Mon­keypatch­ing in Pytest allows tem­por­ary, dynamic changes to the code dur­ing runtime, which are reversed once the test con­cludes. This ensures test isol­a­tion and min­im­izes the risk of side effects across tests. It’s a built-in fea­ture that is easy to use and does not require external lib­rar­ies, offer­ing flex­ib­il­ity to alter meth­ods, func­tions, attrib­utes, or envir­on­ment vari­ables just for the dur­a­tion of a test.

A mon­keypatch can be used in a test by passing the mon­keytest fix­ture to the func­tion. From there, all meth­ods to patch pro­gram parts are avail­able. The example below patches the rand­int method of the radom pack­age, because test­ing is not very reli­able when the pro­gram uses ran­dom num­bers. The func­tion is replaced with a lambda func­tion to always return the integer 5.

A monkepatch code example
The mon­keypatch replaces the rand­int method with a lambda function

Final Thoughts

Pytest stands out for its sim­pler syn­tax that reduces boil­er­plate, advanced fix­ture man­age­ment, and easy test dis­cov­ery that doesn’t require expli­cit con­fig­ur­a­tion. Para­met­er­iz­a­tion is straight­for­ward, provid­ing the abil­ity to run tests with mul­tiple data sets. Asser­tions are inform­at­ive, giv­ing clear feed­back for debug­ging. The eco­sys­tem of plu­gins for pytest allows for extens­ive extens­ib­il­ity. Addi­tion­ally, pytest can run unittest tests, mak­ing integ­ra­tion smoother.

How­ever, pytest is an external resource, and in some scen­arios, cli­ents or stake­hold­ers might prefer or require the use of the stand­ard library’s unittest due to its built-in nature. This depend­ency on an external tool is some­times a con­sid­er­a­tion when choos­ing a test­ing framework.