Siv Scripts

Solving Problems Using Code

Sat 28 July 2018

Lock Your Mock

Posted by Aly Sivji in Quick Hits   

Python 3.7 was released a few weeks back. Lots of new features in this minor release: data classes, new breakpoint keyword, dicts preserving insertion order, amongst others.

One feature that hasn't got as much press, or really any press, is an update to the unittest.mock module in the Standard Library. Starting in Python 3.7, instances of Mock and MagicMock can be sealed, preventing the creation of attribute mocks.

In this quick hit, we will explore mock.seal.

Test Doubles

Test doubles refer to objects which are used to replace production objects for testing purposes.

When our program is required to reach outside of our function, class, or module to interact with the outside world, we have to account for this external dependency. This adds an additional layer of complexity to our testing process.

We can create test doubles with predefined behaviors and replace them with production objects to make our tests deterministic.

In the following example, we have a function that hits a REST API endpoint to get historical stock market data. Next, we iterate thru the data to find the day with the largest intraday spread (intraday_spread = daily_high - daily_low) for a given ticker.

from collections import defaultdict
from decimal import Decimal
import json

import requests

def calculate_max_spread(stock):
    """Find day with max spread (daily hi - daily lo) for a stock ticker"""

    url = ""
    params = {
        "function": "TIME_SERIES_DAILY",
        "symbol": "MSFT",  # demo key works for MSFT only
        "apikey": "demo"

    r = requests.get(url, params=params)

    if r.status_code != 200:
        raise Exception

    daily_stock_prices = r.json()["Time Series (Daily)"]
    daily_spreads = []
    spread_by_date = defaultdict(list)

    for date, stock_data in daily_stock_prices.items():
        hi = Decimal(stock_data["2. high"])
        lo = Decimal(stock_data["3. low"])
        spread = float(hi - lo)
        spread_by_date[spread] = date

    max_spread = max(daily_spreads)

    return (spread_by_date[max_spread], max_spread)

This code has a dependency on an external data source. If the 3rd party website goes down, our function will raise an Exception. This will result in test failures where errors could be caused either by our code or the status of a third party website. We can't be sure unless we dig into the failure.

To test this code, we can replace the external APIs with mock objects we create. Specifically, we need to patch requests.get to return deterministic test data. I explored this pattern in a previous blog post on pytest fixtures.


In Python, the distinction between test doubles is blurred as the unittest.mock framework provides a robust implementation of test doubles we can use for different types of test verification.

From the Standard Library Documentation:

MagicMock objects create all attributes and methods as you access them and store details of how they have been used. You can configure them, to specify return values or limit what attributes are available, and then make assertions about how they have been used

Let's explore this in more detail.

In [1]:
from unittest.mock import MagicMock
In [2]:
my_mock = MagicMock(name="my_mock")
<MagicMock name='my_mock' id='4414002624'>

We can add attributes and functions to our my_mock object; this will return a new MagicMock.

In [3]:
<MagicMock name='my_mock.new_attribute' id='4413299120'>
In [4]:
my_mock.new_function.return_value = 2
<MagicMock name='my_mock.new_function' id='4414186048'>
In [5]:

Notice the id of the above objects are different. This means they different objects.

Mocks are used for behavior verficiation testing, i.e. we can check that our mock was used how we think it was used. Let's confirm our new_function mock was called:

In [6]:
In [7]:
# Call new_function once more
In [8]:
# Should raise `AssertionError`
AssertionError                            Traceback (most recent call last)
<ipython-input-8-9e7a18f695e0> in <module>()
      1 # Should raise `AssertionError`
----> 2 my_mock.new_function.assert_called_once()

~/.pyenv/versions/3.7.0/lib/python3.7/unittest/ in assert_called_once(_mock_self)
    799             msg = ("Expected '%s' to have been called once. Called %s times." %
    800                    (self._mock_name or 'mock', self.call_count))
--> 801             raise AssertionError(msg)
    803     def assert_called_with(_mock_self, *args, **kwargs):

AssertionError: Expected 'new_function' to have been called once. Called 2 times.

As we can create attributes on our mocks on demand, there is a possibility that our tests can pass with typos in our assert method.

In [9]:
# notice the spelling error: asert vs assert
<MagicMock name='my_mock.new_function.asert_called_once()' id='4414874680'>

Because of the flexibility of MagicMock, i.e. it returns a truthy value instead of raising an AssertionError.

This can cause tests to pass when they should fail.


In Python 3.7, we can strictly define mock attributes:

The new seal() function allows sealing Mock instances, which will disallow further creation of attribute mocks. The seal is applied recursively to all attributes that are themselves mocks. (Contributed by Mario Corchero in bpo-30541.)

In [10]:
import sys
from unittest import mock
In [11]:
3.7.0 (default, Jun 28 2018, 13:41:41) 
[Clang 8.0.0 (clang-800.0.42.1)]
In [12]:
sealed_mock = MagicMock(name="sealed_mock")
<MagicMock name='sealed_mock' id='4414900376'>
In [13]:
sealed_mock.only_function.return_value = 2
<MagicMock name='sealed_mock.only_function' id='4414917208'>

Now let's seal the mock:

In [14]:
In [15]:
# add new attribute results in error
AttributeError                            Traceback (most recent call last)
<ipython-input-15-8431f05a1839> in <module>()
      1 # add new attribute results in error
----> 2 sealed_mock.new_function

~/.pyenv/versions/3.7.0/lib/python3.7/unittest/ in __getattr__(self, name)
    597             result = self._get_child_mock(
    598                 parent=self, name=name, wraps=wraps, _new_name=name,
--> 599                 _new_parent=self
    600             )
    601             self._mock_children[name]  = result

~/.pyenv/versions/3.7.0/lib/python3.7/unittest/ in _get_child_mock(self, **kw)
    903             attribute = "." + kw["name"] if "name" in kw else "()"
    904             mock_name = self._extract_mock_name() + attribute
--> 905             raise AttributeError(mock_name)
    907         return klass(**kw)

AttributeError: sealed_mock.new_function

Sealing our mock prevents the creation of unwanted attributes.

We could also autospec mocks to create objects that have the same attributes as the production objects we are replacing. See: mock.create_autospec, mock.patch, mock.patch.object.

mock.seal offers additional flexibility to spec out minimal interfaces for testing.

Patching External API with Mock

Going back to our stock example from above, we can test our module by monkeypatching the requests library with a mock. This mock returns known data which we can use to write deterministic tests.

I am currently working on a series of testing blog posts that will go into this in more detail. For now, I will leave y'all with a snippet that shows how I would go about testing.

def json_loader(filename):
    """Loads data from JSON file"""
    with open(filename, 'r') as f:
        data = json.load(f)
    return data

def test_calculate_max_spread():
    # Arrange
    get_mock = mock.MagicMock()
    get_mock.status_code = 200
    get_mock.json.return_value = json_loader("stock_data.json")

    # Act
    with mock.patch.object(requests, 'get', return_value=get_mock):
        response = calculate_max_spread("MSFT")

    # Assert
    assert response == ('2018-03-27', 6.629)