Python’s unittest.mock

Along with factories, parametrize and fixtures, one more thing that you’ll see a lot of in Hypothesis’s Python tests that you probably won’t know from non-test Python code are these things called mocks, which we’ll look at in this post.

Mock

“Mock objects” are “are simulated objects that mimic the behavior of real objects in controlled ways”. Sometimes the function or method that you’re trying to test requires an object from another class, but you don’t want to use a real object of that other class in your test. For example you could be writing a test for the foo() function and it requires a Bar object as argument:

def foo(bar):
    ...

Your test will need a Bar object to pass foo(), but there can be several reasons why you wouldn’t want to use a real Bar object in your test:

Instead of using the real Bar class you can create a MockBar class for the tests to use:

def foo(bar):
    ...

# Tests:

class MockBar(object):
    ...

def test_foo():
    mock_bar = MockBar()

    foo(mock_bar)

    assert ...

The MockBar class would simulate the methods of the real Bar class - MockBar would have the same methods, they’d take the same arguments, and they’d return whatever values the test wants them to return to foo() (for example, these might just be fake methods that return hard-coded values that look like what the real Bar class might return).

But MockBar objects would be easy for tests to create, fast, have no real-world side effects, and not break if the real Bar class has bugs. MockBar could also keep a record of what foo() does with it - what methods it calls and what arguments it passes to them. The test’s assert statements can query this record to test things about how foo() uses MockBar.

For more on mocks see wikipedia, Martin Fowler and wiki.c2.com.

Many programming languages come with a way of doing mock objects, and the mock library is Python’s way (in Python 3.3+ the mock library became part of the standard library).

The mock library is all about a single core class - MagicMock - that has several unusual properties. Instead of writing a lot of different mock classes like MockBar above, you can just use the one MagicMock class to easily replace objects of any class that your tests need, without having to write a lot of mock code.

Note: The mock library actually has two very similar classes - Mock and MagicMock. The difference is that MagicMock supports Python’s magic methods whereas Mock doesn’t. This usually isn’t important, but as the mock user guide says it’s sensible to use MagicMock by default. The Hypothesis tests tend to use Mock more often, though.

Note: Some people make distinctions between different types of replacement objects used by tests, for example Sinon.js has separate classes for fakes, spies, stubs, and mocks. Python’s Mock and MagicMock are capable of playing all of these roles and in this tutorial we’re just going to use the one word mock for everything.

You’ll find Mock and MagicMock used a lot in the Hypothesis tests - to create mock objects to pass in to methods under test as arguments, and in many other ways too. This post will demonstrate and explain the features of Mock / MagicMock so that you can understand what the tests are doing, and use mocks yourself. As usual we won’t cover everything but will concentrate on the features commonly used in the Hypothesis tests. You can go to the mock website for the rest.

You can access any attribute on a MagicMock

Normally in Python you can only access a given attribute name on an object if that object has an attribute with that name, otherwise you’ll get an AttributeError. For example here the foo object has no attribute named bar so trying to do foo.bar raises AttributeError:

>>> class Foo(object):
...     pass

>>> foo = Foo()
>>> foo.bar
Traceback (most recent call last):
  ...
AttributeError: 'Foo' object has no attribute 'bar'

MagicMock objects are different, you can access any attribute name on a mock object and it will always return another mock object:

>>> import mock
>>> my_mock = mock.MagicMock()
>>> my_mock.foo
<MagicMock name='mock.foo' id='139654612385424'>
>>> my_mock.bar
<MagicMock name='mock.bar' id='139654612362320'>
>>> my_mock.whatever_name_you_want
<MagicMock name='mock.whatever_name_you_want' id='139654612287632'>

By default a MagicMock never raises AttributeError. This property of mock objects allows tests to pass them in to the code under test instead of the real objects that would be used in production, and have the code under test still work. For example here’s a simple test for one behavior of an event queue:

def test_push_appends_event_to_queue(self):
    event_queue = EventQueue()
    event = mock.MagicMock()

    event_queue.push(event)

    assert list(event_queue.queue) == [event]

We push() a mock object onto event_queue instead of the real event object that it’s expecting. Even if the EventQueue code accesses some attributes of the mock object it won’t crash since you can read any attribute of a mock. And then we’re still able to test what we wanted to test - that it appends the mock object we gave it to the queue: assert list(event_queue.queue) == [event].

Each attribute returns a different other MagicMock object

In the example above you can tell from the different MagicMock ids that each attribute that is accessed - foo, bar or whatever_name_you_want - returns a different mock object:

>>> my_mock = mock.MagicMock()
>>> my_mock.foo
<MagicMock name='mock.foo' id='139654612385424'>
>>> my_mock.bar
<MagicMock name='mock.bar' id='139654612362320'>
>>> my_mock.whatever_name_you_want
<MagicMock name='mock.whatever_name_you_want' id='139654612287632'>

Two MagicMock objects with different ids are considered unequal:

>>> my_mock.foo == my_mock.bar
False

The same attribute always returns the same other MagicMock

Each different attribute of a MagicMock object returns a different other MagicMock object. On the other hand, if you access the same attribute of a MagicMock multiple times it will always return the same other MagicMock object with the same id:

>>> my_mock.foo
<MagicMock name='mock.foo' id='139654612385424'>
>>> my_mock.foo
<MagicMock name='mock.foo' id='139654612385424'>

Two MagicMock objects with the same id are considered equal:

>>> my_mock.foo == my_mock.foo
True

Note: The same attribute name on two different MagicMocks will return two different other MagicMocks though:

>>> my_mock = mock.MagicMock()
>>> my_other_mock = mock.MagicMock()
>>> my_mock.foo == my_other_mock.foo
False

Tests can make use of this property of mock objects because a given attribute of a mock will return the same other mock both when accessed by the code under test and when accessed by the test code itself, but if the code and the test access different attributes they’ll get different other mock objects.

For example lets look at a test for a simple presenter class. AnnotationSearchIndexPresenter is a class that takes an annotation object and provides an asdict() method that returns a dictionary representation of that annotation suitable for storing in our Elasticsearch index. A simplified version of the class might look something like this:

class AnnotationSearchIndexPresenter(object):

    def __init__(self, annotation):
        self.annotation = annotation

    def asdict(self):
        """Return a search-indexable dictionary representation of self.annotation."""
        return {
            'target': [
                {
                    'scope': self.annotation.target_uri_normalized,
                    ...
                },
                ...
            ],
            ...
        }

AnnotationSearchIndexPresenter translates and transforms attributes of the annotation object into a dictionary with a different format and structure - the format and structure required by Hypothesis’s search index. For example annotation.target_uri_normalized becomes the first item in a ['target'][0]['scope'] list in the dict that asdict() returns (don’t ask why).

A simple test for this could work by passing a mock object to AnnotationSearchIndexPresenter:

def test_it_copies_target_uri_normalized_to_target_scope(self):
    annotation = mock.MagicMock()

    annotation_dict = AnnotationSearchIndexPresenter(annotation).asdict()

    assert annotation_dict['target'][0]['scope'] == [annotation.target_uri_normalized]

The test passes because the first time our mock annotation’s annotation.target_uri_normalized attribute is accessed (by the AnnotationSearchIndexPresenter.asdict() code when the test calls it) it returns the same other mock object as the second time it’s accessed (by the assert statement in the test itself). The assert statement relies on this property of mocks to pass. If AnnotationSearchIndexPresenter had used some other attribute of the mock annotation, say annotation.target_uri instead of annotation.target_uri_normalized, that would have returned a different other mock object and the assert would have failed.

You can also set any attribute on a MagicMock

If you want an attribute of a MagicMock to have some value other than being another MagicMock you can just set it as you could do with a normal Python object:

>>> my_mock.foo = 23
>>> my_mock.foo
23

Tests can use this when they need the value of a mock object to be some other type of object than a mock. For example Annotation.created is a datetime object and AnnotationSearchIndexPresenter turns it into a string (maybe by calling strftime). To test that AnnotationSearchIndexPresenter formats this string correctly we really want our mock annotation.created to be a datetime not another mock:

import datetime

def test_it_formats_created_into_a_string():
    annotation = mock.MagicMock()
    annotation.created = datetime.datetime(2016, 2, 24, 18, 3, 25, 768)

    annotation_dict = AnnotationSearchIndexPresenter(annotation).asdict()

    assert annotation_dict['created'] == '2016-02-24T18:03:25.000768+00:00'

Of course it’s also possible for the code under test to set an attribute of a mock object to a value, and then a test might assert that the mock’s attribute was set to the expected value:

def set_foo_to_bar(thing):
    thing.foo = 'bar'

def test_it_sets_foo_to_bar():
    thing = mock.MagicMock()

    set_foo_to_bar(thing)

    assert thing.foo == 'bar'

Code setting attributes of its dependencies is often not a great design, but this does come up sometimes.

You can pass any keyword args to the MagicMock constructor

Unlike a typical Python object, you can pass any keyword argument to the MagicMock constructor and its value will be used as the value of the attribute on the MagicMock instead of returning another MagicMock:

>>> my_mock = mock.MagicMock(foo=23, bar=True)
>>> my_mock.foo
23
>>> my_mock.bar
True
>>> my_mock.something_else
<MagicMock name='mock.something_else' id='139654612225680'>

For example instead of creating the mock and then setting the attribute on it, the test above could have created the mock with created already set:

annotation = mock.MagicMock(created=datetime.datetime(2016, 2, 24, 18, 3, 25, 768))

Of course multipe attributes can be set at once as well:

annotation = mock.MagicMock(
    created=datetime.datetime(2016, 2, 24, 18, 3, 25, 768),
    target_uri_normalized='http://example.com/normalized',
    foo=True,
    bar=23,
    ...
)

MagicMocks are callable

You can also call a MagicMock object as if it were a method or function, and by default it’ll return another MagicMock object just like accessing an attribute does:

>>> my_mock = mock.MagicMock()
>>> my_mock()
<MagicMock name='mock()' id='140404522909008'>

This means that your tests can use MagicMocks not only to replace objects that the code under test uses, but also to replace functions and methods. When the code under test tries to call a MagicMock instead of the real function or method it’ll work, and it will just get another MagicMock back as the return value.

As with accessing attributes, by default the same MagicMock always returns the same other MagicMock when called but two different MagicMocks return two different other MagicMocks:

>>> my_mock = mock.MagicMock()
>>> my_mock() == my_mock()
True
>>> my_other_mock = mock.MagicMock()
>>> my_mock() == my_other_mock()
False

You can pass any arguments or keyword arguments when calling a mock. It always return the same other mock, regardless of what arguments it was called with:

>>> my_mock(True)
<MagicMock name='mock()' id='140404522909008'>
>>> my_mock(1, 2, 3)
<MagicMock name='mock()' id='140404522909008'>
>>> my_mock("foo", bar=[27, False])
<MagicMock name='mock()' id='140404522909008'>

Note: A mock returns a different mock object when called, it doesn’t return itself:

>>> my_mock() == my_mock
False

The other mock that it returns is accessible as the special return_value attribute:

>>> my_mock() == my_mock.return_value
True

One simple use of the callability of MagicMock is to test that a function returns the return value of another function that it depends on. In this code the bar() function is passed in to the foo() function as an argument, foo() does a bunch of important stuff (perhaps to calculate the arguments it’ll pass to bar()) and then returns the result of calling bar():

def foo(bar):
    ...
    return bar(arg1, arg2, ...)

Here’s a simple test for that last part of foo()‘s behavior, the return:

def test_foo_returns_what_bar_returned():
    bar = mock.MagicMock()

    returned = foo(bar)

    assert returned == bar.return_value

Tip: We could have written assert returned == bar() and it would have worked the same. But as we’ll see below, MagicMocks keep a record of each time they’re called, so it’s a good habit for the test code itself not to call its own MagicMocks because this can pollute the call record. Even at times when you could get away with it, it’s just good to be consistent and always use the special return_value attribute instead.

Tip: If you ever want a mock object that you don’t want the code under test to be able to call there’s a separate class for that - NonCallableMagicMock.

You can ask a MagicMock what calls have been made to it

For the foo() method above we might want to test that it calls bar() with the right arguments, as well as that it returns the result of calling bar(). We can do this using MagicMock‘s assert_called_once_with() method:

def test_foo_calls_bar_correctly():
    bar = mock.MagicMock()

    foo(bar)

    bar.assert_called_once_with("expected_arg_1", "expected_arg_2")

assert_called_once_with() will raise an AssertionError, causing the test to fail, if foo() does not call bar(), if it calls bar() more than once, or if it calls bar() with the wrong arguments.

These kind of assertions about how a MagicMock was used are often useful in testing integration functions - functions whose job is to glue together other functions and methods, calling them with the right arguments and doing the right things with their return values.

The mock library provides a whole collection of tools for testing how a mock was used, including:

You can call any method on a MagicMock

Normally you can only call a method on an object if that object’s class has that method, otherwise you’ll get AttributeError:

>>> class Foo(object):
...     def do_something(self):
...         pass
... 
...
>>> foo = Foo()
>>> foo.bar()
Traceback (most recent call last):
  ...
AttributeError: 'Foo' object has no attribute 'bar'

If you try to call a method that does exist, but you pass it a positional or keyword argument that the method doesn’t have, you’ll get a TypeError:

>>> foo.do_something(1)
Traceback (most recent call last):
  ...
TypeError: do_something() takes exactly 1 argument (2 given)

>>> foo.do_something(some_arg=12)
Traceback (most recent call last):
  ...
TypeError: do_something() got an unexpected keyword argument 'some_arg'

Since any attribute name on a MagicMock is another MagicMock, and since MagicMocks are callable, that means that as well as accessing any attribute name on a MagicMock you can also call any method on a MagicMock and pass any positional or keyword arguments:

>>> my_mock = mock.MagicMock()
>>> my_mock.foo()
<MagicMock name='mock.foo()' id='140404522292688'>
>>> my_mock.bar(16)
<MagicMock name='mock.bar()' id='140404522339536'>
>>> my_mock.whatever_method_name_you_want("foobar")
<MagicMock name='mock.whatever_method_name_you_want()' id='140404522411024'>

This means that if the code under test uses an object that it calls methods on, a test can pass in a MagicMock in place of that object and the code will still work:

def foo(bar):
    ...
    return bar.some_method(arg1, arg2, ...)

def test_foo_returns_what_some_method_returned():
    bar = mock.MagicMock()

    returned = foo(bar)

    assert returned == bar.some_method.return_value

Of course, each method returns another mock when called. The same method always returns the same other mock object (no matter what arguments its called with), but each different method returns a different other mock.

If foo() had called bar.some_other_method() instead, it would have returned a different MagicMock (some_other_method.return_value rather than some_method.return_value) and the assert would have failed.

Note: When you access a name on a mock object it returns a different other mock object to if you call that name:

>>> my_mock.foo() == my_mock.foo
False

The mock that calling returns is accessible as return_value:

>>> my_mock.foo() == my_mock.foo.return_value
True

And of course, if you’ve set a certain attribute name on a mock to a non-callable value then you can’t call it anymore:

>>> my_mock.foo = 23
>>> my_mock.foo
23
>>> my_mock.foo()
Traceback (most recent call last):
  ...
TypeError: 'int' object is not callable

You can tell a MagicMock method what value to return

We’ve seen that the special return_value attribute is the value that the mock will return if called (by default, another mock). You can set return_value to something else and the mock will return that when called instead:

>>> my_mock()
<MagicMock name='mock()' id='140404522909008'>
>>> my_mock.return_value = 'custom return value'
>>> my_mock()
'custom return value'

This of course means that you can control the return value of any methods on the mock as well:

>>> my_mock.bar.return_value = 26.2
>>> my_mock.bar()
26.2

You can also set the return value of a mock as a constructor argument:

>>> my_mock = mock.MagicMock(return_value='custom_value')
>>> my_mock()
'custom_value'

>>> my_mock = mock.MagicMock(foo=mock.MagicMock(return_value='custom_value'))
>>> my_mock.foo()
'custom_value'

Tests can set the return_value of a mock when they need it be something other than another mock object. In one of our examples above we needed annotation.created to be a datetime object rather than another MagicMock. If annotation.created() were a method instead of an attribute then we might have needed it to return a datetime instead:

def test_it_formats_created_into_a_string():
    annotation = mock.MagicMock()
    annotation.created.return_value = datetime.datetime(2016, 2, 24, 18, 3, 25, 768)

    annotation_dict = AnnotationSearchIndexPresenter(annotation).asdict()

    assert annotation_dict['created'] == '2016-02-24T18:03:25.000768+00:00'

When AnnotationSearchIndexPresenter calls annotation.created() it will get our datetime object as the return value, and our assert depends on that.

You can make a MagicMock method raise an exception

side_effect is another special attribute of mock objects. If you set side_effect to an exception class or exception object then the mock will raise the exception when called:

>>> my_mock.side_effect = RuntimeError('Something broke')
>>> my_mock()
Traceback (most recent call last):
  ...
RuntimeError: Something broke

You can use this to test that a method under test handles exceptions properly when they’re raised by other functions or methods that the method under test calls.

As with return_value you can also pass side_effect as an argument to the MagicMock() constructor: my_mock = mock.MagicMock(side_effect=...).

A simple example of a mock raising an exception is a test for the EventQueue class that we saw earlier. When EventQueue.publish_all() is called it calls the notify() method of each event object on the queue. If one of those notify() methods raises an exception EventQueue should not crash, but should catch and log the exception:

def test_publish_all_logs_exception(self):
    event = mock.MagicMock()
    log   = mock.MagicMock()
    queue = eventqueue.EventQueue(log)
    queue.push(event)

    # Make event.notify() raise ValueError when called.
    event.notify.side_effect = ValueError('exploded!')

    # When publish_all() calls event.notify() it'll raise the ValueError,
    # rather than crashing publish.all() should catch this exception and log it.
    queue.publish_all()

    assert log.exception.called

You can make a MagicMock method return a different value each time

Sometimes your test needs a mock to return a sequence of different values each time it’s called. If you set side_effect to an iterable then each time the mock is called it’ll return the next item from that iterable:

>>> my_mock.side_effect = [1, 2, 3]
>>> my_mock()
1
>>> my_mock()
2
>>> my_mock()
3
>>> my_mock()
Traceback (most recent call last):
  ...
StopIteration

If one of the values that the iterable returns is an exception then the mock will raise an exception that time it’s called:

>>> my_mock.side_effect = [1, RuntimeError, 3]
>>> my_mock()
1
>>> my_mock()
Traceback (most recent call last):
  ...
RuntimeError
>>> my_mock()
3

You can attach a normal function to a MagicMock method

Lastly, if you set side_effect to a function then that function will be called when the mock is called, and the mock will return whatever that function returns. This can be useful when your test needs your mock to return a value dynamically, for example to return a value that depends on the value it was called with:

>>> def fake_encrypt(input):
...     return input + "_encrypted"
... 
>>> my_mock = mock.MagicMock()
>>> my_mock.encrypt.side_effect = fake_encrypt
>>> my_mock.encrypt("hello")
'hello_encrypted'

Setting side_effect to a function also means that the mock can only be called with the same arguments and keyword arguments that the side effect function takes, otherwise you’ll get a TypeError just as you would when calling any normal Python function with the wrong arguments. This is one way that your tests can ensure that the method under test never calls the mock with invalid arguments:

>>> my_mock.encrypt("arg1", "arg2")
Traceback (most recent call last):
  ...
TypeError: fake_encrypt() takes exactly 1 argument (2 given)

Conclusion

This concludes our tour of the MagicMock class and its features. The class has a lot of features, and this has been a long post, but this should give you an understanding of most of what’s going on when mocks are in use in the Hypothesis tests. For the rest of the details, go to the mock website.

Know that you know how to use mocks, the next post will explain some of the dangers of using mocks.