Planet (Former) Advogato

This is a complement to Advogato, it is an aggregation of blogs of those who used to post on Advogato, but for one reason or another moved their blog from Advogato. It is provided as a service to those who would like to read the "greater Advogato" community.

This site works only as a Planet, it aggregates the post only, to comment on a blog entry, click on the title or time to go to the blog entry on the original site, hopefully it will have a comment facility.

September 28, 2017

Michael Still [mikal]

I think I found a bug in python's unittest.mock library

Mocking is a pretty common thing to do in unit tests covering OpenStack Nova code. Over the years we've used various mock libraries to do that, with the flavor de jour being unittest.mock. I must say that I strongly prefer unittest.mock to the old mox code we used to write, but I think I just accidentally found a fairly big bug.

The problem is that python mocks are magical. Its an object where you can call any method name, and the mock will happily pretend it has that method, and return None. You can then later ask what "methods" were called on the mock.

However, you use the same mock object later to make assertions about what was called. Herein is the problem -- the mock object doesn't know if you're the code under test, or the code that's making assertions. So, if you fat finger the assertion in your test code, the assertion will just quietly map to a non-existent method which returns None, and your code will pass.

Here's an example:

    #!/usr/bin/python3
    
    from unittest import mock
    
    
    class foo(object):
        def dummy(a, b):
            return a + b
    
    
    @mock.patch.object(foo, 'dummy')
    def call_dummy(mock_dummy):
        f = foo()
        f.dummy(1, 2)
    
        print('Asserting a call should work if the call was made')
        mock_dummy.assert_has_calls([mock.call(1, 2)])
        print('Assertion for expected call passed')
    
        print()
        print('Asserting a call should raise an exception if the call wasn\'t made')
        mock_worked = False
        try:
            mock_dummy.assert_has_calls([mock.call(3, 4)])
        except AssertionError as e:
            mock_worked = True
            print('Expected failure, %s' % e)
    
        if not mock_worked:
            print('*** Assertion should have failed ***')
    
        print()
        print('Asserting a call where the assertion has a typo should fail, but '
              'doesn\'t')
        mock_worked = False
        try:
            mock_dummy.typo_assert_has_calls([mock.call(3, 4)])
        except AssertionError as e:
            mock_worked = True
            print('Expected failure, %s' % e)
            print()
    
        if not mock_worked:
            print('*** Assertion should have failed ***')
            print(mock_dummy.mock_calls)
            print()
    
    
    if __name__ == '__main__':
        call_dummy()
    


If I run that code, I get this:

    $ python3 mock_assert_errors.py 
    Asserting a call should work if the call was made
    Assertion for expected call passed
    
    Asserting a call should raise an exception if the call wasn't made
    Expected failure, Calls not found.
    Expected: [call(3, 4)]
    Actual: [call(1, 2)]
    
    Asserting a call where the assertion has a typo should fail, but doesn't
    *** Assertion should have failed ***
    [call(1, 2), call.typo_assert_has_calls([call(3, 4)])]
    


So, we should have been told that typo_assert_has_calls isn't a thing, but we didn't notice because it silently failed. I discovered this when I noticed an assertion with a (smaller than this) typo in its call in a code review yesterday.

I don't really have a solution to this right now (I'm home sick and not thinking straight), but it would be interesting to see what other people think.

Tags for this post: python unittest.mock mock testing
Related posts: Implementing SCP with paramiko; Packet capture in python; A pythonic example of recording metrics about ephemeral scripts with prometheus; mbot: new hotness in Google Talk bots; Starfish Prime; Calculating a SSH host key with paramiko Comment

September 28, 2017 05:58 AM

September 24, 2017

Rachel Chalmers [rachel]

war mare

I ride Chione, the bright golden Haflinger dressage pony of my heart. I’m holding my arms in a round O now, like first position in ballet, an innovation from the great New Zealand coach Greg Best, apparently. It stops me bracing with my hands and gives me a whole other dimension of range of motion in my arms.

Chione flows forward into my softer contact. I sit to her trot, with my lower leg relaxed and my inner thigh engaged. I shift my inside hipbone forward. She steps forward with her outside hind leg into a perfect canter depart.

Everything is warm and light and nothing hurts.

September 24, 2017 01:58 AM