Saturday 12 June 2010

Mock recipes

I posted recently a simple introduction to the mock library. Today, I'd like to show you some of its indirect uses, that I found helpful in my everyday testing needs.

Custom patcher

It is very easy to set up mock objects to respond to events. You just access attributes and set either them or their return value. When using the patch decorator, a mock instance is provided to your test method as the last argument, so you set it up just before excercising your production code.

@mock.patch("sys.stdin")
def test_stdin_reader(mock_stdin):
mock_stdin.write.return_value = "hello" # setting up mock_stdin
run_code_under_test()
validate_results()

In this example the setting up part is really easy - just a single line. But in real-life projects, you might find yourself setting whole attribute hierarchies on mocks, repeatedly in every many test methods.

That's when a custom patcher allows you to implement the DRY principle in an elegant manner.

class MyTest(unittest.TestCase):    
@patch_db
def test_one(self):
pass
@patch_db
def test_two(self):
pass

It's very easy to write a custom patcher.

def patch_db(func):    
@patch("db.module")
def wrapper(*args):
mock_db = args[-1]
# ... perform complicated setup on mock_db
func(*args[:-1])
return wrapper


Cutting off

The cut_off patcher is useful when you need to mock out several objects, and don't care about the mock instances. Say you want to deactivate internet access in a module that uses both urllibs.

@cut_off("mymodule.urllib", "mymodule.urllib2")
def test_something():
pass

The implementation of cut_off is very simple too.

def cut_off(*patches):    
def decorator(meth):
def wrapper(self, *args):
return meth(self, *args[:-len(patches)])
for obj in patches:
wrapper = mock.patch(obj)(wrapper)
return wrapper
return decorator

Monday 7 June 2010

Analyzing PyPI packages

PyPI, which stands for Python Package Index is a global repository for Python
packages. Every time you need a Python tool or library, you can simply type
easy_install mypackage, and have it downloaded and
installed for you. It is also a great source when trying to investigate current
practices in the Python world.

Disclaimer


There are couple of troubles when analyzing PyPI. First - it is a moving target.
Since I first run the download script (which was 3 days ago), it grew by 20 new
packages. So, please bear in mind, this information won't very exact. Still, it
provides a nice overview. Second - not all packages are hosted in PyPI. For some
(quite a lot, actually) cases, we only get a link to the actual download source.
This grows the chance of a host being, and causes the download to fail. Third -
PyPI packages are terribly diverse. In order to analyze it in a timely manner, I
picked only the ones that could be downloaded as either tarballs or zips. This
reduced the sample by a quarter (from 10112 to 7625), which I believe is still a
representative enough.

Setup.py usage


Most of the packages (96%) used setup.py. The rest either simply didn't use it
or used a non-standard directory layout (accordingly: 187 and 47). Out of
setup.py users, setuptools was more than three time more popular than standard
distutils. 73 packages couldn't be identified as using either of these, and this
is mostly caused by custom setup function wrappers (see 4Suite for example of
this).



Test runners


I was curious, how people run their tests, so I identified several ways it could
be done:
  1. using a top-level shell script: 20
  2. using a top-level python script: 326
  3. using setuptools' test command: 961



Note: these stats don't include another popular way of running tests, used by
Django apps.

There where 1048 packages having a toplevel directory containing string "test",
among which the most popular varations were unsurprisingly "test" (477) and
"tests"(456).

Tuesday 1 June 2010

5 things you can do with a Python list in one line

This is directly inspired by an excellent post by Drew Olson 5 things you can do with a Ruby array in one line. When reading it, I couldn't help but thinking of the Python versions (and how I like them more :>). So here it is:

  1. Summing elements
    puts my_array.inject(0){|sum,item| sum + item}
    sum(my_list)
  2. Double every item.
    my_array.map{|item| item*2 }
    [2 * x for x in my_list]
  3. Finding all items that meet your criteria.
    my_array.find_all{|item| item % 3 == 0 }
    [x for x in my_list if x % 3 == 0]
  4. Combine techniques.
    my_array.find_all{|item| item % 3 == 0 }.inject(0){|sum,item| sum + item }
    sum(x for x in my_list if x % 3 == 0)
  5. Sorting.
    my_array.sort
    my_array.sort_by{|item| item*-1}
    sorted(my_list)
    sorted(my_list, reverse=True)