Introduction to Selenium for Python programmers

Selenium is an application that automates web browsers, helping you test your web application from a user perspective, in an automated manner. These properties make Selenium tests a perfect fit for validating your js-level functionality and implementing acceptance tests.

Of course, it has some drawbacks: you need to run your application from another process, which gives you some pain with checking the backend state of things. The tests might be quite slow, and - if you don't write them well - extremely fragile.

However, starting with basic Selenium tests is very simple, which I'm going to prove below. We will create a trivial website, with a single element only: a link to Google. Next, we will implement a Selenium test that makes sure this indeed happens.

Prepare the environment

We will work in a virtualenv called "seltest". If you don't know what virtualenv is, you likely want to read this first. Enter the directory of your choice and run the following commands:

mkdir seltest
cd seltest
virtualenv --no-site-packages .

We will work from within the seltest directory, so that we don't need to activate the virtualenv, and instead call our binaries by "bin/python" or "bin/pip".

Let's download the Selenium executable and its python bindings:

wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0a4.jar
bin/pip install selenium

You can already play with Selenium. First, start another terminal window and run:

java -jar selenium-server-standalone-2.0a4.jar

Then from our main terminal:

$ bin/python
>>> from selenium.remote import connect
>>> from selenium import FIREFOX
>>> browser = connect(FIREFOX) # this will run the browser
>>> browser.get("http://www.yahoo.com") # you should see the browser navigating to yahoo
>>> browser.close() # this will close the session

Prepare and run the website

The website will consist of a single link, we can skip all the obligatory html boilerplate at this stage. Save the text

<a href="http://google.com">Go to Google</a>

into a file index.html, and from another terminal (yes, you will need three terminal windows) run:

Python -m SimpleHTTPServer

This will start serving your page on the port 8000, you can visit the page from your web browser on http://localhost:8000/

Implement the test

Open the file selenium_test.py in your editor of choice and dump the following

import unittest
from selenium.remote import connect
from selenium import FIREFOX

class SelTest(unittest.TestCase):
    def setUp(self):
        self.browser = connect(FIREFOX)
    def tearDown(self):
        self.browser.close()
    def test_simple(self):
        self.browser.get("http://localhost:8000/")
        link = self.browser.find_element_by_partial_link_text("Google")
        link.click()
        self.assertEqual(self.browser.get_title(), "Google")

if __name__ == "__main__":
    unittest.main()

The setUp and tearDown methods manage the browser session, and the actual test lives in the test_simple method. We are using four methods from the browser object: get, find_element_by_partial_link_text, click and get_title. In case you wonder where these come from, look for the WebDriver class definition. You can find it in lib/python2.6/site-packages/selenium/remote/webdriver.py in your environment.

Run the test

Now, you are ready to run your test.

bin/python selenium_test.py

You should see something along the lines of:

$ bin/python selenium_test.py 
.
----------------------------------------------------------------------
Ran 1 test in 5.521s

OK

Which indicates, that all your tests passed correctly.

Optimizing fabfiles

I really like my deploys to be as fast as possible. Unfortunately, the RTT between my and my server makes this quite hard. Today, I came up with a simple optimisation, that lets you make your fabric commands faster (saving on RTT). Say you have a series of consecutive "run" calls. Each call needs to get sent, evaluated and the results need to come back. Why wait for them, when we don't want to continue after failure anyway? The simple fix is to change this:

def my_task():
    run("command_1")
    run("command 2")
    run("command 3")

... into this:

def my_task():
    commands = []
    _run = commands.append
    _run("command_1")
    _run("command 2")
    _run("command 3")
    run(" && ".join(commands))

This way, all your commands get called, and the execution still stops on first failure.

Customising Django's uniqueness validation message

In case you've been wondering: you need to override the unique_error_message method on your model. The unique_check argument is a tuple containing field names that are supposed to be unique together (for regular uniqueness this is a one-element tuple). See the example below for validating the slug field:

class MyModel(models.Model):
    slug = models.SlugField(max_length=200, unique=True)
    def unique_error_message(self, model_class, unique_check):
        if unique_check == ("slug",):
            return u"This slug is already taken"
        else:
            return super(Office, self).unique_error_message(model_class, unique_check)

Issues with Django and MySQL on Mac OS X

Today, I spent more time than planned setting up a Django installation with MySQL database backend. Below are the problems I encountered and the ways how I dealt with them.

mysql_config

I'm using what's supposed to be the simplest MySQL installation out there - the official dmg from http://dev.mysql.com/. The installer puts all the files into

/usr/local/mysql

(with the regular bin, lib and include directories inside). The installation also includes the usual mysql_config executable which takes care for pointing out all the paths required by the MySQL-python package. The problem is that without the system path preconfigured the MySQL-python installer isn't able to find it. I had to point out to the package where mysql_config is. Luckily, there's a setting for that. I downloaded the package:

bin/pip install MySQL-python --no-install

(it raised an error but downloaded the files just fine). Then, in a file called site.cfg, I switched the following line

#mysql_config = /usr/local/bin/mysql_config

into

mysql_config = /usr/local/mysql/bin/mysql_config

- uncommenting it and putting the right path in place. I was free to continue with the installation:

bin/pip install build/MySQL-python

The library path

Unfortunately, this didn't solve all the problems, as I kept getting the following error when trying to run any code that used the package:

Error loading MySQLdb module: dlopen(/path/to/site-packages/_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib

As it turns out, the Ruby guys have a similar problem with that, and it's in a Rails-related post, where I found a solution. For some mysterious reason, the name of mysqlclient library was saved without the full absolute path, as I could see running the otool command:

$otool -L lib/python2.6/site-packages/_mysql.so 
lib/python2.6/site-packages/_mysql.so:
 libmysqlclient.16.dylib (compatibility version 16.0.0, current version 16.0.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)

When I changed it to the full path with the install_name_tool command:

sudo install_name_tool -change libmysqlclient.16.dylib /usr/local/mysql/lib/libmysqlclient.16.dylib lib/python2.6/site-packages/_mysql.so

all worked fine. Hope that helps someone in similar trouble.

Mocking empty collections with FalseMock

A friend of mine described to me a PITA he had with mock - it doesn't play well with a common Python idiom:

if collection:
    for element in collection:
        do_something(element)

What he expected as default behaviour was for the mock to be iteratable as an empty collection. Instead, he got:

>>> m = mock.Mock()
>>> for x in m:
...     print x
... 
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'Mock' object is not iterable

As it turns out, there is a simple solution to this problem: all you need to do is to implement the magic method yourself.

>>> class FalseMock(mock.Mock):
...     def __nonzero__(self):
...             return False

This behaves as you would expect: It evaluates to False.

>>> m = FalseMock()
>>> bool(m)
False

It still works as every other Mock object would:

>>> m.a.b.c.d.return_value = "hi"
>>> m.a.b.c.d()
'hi'

It keeps its behaviour in all Mocks generated as a result of getting an attribute

>>> m.a.b.c.d

Distutils2 Summer of Code

This summer, I took part in Google Summer of Code project as a student. I worked on distutils2 project implementing new commands and improving the existing ones. In poarticular, my tasks were:

• to implement a test command (similar to the one from setuptools/distribute
• implement command hooks
• port the upload_docs command
• enhance the check command

I started with a low hanging fruit: porting upload_docs. This didn't take very long. In fact I spent most of my time with the test code, implementing mock PyPI server, which - I'm proud to say - turned out quite useful for other student's tests (although, my initial implementation used wsgiref - not available in Python 2.4 and Alexis refactored practically all of it, so it's rather his child).

After that, I started working on the test command. The new command's API is a little bit different than the one you might know from setuptools/distribute: the options "test-suite" and "test-loader" got replaced with "suite" and "runner". You would use "suite" option in place of "test-suite", but setting the "runner" to a dotted path to a Python callable will cause the test command to invoke that callable in place of default unittest test runner. In absence of both options, the test command will invoke test discovery as implemented in unittest (Python 2.7, 3.2 and newer) or unittest2 (if installed, for older Python versions).

The next task was to implement command hooks. The command pre- and post-hooks are Python callables that accept the command instance (giving it access to all its options). You specify them in your setup.cfg file as the command's options. In addition to that, you have to specify a postfix for you hook, so that it wouldn't override hooks from other files:

[install]
  pre-hook.my_postfix = path.to.hook

The improvements to the check command were arguably most neglected task in my project. I gathered the optional checks (originally only check for validity of ReStructuredText in package's description) under a single option --all, and implemented additional check: one that validates importability of the hooks.

Not all changes are upstream yet. The hooks and the upload_docs command are already in the central repo. The test command and check improvements are waiting for the merge. Tomorrow, I'm leaving for my summer kayak trip, but I'm looking forward to contributing more to distutils in the future.

Overall, I am very happy with the time I spent working on the project. Even in spite of some problems in my way, I enjoyed the experience of working with a team of extremely smart guys. The regular IRC meetings were instructive and enjoyable. I would also like to thank my mentor: Fred Drake, who offered all the help and feedback I needed.

Mock recipes

I posted recently a simple introduction to the mock library. Today, I'd like to show you some of its indirect uses, that I found helpful in my everyday testing needs.

Custom patcher

It is very easy to set up mock objects to respond to events. You just access attributes and set either them or their return value. When using the patch decorator, a mock instance is provided to your test method as the last argument, so you set it up just before excercising your production code.

@mock.patch("sys.stdin")
def test_stdin_reader(mock_stdin):
    mock_stdin.write.return_value = "hello"  # setting up mock_stdin
    run_code_under_test()    
    validate_results()

In this example the setting up part is really easy - just a single line. But in real-life projects, you might find yourself setting whole attribute hierarchies on mocks, repeatedly in every many test methods.

That's when a custom patcher allows you to implement the DRY principle in an elegant manner.

class MyTest(unittest.TestCase):    
    @patch_db    
    def test_one(self):
        pass
    @patch_db    
    def test_two(self):
        pass

It's very easy to write a custom patcher.

def patch_db(func):    
    @patch("db.module")    
    def wrapper(*args):        
        mock_db = args[-1]        
        # ... perform complicated setup on mock_db        
        func(*args[:-1])    
    return wrapper

Cutting off

The cut_off patcher is useful when you need to mock out several objects, and don't care about the mock instances. Say you want to deactivate internet access in a module that uses both urllibs.

@cut_off("mymodule.urllib", "mymodule.urllib2")
def test_something():
    pass

The implementation of cut_off is very simple too.

def cut_off(*patches):    
    def decorator(meth):        
        def wrapper(self, *args):            
            return meth(self, *args[:-len(patches)])        
        for obj in patches:            
            wrapper = mock.patch(obj)(wrapper)        
        return wrapper    
    return decorator