Introduction to Selenium for Python programmers

Selenium is an application that automates web browsers, helping you test your web application from a user perspective, in an automated manner. These properties make Selenium tests a perfect fit for validating your js-level functionality and implementing acceptance tests.

Of course, it has some drawbacks: you need to run your application from another process, which gives you some pain with checking the backend state of things. The tests might be quite slow, and - if you don't write them well - extremely fragile.

However, starting with basic Selenium tests is very simple, which I'm going to prove below. We will create a trivial website, with a single element only: a link to Google. Next, we will implement a Selenium test that makes sure this indeed happens.

Prepare the environment

We will work in a virtualenv called "seltest". If you don't know what virtualenv is, you likely want to read this first. Enter the directory of your choice and run the following commands:

mkdir seltest
cd seltest
virtualenv --no-site-packages .

We will work from within the seltest directory, so that we don't need to activate the virtualenv, and instead call our binaries by "bin/python" or "bin/pip".

Let's download the Selenium executable and its python bindings:

wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0a4.jar
bin/pip install selenium

You can already play with Selenium. First, start another terminal window and run:

java -jar selenium-server-standalone-2.0a4.jar

Then from our main terminal:

$ bin/python
>>> from selenium.remote import connect
>>> from selenium import FIREFOX
>>> browser = connect(FIREFOX) # this will run the browser
>>> browser.get("http://www.yahoo.com") # you should see the browser navigating to yahoo
>>> browser.close() # this will close the session

Prepare and run the website

The website will consist of a single link, we can skip all the obligatory html boilerplate at this stage. Save the text

<a href="http://google.com">Go to Google</a>

into a file index.html, and from another terminal (yes, you will need three terminal windows) run:

Python -m SimpleHTTPServer

This will start serving your page on the port 8000, you can visit the page from your web browser on http://localhost:8000/

Implement the test

Open the file selenium_test.py in your editor of choice and dump the following

import unittest
from selenium.remote import connect
from selenium import FIREFOX

class SelTest(unittest.TestCase):
    def setUp(self):
        self.browser = connect(FIREFOX)
    def tearDown(self):
        self.browser.close()
    def test_simple(self):
        self.browser.get("http://localhost:8000/")
        link = self.browser.find_element_by_partial_link_text("Google")
        link.click()
        self.assertEqual(self.browser.get_title(), "Google")

if __name__ == "__main__":
    unittest.main()

The setUp and tearDown methods manage the browser session, and the actual test lives in the test_simple method. We are using four methods from the browser object: get, find_element_by_partial_link_text, click and get_title. In case you wonder where these come from, look for the WebDriver class definition. You can find it in lib/python2.6/site-packages/selenium/remote/webdriver.py in your environment.

Run the test

Now, you are ready to run your test.

bin/python selenium_test.py

You should see something along the lines of:

$ bin/python selenium_test.py 
.
----------------------------------------------------------------------
Ran 1 test in 5.521s

OK

Which indicates, that all your tests passed correctly.

Optimizing fabfiles

I really like my deploys to be as fast as possible. Unfortunately, the RTT between my and my server makes this quite hard. Today, I came up with a simple optimisation, that lets you make your fabric commands faster (saving on RTT). Say you have a series of consecutive "run" calls. Each call needs to get sent, evaluated and the results need to come back. Why wait for them, when we don't want to continue after failure anyway? The simple fix is to change this:

def my_task():
    run("command_1")
    run("command 2")
    run("command 3")

... into this:

def my_task():
    commands = []
    _run = commands.append
    _run("command_1")
    _run("command 2")
    _run("command 3")
    run(" && ".join(commands))

This way, all your commands get called, and the execution still stops on first failure.

Customising Django's uniqueness validation message

In case you've been wondering: you need to override the unique_error_message method on your model. The unique_check argument is a tuple containing field names that are supposed to be unique together (for regular uniqueness this is a one-element tuple). See the example below for validating the slug field:

class MyModel(models.Model):
    slug = models.SlugField(max_length=200, unique=True)
    def unique_error_message(self, model_class, unique_check):
        if unique_check == ("slug",):
            return u"This slug is already taken"
        else:
            return super(Office, self).unique_error_message(model_class, unique_check)

Issues with Django and MySQL on Mac OS X

Today, I spent more time than planned setting up a Django installation with MySQL database backend. Below are the problems I encountered and the ways how I dealt with them.

mysql_config

I'm using what's supposed to be the simplest MySQL installation out there - the official dmg from http://dev.mysql.com/. The installer puts all the files into

/usr/local/mysql

(with the regular bin, lib and include directories inside). The installation also includes the usual mysql_config executable which takes care for pointing out all the paths required by the MySQL-python package. The problem is that without the system path preconfigured the MySQL-python installer isn't able to find it. I had to point out to the package where mysql_config is. Luckily, there's a setting for that. I downloaded the package:

bin/pip install MySQL-python --no-install

(it raised an error but downloaded the files just fine). Then, in a file called site.cfg, I switched the following line

#mysql_config = /usr/local/bin/mysql_config

into

mysql_config = /usr/local/mysql/bin/mysql_config

- uncommenting it and putting the right path in place. I was free to continue with the installation:

bin/pip install build/MySQL-python

The library path

Unfortunately, this didn't solve all the problems, as I kept getting the following error when trying to run any code that used the package:

Error loading MySQLdb module: dlopen(/path/to/site-packages/_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib

As it turns out, the Ruby guys have a similar problem with that, and it's in a Rails-related post, where I found a solution. For some mysterious reason, the name of mysqlclient library was saved without the full absolute path, as I could see running the otool command:

$otool -L lib/python2.6/site-packages/_mysql.so 
lib/python2.6/site-packages/_mysql.so:
 libmysqlclient.16.dylib (compatibility version 16.0.0, current version 16.0.0)
 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)

When I changed it to the full path with the install_name_tool command:

sudo install_name_tool -change libmysqlclient.16.dylib /usr/local/mysql/lib/libmysqlclient.16.dylib lib/python2.6/site-packages/_mysql.so

all worked fine. Hope that helps someone in similar trouble.