tag:blogger.com,1999:blog-77890598769275000892024-03-13T07:25:13.402-07:00Most recent callMostly python-related stuff.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.comBlogger61125tag:blogger.com,1999:blog-7789059876927500089.post-69539981218417680452011-01-19T05:10:00.000-08:002011-01-20T09:57:26.254-08:00Introduction to Selenium for Python programmersSelenium is an application that automates web browsers, helping you test your web application from a user perspective, in an automated manner. These properties make Selenium tests a perfect fit for validating your js-level functionality and implementing acceptance tests.<br /><br />Of course, it has some drawbacks: you need to run your application from another process, which gives you some pain with checking the backend state of things. The tests might be quite slow, and - if you don't write them well - extremely fragile.<br /><br />However, starting with basic Selenium tests is very simple, which I'm going to prove below. We will create a trivial website, with a single element only: a link to Google. Next, we will implement a Selenium test that makes sure this indeed happens.<br /><br /><h4>Prepare the environment</h4><br />We will work in a virtualenv called "seltest". If you don't know what virtualenv is, you likely want to read <a href="http://mitchfournier.com/2010/06/25/getting-started-with-virtualenv-isolated-python-environments/">this</a> first. Enter the directory of your choice and run the following commands:<br /><br /><pre><code>mkdir seltest<br />cd seltest<br />virtualenv --no-site-packages .</code></pre><br /><br />We will work from within the seltest directory, so that we don't need to activate the virtualenv, and instead call our binaries by "bin/python" or "bin/pip".<br /><br />Let's download the Selenium executable and its python bindings:<br /><br /><pre><code>wget http://selenium.googlecode.com/files/selenium-server-standalone-2.0a4.jar<br />bin/pip install selenium</code></pre><br /><br />You can already play with Selenium. First, start another terminal window and run:<br /><br /><pre><code>java -jar selenium-server-standalone-2.0a4.jar</code></pre><br /><br />Then from our main terminal:<br /><br /><pre><code>$ bin/python<br />>>> from selenium.remote import connect<br />>>> from selenium import FIREFOX<br />>>> browser = connect(FIREFOX) # this will run the browser<br />>>> browser.get("http://www.yahoo.com") # you should see the browser navigating to yahoo<br />>>> browser.close() # this will close the session</code></pre><br /><br /><h4>Prepare and run the website</h4><br /><br />The website will consist of a single link, we can skip all the obligatory html boilerplate at this stage. Save the text<br /><br /><pre><a href="http://google.com">Go to Google</a></pre><br /><br />into a file index.html, and from another terminal (yes, you will need three terminal windows) run:<br /><br /><pre><code>Python -m SimpleHTTPServer</code></pre><br /><br />This will start serving your page on the port 8000, you can visit the page from your web browser on <a href="http://localhost:8000/">http://localhost:8000/</a><br /><br /><br /><h4>Implement the test</h4><br /><br />Open the file selenium_test.py in your editor of choice and dump the following<br /><br /><pre><code>import unittest<br />from selenium.remote import connect<br />from selenium import FIREFOX<br /><br />class SelTest(unittest.TestCase):<br /> def setUp(self):<br /> self.browser = connect(FIREFOX)<br /> def tearDown(self):<br /> self.browser.close()<br /> def test_simple(self):<br /> self.browser.get("http://localhost:8000/")<br /> link = self.browser.find_element_by_partial_link_text("Google")<br /> link.click()<br /> self.assertEqual(self.browser.get_title(), "Google")<br /><br />if __name__ == "__main__":<br /> unittest.main()</code></pre><br /><br />The setUp and tearDown methods manage the browser session, and the actual test lives in the test_simple method. We are using four methods from the browser object: get, find_element_by_partial_link_text, click and get_title. In case you wonder where these come from, look for the WebDriver class definition. You can find it in lib/python2.6/site-packages/selenium/remote/webdriver.py in your environment.<br /><br /><br /><h4>Run the test</h4><br /><br />Now, you are ready to run your test.<br /><br /><pre><code><br />bin/python selenium_test.py<br /></code></pre><br /><br />You should see something along the lines of:<br /><br /><pre><code>$ bin/python selenium_test.py <br />.<br />----------------------------------------------------------------------<br />Ran 1 test in 5.521s<br /><br />OK</code></pre><br /><br />Which indicates, that all your tests passed correctly.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com10tag:blogger.com,1999:blog-7789059876927500089.post-81036743070638515852011-01-15T11:18:00.000-08:002011-01-15T11:19:49.878-08:00Optimizing fabfilesI really like my deploys to be as fast as possible. Unfortunately, the RTT between my and my server makes this quite hard. Today, I came up with a simple optimisation, that lets you make your fabric commands faster (saving on RTT). Say you have a series of consecutive "run" calls. Each call needs to get sent, evaluated and the results need to come back. Why wait for them, when we don't want to continue after failure anyway? The simple fix is to change this:<br /><pre><code>def my_task():<br /> run("command_1")<br /> run("command 2")<br /> run("command 3")</code></pre>... into this:<br /><pre><code>def my_task():<br /> commands = []<br /> _run = commands.append<br /> _run("command_1")<br /> _run("command 2")<br /> _run("command 3")<br /> run(" && ".join(commands))</code></pre>This way, all your commands get called, and the execution still stops on first failure.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com2tag:blogger.com,1999:blog-7789059876927500089.post-78324006944107711392011-01-09T16:47:00.000-08:002011-01-09T16:49:22.478-08:00Customising Django's uniqueness validation messageIn case you've been wondering: you need to override the unique_error_message method on your model. The unique_check argument is a tuple containing field names that are supposed to be unique together (for regular uniqueness this is a one-element tuple). See the example below for validating the slug field:<br /><pre><code>class MyModel(models.Model):<br /> slug = models.SlugField(max_length=200, unique=True)<br /> def unique_error_message(self, model_class, unique_check):<br /> if unique_check == ("slug",):<br /> return u"This slug is already taken"<br /> else:<br /> return super(Office, self).unique_error_message(model_class, unique_check)</code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-65966617798454029552011-01-03T13:26:00.000-08:002011-01-03T13:26:07.153-08:00Issues with Django and MySQL on Mac OS XToday, I spent more time than planned setting up a Django installation with MySQL database backend. Below are the problems I encountered and the ways how I dealt with them.<br />
<h4>mysql_config</h4>I'm using what's supposed to be the simplest MySQL installation out there - the official dmg from <a href="http://dev.mysql.com/downloads/mysql">http://dev.mysql.com/</a>. The installer puts all the files into <pre>/usr/local/mysql</pre>(with the regular bin, lib and include directories inside). The installation also includes the usual mysql_config executable which takes care for pointing out all the paths required by the MySQL-python package. The problem is that without the system path preconfigured the MySQL-python installer isn't able to find it. I had to point out to the package where mysql_config is. Luckily, there's a setting for that. I downloaded the package:<pre><code>bin/pip install MySQL-python --no-install</code></pre>(it raised an error but downloaded the files just fine). Then, in a file called site.cfg, I switched the following line<pre><code>#mysql_config = /usr/local/bin/mysql_config</code></pre>into <pre><code>mysql_config = /usr/local/mysql/bin/mysql_config</code></pre>- uncommenting it and putting the right path in place. I was free to continue with the installation:<br />
<pre><code>bin/pip install build/MySQL-python</code></pre><br />
<h4>The library path</h4>Unfortunately, this didn't solve all the problems, as I kept getting the following error when trying to run any code that used the package: <pre>Error loading MySQLdb module: dlopen(/path/to/site-packages/_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib</pre><br />
As it turns out, the Ruby guys have a similar problem with that, and it's in a <a href="http://freddyandersen.wordpress.com/2010/10/03/mysql-5-5-snow-leopard-and-rails/">Rails-related post</a>, where I found a solution. For some mysterious reason, the name of mysqlclient library was saved without the full absolute path, as I could see running the otool command:<br />
<pre><code>$otool -L lib/python2.6/site-packages/_mysql.so
lib/python2.6/site-packages/_mysql.so:
libmysqlclient.16.dylib (compatibility version 16.0.0, current version 16.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.1)
</code></pre><br />
When I changed it to the full path with the install_name_tool command:<br />
<pre><code>sudo install_name_tool -change libmysqlclient.16.dylib /usr/local/mysql/lib/libmysqlclient.16.dylib lib/python2.6/site-packages/_mysql.so</code></pre><br />
all worked fine. Hope that helps someone in similar trouble.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com7tag:blogger.com,1999:blog-7789059876927500089.post-78642774786158851932010-12-20T07:54:00.000-08:002010-12-20T07:54:56.216-08:00Mocking empty collections with FalseMockA friend of mine described to me a PITA he had with mock - it doesn't play well with a common Python idiom:<pre><code class="python">if collection:
for element in collection:
do_something(element)
</code></pre>What he expected as default behaviour was for the mock to be iteratable as an empty collection. Instead, he got:<br />
<pre><code class="python">>>> m = mock.Mock()
>>> for x in m:
... print x
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'Mock' object is not iterable</code></pre>As it turns out, there is a simple solution to this problem: all you need to do is to implement the magic method yourself.<br />
<pre><code class="python">>>> class FalseMock(mock.Mock):
... def __nonzero__(self):
... return False</code></pre>This behaves as you would expect: It evaluates to False.<br />
<pre><code class="python">>>> m = FalseMock()
>>> bool(m)
False</code></pre>It still works as every other Mock object would:<br />
<pre><code class="python">>>> m.a.b.c.d.return_value = "hi"
>>> m.a.b.c.d()
'hi'</code></pre>It keeps its behaviour in all Mocks generated as a result of getting an attribute<br />
<pre><code class="python">>>> m.a.b.c.d
<FalseMock name='mock.a.b.c.d' id='4300700240'></code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com2tag:blogger.com,1999:blog-7789059876927500089.post-75779602937344735742010-08-16T11:02:00.000-07:002010-08-16T11:02:44.505-07:00Distutils2 Summer of CodeThis summer, I took part in Google Summer of Code project as a student. I worked on distutils2 project implementing new commands and improving the existing ones. In poarticular, my tasks were:<ul><li>to implement a test command (similar to the one from setuptools/distribute</li><li>implement command hooks</li><li>port the upload_docs command</li><li>enhance the check command</li></ul>I started with a low hanging fruit: porting upload_docs. This didn't take very long. In fact I spent most of my time with the test code, implementing mock PyPI server, which - I'm proud to say - turned out quite useful for other student's tests (although, my initial implementation used wsgiref - not available in Python 2.4 and Alexis refactored practically all of it, so it's rather his child).<br />
<br />
After that, I started working on the test command. The new command's API is a little bit different than the one you might know from setuptools/distribute: the options "test-suite" and "test-loader" got replaced with "suite" and "runner". You would use "suite" option in place of "test-suite", but setting the "runner" to a dotted path to a Python callable will cause the test command to invoke that callable in place of default unittest test runner. In absence of both options, the test command will invoke test discovery as implemented in unittest (Python 2.7, 3.2 and newer) or unittest2 (if installed, for older Python versions).<br />
<br />
The next task was to implement command hooks. The command pre- and post-hooks are Python callables that accept the command instance (giving it access to all its options). You specify them in your setup.cfg file as the command's options. In addition to that, you have to specify a postfix for you hook, so that it wouldn't override hooks from other files:<br />
<br />
<pre><code>[install]
pre-hook.my_postfix = path.to.hook</code></pre><br />
The improvements to the check command were arguably most neglected task in my project. I gathered the optional checks (originally only check for validity of ReStructuredText in package's description) under a single option <i>--all</i>, and implemented additional check: one that validates importability of the hooks.<br />
<br />
Not all changes are upstream yet. The hooks and the upload_docs command are already in the central repo. The test command and check improvements are waiting for the merge. Tomorrow, I'm leaving for my summer kayak trip, but I'm looking forward to contributing more to distutils in the future.<br />
<br />
Overall, I am very happy with the time I spent working on the project. Even in spite of some problems in my way, I enjoyed the experience of working with a team of extremely smart guys. The regular IRC meetings were instructive and enjoyable. I would also like to thank my mentor: Fred Drake, who offered all the help and feedback I needed.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com1tag:blogger.com,1999:blog-7789059876927500089.post-90333481358420589172010-06-12T12:57:00.000-07:002010-06-13T15:29:17.959-07:00Mock recipesI posted recently a <a href="http://konryd.blogspot.com/2010/05/mockity-mock-mock-some-love-for-mock.html">simple introduction</a> to the mock library. Today, I'd like to show you some of its indirect uses, that I found helpful in my everyday testing needs.<br /><h4>Custom patcher</h4>It is very easy to set up mock objects to respond to events. You just access attributes and set either them or their return value. When using the patch decorator, a mock instance is provided to your test method as the last argument, so you set it up just before excercising your production code.<br /><br /><pre><code>@mock.patch("sys.stdin")<br />def test_stdin_reader(mock_stdin):<br /> mock_stdin.write.return_value = "hello" # setting up mock_stdin<br /> run_code_under_test() <br /> validate_results()</code></pre> <br />In this example the setting up part is really easy - just a single line. But in real-life projects, you might find yourself setting whole attribute hierarchies on mocks, repeatedly in every many test methods.<br /><br />That's when a custom patcher allows you to implement the DRY principle in an elegant manner.<br /><br /><pre><code>class MyTest(unittest.TestCase): <br /> @patch_db <br /> def test_one(self):<br /> pass<br /> @patch_db <br /> def test_two(self):<br /> pass</code></pre><br />It's very easy to write a custom patcher.<br /><br /><pre><code>def patch_db(func): <br /> @patch("db.module") <br /> def wrapper(*args): <br /> mock_db = args[-1] <br /> # ... perform complicated setup on mock_db <br /> func(*args[:-1]) <br /> return wrapper</code></pre><br /><br /><h4>Cutting off</h4>The cut_off patcher is useful when you need to mock out several objects, and don't care about the mock instances. Say you want to deactivate internet access in a module that uses both urllibs.<br /><br /><pre><code>@cut_off("mymodule.urllib", "mymodule.urllib2")<br />def test_something():<br /> pass</code></pre><br />The implementation of cut_off is very simple too.<br /><br /><pre><code>def cut_off(*patches): <br /> def decorator(meth): <br /> def wrapper(self, *args): <br /> return meth(self, *args[:-len(patches)]) <br /> for obj in patches: <br /> wrapper = mock.patch(obj)(wrapper) <br /> return wrapper <br /> return decorator</code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-64828539394856493052010-06-07T02:46:00.000-07:002010-06-07T02:46:55.237-07:00Analyzing PyPI packagesPyPI, which stands for Python Package Index is a global repository for Python<br />
packages. Every time you need a Python tool or library, you can simply type<br />
<code>easy_install mypackage</code>, and have it downloaded and<br />
installed for you. It is also a great source when trying to investigate current<br />
practices in the Python world.<br />
<br />
<h4>Disclaimer</h4><br />
There are couple of troubles when analyzing PyPI. First - it is a moving target.<br />
Since I first run the download script (which was 3 days ago), it grew by 20 new<br />
packages. So, please bear in mind, this information won't very exact. Still, it<br />
provides a nice overview. Second - not all packages are hosted in PyPI. For some<br />
(quite a lot, actually) cases, we only get a link to the actual download source.<br />
This grows the chance of a host being, and causes the download to fail. Third -<br />
PyPI packages are terribly diverse. In order to analyze it in a timely manner, I<br />
picked only the ones that could be downloaded as either tarballs or zips. This<br />
reduced the sample by a quarter (from 10112 to 7625), which I believe is still a<br />
representative enough.<br />
<br />
<h4>Setup.py usage</h4><br />
Most of the packages (96%) used setup.py. The rest either simply didn't use it<br />
or used a non-standard directory layout (accordingly: 187 and 47). Out of<br />
setup.py users, setuptools was more than three time more popular than standard<br />
distutils. 73 packages couldn't be identified as using either of these, and this<br />
is mostly caused by custom setup function wrappers (see 4Suite for example of<br />
this).<br />
<br />
<img src="http://chart.apis.google.com/chart?cht=p&chs=350x150&chd=t:56,17,1&chl=setuptools|distutils|other"/><br />
<br />
<h4>Test runners</h4><br />
I was curious, how people run their tests, so I identified several ways it could<br />
be done:<ol><li>using a top-level shell script: 20</li>
<li>using a top-level python script: 326</li>
<li>using setuptools' test command: 961</li>
</ol><br />
<img src="http://chart.apis.google.com/chart?cht=p&chs=350x150&chd=t:96,32,2&chl=setuptools|python|shell"/><br />
<br />
Note: these stats don't include another popular way of running tests, used by<br />
Django apps.<br />
<br />
There where 1048 packages having a toplevel directory containing string "test",<br />
among which the most popular varations were unsurprisingly "test" (477) and<br />
"tests"(456).Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com7tag:blogger.com,1999:blog-7789059876927500089.post-18611768990864217422010-06-01T01:56:00.000-07:002010-06-01T01:56:34.790-07:005 things you can do with a Python list in one lineThis is directly inspired by an excellent post by Drew Olson <a href="http://drewolson.wordpress.com/2007/03/23/5-things-you-can-do-with-a-ruby-array-in-one-line-plus-a-free-bonus/">5 things you can do with a Ruby array in one line</a>. When reading it, I couldn't help but thinking of the Python versions (and how I like them more :>). So here it is:<br />
<br />
<ol><li>Summing elements<br />
<pre><code class="ruby">puts my_array.inject(0){|sum,item| sum + item}</code></pre><pre><code class="python">sum(my_list)</code></pre></li>
<li>Double every item.<br />
<pre><code class="ruby">my_array.map{|item| item*2 }</code></pre><pre><code class="python">[2 * x for x in my_list]</code></pre></li>
<li>Finding all items that meet your criteria.<br />
<pre><code class="ruby">my_array.find_all{|item| item % 3 == 0 }</code></pre><pre><code class="python">[x for x in my_list if x % 3 == 0]</code></pre></li>
<li>Combine techniques.<br />
<pre><code class="ruby">my_array.find_all{|item| item % 3 == 0 }.inject(0){|sum,item| sum + item }</code></pre><pre><code class="python">sum(x for x in my_list if x % 3 == 0)</code></pre></li>
<li>Sorting.<br />
<pre><code class="ruby">my_array.sort
my_array.sort_by{|item| item*-1}</code></pre><pre><code class="python">sorted(my_list)
sorted(my_list, reverse=True)</code></pre></li>
</ol>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com13tag:blogger.com,1999:blog-7789059876927500089.post-25530625279237468092010-05-28T03:24:00.000-07:002010-05-29T12:37:06.817-07:00Mockity mock mock - some love for the mock moduleIt looks like Python mocking libraries are the new web frameworks - <a href="http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#MockTestingTools">everyone wrote one</a>. Let me show you my favourite mocking library so far - the mock module, written by <a href="http://www.voidspace.org.uk/">Michael Foord</a>. It's easy_installable, so you can get to play with it in a moment.<pre><code>easy_install mock</code></pre>What makes it different than other modules like that? Most mocking libraries follow the old record-replay pattern, which works roughly like this:<ol><li>Teach a mock what to expect.</li>
<li>Stick it into your code and expect an explosion if something didn't go as planned.</li>
</ol><br />
The excercise-inspect pattern in mock module reverses that approach:<br />
<ol><li>Stick the Mock object into your code (Mock being the class provided by mock module).</li>
<li>Make sure that things happened the way you intended.</li>
</ol><br />
<br />
The tricky part is: how the hell is Mock able to fit everywhere? Well, it lets you do anything with it: get and set all attributes, and even call it when needed. And it leaves other Mocks everywhere it goes (it pretty much <a href="http://twitter.com/uninen/status/14769685164">works like cancer</a>). Let's play with Mock a little bit:<br />
<pre><code class="python">
>>> from mock import Mock
>>> m = Mock()
>>> m.foo = 1
>>> m.foo # all attributes are recorded
1
>>> bar_attribute = m.bar # if no attribute was set, a mock instance is returned
<mock.Mock object at 0xdcff0>
>>> m.bar == bar_attribute # an attribute that was queried once stays there
True
>>> ret_val = m() # we can even treat mock as a function
>>> m() == ret_val == m.return_value # with similar cacheing behaviour
True
>>> m.bar() # this is true of all methods as well (they're Mocks to, right?)
<mock.Mock object at 0xe4070>
>>> m.bar.called # mock remembers everything that happened to it
True
>>> m.baz.return_value = "hello!" # if we really need some setup
>>> m.baz() # nothing prevents us from doing it
'hello!'
>>> m.who.was.this.demeter.anyway.return_value.and_its_attribute # so just remember
<mock.Mock object at 0xe42f0>
>>> _ == m.who.was.this.demeter.anyway().and_its_attribute # there are Mocks all the way down!
True</code></pre><br />
Pretty cool, huh? But wait, there's more! Why should we laborously prepare, and then revert all the mocking ourselves? Let the library stick it where we need. We just need to import patch decorator.<br />
<br />
<pre><code class="python">import sys
from mock import patch
def read_4_chars():
return sys.stdin.read(4)
@patch("sys.stdin")
def test_something(mock_stdin):
mock_stdin.read.return_value = "abcd"
assert read_4_chars() == "abcd"
assert mock_stdin.read.called
assert mock_stdin.read.call_args == ((4,), {})
def test_something_2():
"""You can use it with *with* statement as well!"""
with patch("sys.stdin") as mock_stdin:
mock_stdin.read.return_value = "abcd"
assert read_4_chars() == "abcd"
assert mock_stdin.read.called
assert mock_stdin.read.call_args == ((4,), {})</code></pre><br />
There is much more that mock module has to offer. I suggest reading mock's <a href="http://www.voidspace.org.uk/python/mock/">excellent documentation</a>.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com4tag:blogger.com,1999:blog-7789059876927500089.post-11533700606101766032009-09-10T03:13:00.000-07:002009-09-10T03:21:03.198-07:00How to mount *.bin disk image on Linux/Mac (without *.cue file)First, you need <a href="http://he.fi/bchunk/">binchunker</a> - a utitily to convert the bin/cue pair into an iso image. It is available on macports and should be easy to get on most linux distros.<br /><br />Now, you need to prepare your *.cue file. Suppose the you have a "blah.bin" file. Enter following text into "blah.cue":<br /><pre><code>FILE “blah.bin” BINARY<br />TRACK 01 MODE1/2352<br />INDEX 01 00:00:00 </code></pre>Now, enter the following spell:<br /><pre><code>bchunk blah.bin blah.cue blah.iso</code></pre>There, you have it.<br /><br />Thanks to <a href="http://gavin.mclelland.ca/2007/10/04/convert_bincue_to_iso_on_mac_osx/">these</a> <a href="http://www.tech-recipes.com/rx/881/how-do-i-open-or-mount-a-bin-file-without-a-cue-file/">guys</a> for help.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-26606037245917659412009-05-14T02:51:00.000-07:002009-05-16T08:41:11.196-07:00Reporting assertions as a way for test parametrizationThere's been a discussion recently on Testing In Pyton mailing list on introducing test parametrization. Here's my approach to the problem.<br /><br />The idea, instead of finding a way to generate test methods, is to promote assertions to report the results in similar manner that testmethods do. In order to achieve that, I had to override few methods in TestCase, but the resulting API looks quite clean. <br />You can grab the source from <a href="http://code.google.com/p/konryd-scripts/source/browse/trunk/reporting_assertion/">here</a>. And here's an example. Run this:<pre><code>import unittest<br />from reporting_assertion import TestCase, reporting<br /><br />class MyTestCase(TestCase):<br /> @reporting<br /> def some_assertion(self, a, b):<br /> assert a == b<br /> def test_example(self):<br /> for x in (1, 2, 3):<br /> self.some_assertion(x, 2)<br /><br />if __name__ == "__main__":<br /> unittest.main()<br /> </code></pre>In order to get this report:<br /><pre>$ python example.py <br />FF.<br />======================================================================<br />FAIL: test_example.some_assertion(1, 2) (__main__.MyTestCase)<br />----------------------------------------------------------------------<br />Traceback (most recent call last):<br /> File "/[path]/reporting_assertion/reporting_assertion.py", line 12, in wrapper<br /> ret = assertion(self, *args, **kwargs)<br /> File "example.py", line 8, in some_assertion<br /> assert a == b<br />AssertionError<br /><br />======================================================================<br />FAIL: test_example.some_assertion(3, 2) (__main__.MyTestCase)<br />----------------------------------------------------------------------<br />Traceback (most recent call last):<br /> File "/[path]/reporting_assertion/reporting_assertion.py", line 12, in wrapper<br /> ret = assertion(self, *args, **kwargs)<br /> File "example.py", line 8, in some_assertion<br /> assert a == b<br />AssertionError<br /><br />----------------------------------------------------------------------<br />Ran 1 test in 0.001s<br /><br />FAILED (failures=2)</pre><br /><br />This approach has few drawbacks. You can't collect assertions when doing collect-only run, it's tightly coupled with TestCase implementation (needs to know how to report results), and - for the same reason - it works only for assertions bound to TestCase instance (the decorator has an optional _self= keyword argument for decorating unbound assertions, but the results are not reusable across TestCase instances). The last problem is __str__ evaluation: if the objects under have side-effects in this method, you might get quite unexpected results.<br /><br />It is, however quite powerful: the data fed into the assertions is generated in the code itself (so the parameters don't need to be static). It's also very easy to use: just decorate your assertion in order to get the reports.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-12034919495437302932009-02-09T16:03:00.000-08:002009-02-09T16:10:34.220-08:00QuoteI read <a href="http://www-cs-students.stanford.edu/~blynn/gitmagic/">git magic</a> today and found a link to the <a href="http://lkml.org/lkml/2005/4/6/121">original thread</a> on <a href="http://lkml.org/">LKML</a>. One of neat quotes in there is a line from Linus' post dated 8th of April 2005:<blockquote>"git" is really trivial, written in four days. Most of that was not actually spent coding, but thinking about the data structures.</blockquote>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-89957263307308661362009-02-02T08:09:00.000-08:002009-02-02T08:18:00.887-08:00Generic constructor assertionIt is quite common to write such initializers:<br /><pre><code>class Foo(object):<br /> def __init__(self, a, b):<br /> self.a = a<br /> self.b = b</code></pre>Why not test it in a generic way?<pre><code>import inspect<br /><br />def assert_simple_constructor(klass):<br /> allargs = inspect.getargspec(klass.__init__)[0]<br /> args = allargs[1:] # skipping self<br /> instance = klass(*args)<br /> for argname in args:<br /> assert hasattr(instance, argname) # skipped the message<br /> assert getattr(instance, argname) == argname # to keep short </code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com1tag:blogger.com,1999:blog-7789059876927500089.post-83804917474675223332009-01-25T15:17:00.000-08:002009-01-25T15:21:06.269-08:00Why "Why 'Why OO Sucks' Sucks"... exposes ignorance<p>A year and a half ago, I <a href="http://konryd.blogspot.com/2007/08/why-why-oo-sucks-sucks.html">wrote a blog post</a> which questioned some statements from Joe Armstrong's <a href="http://www.sics.se/~joe/bluetail/vol1/v1_oo.html">'Why OO Sucks'</a> article. While I still can't agree with all the article, I can see what he *really* meant and what I misunderstood then.</p> <p>So let's get back to the original article and its statements:</p> <dl><dt>1. Data structure and functions should not be bound together.</dt> <dd><p>My response to that went mainly along the lines of "but that's how people tend to think, isn't it". However, what I think now is... well why not? Even in functional languages functions that handle specific types of data know the details of the data, so in order to understand the function you need to know what the data looks like. In order to use that function you need to feed it with the specific type of data it needs. Not so separate anymore?</p> <p>Binding data and functions in OO languaged serves mainly two purposes: either to give some specific view or attribute of an instance (in which case they're nothing more than a convenient interface pure function calls) or to change the instance's inner state (to which I'll get back later).</p></dd> <dt>2. Everything has to be an object.</dt> <dd><p>Again, I said it's comfortable and natural to consider data as objects, but what I didn't know by then was the difference between classes and data types. I didn't know yet about pattern matching and other ways of working with immutable data. All those techniques make working with it equally comfortable and natural (maybe in more mathematical spirit, but I doubt it should be a disadvantage for anyone).</p> <p>A little wiser now, I'm quite convinced that this argument comes down again to objects having private state. If there was an OO language where all objects would have to be immutable, I think the need of turning every piece of data into an object wouldn't be so painful.</p> <dt>3. In an OOPL data type definitions are spread out all over the place.</dt> <dd><p>OK, here I was silly. First, I wrote "OO laguages" while *clearly* thinkng "Java" (well, that was the single OO language I knew then). Second: I totally forgot about inheritance.</p> <p>What I think now is that there's no much difference here between OO and functional languages. If you consider inheritance as a specific case of composition (which it is) you get an equivalent of nested data types with functions handling one type are using other functions to handle the other. Not much difference from functional languages.</p></dd> <dt>4. Objects have private state.</dt> <dd><p>Ah, so here we have it. After playing a bit with functional languages I learnt that state, while unavoidable, is in fact a nasty bastard. Joe Armstrog doesn't like the way the state problem was solved in OO languages. They go along the lines of "yeah... we have state, but don't mind it. As long as you give us correct requests you'll be fine", which isn't true because giving correct requests (calling correct methods) requires to know the state in the first place.</p> <p>The functional languages tend to use pure functions what lets them access the state only when neccessary (and FPL desingers solve the state problem in several different interesting ways).</p> <p>To be honest. I kind of agree with that now. I don't like mutable objects. You never know what to expect from them :)</p> <p>Ah, and my arguments here were also quite silly. First: I felt already that state is not so good, but only in the context of multithreading. Second: I was a fan of information hiding and working with tightly encapsulated data. While I'm not saying that encapsulation is bad (on the contrary!), I think I understand it differently now.</p> <p>Encapsulation is not putting something in black box and forcing others not to see the its inner parts. It's rather finding a "sweet spot" in control flow: the interface that would be convenient enough for anyone to use without a need to look deeper. In other words: I don't associate encapsulation with information hiding anymore.</p></dd> </dl> <p>Concluding this longish article: I still don't agree with the statements of Joe Armstrong's article - in particular: the quiet assumption that a good language is a functional language. But I have to admit that my former article was based mainly on ignorance and misunderstanding Joe Armstrong's arguments.</p> <p>And I still don't like the conspiration theory in the last paragraph there :)</p>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-17859022997536907082009-01-19T15:30:00.000-08:002009-01-19T15:31:53.227-08:00ilen for PythonToday I came across what turned out to be not-so-common need. I wanted to get the length of the generator. Before you comment - yes, I know generators can yield infinite sequences, and that might be one of the reasons why such a function is nowhere to be found. However, if one knows what one's doing it can be of some use. After all, calling list() constructor on iterables is equally risky. Of course, the function is trivial, but managed to give me that that-must-be-somewhere-already feeling. It wasn't :( <pre><code>def ilen(sequence):<br /> result = 0<br /> for _ in sequence:<br /> result += 1<br /> return result</code></pre> BTW. I think itertools is my favourite Python module, what's yours?Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com8tag:blogger.com,1999:blog-7789059876927500089.post-25792647863934742402009-01-01T15:10:00.000-08:002009-01-03T06:57:01.045-08:00Howcome so many 8s?Below, you can see plot of the results of the following code:<br /><pre><code>import sys<br />for x in range(200):<br /> print x, sys.getrefcount(x)<br /></code></pre>I read recently that integers less than 256 are being interned by CPython interpreter, so I took a look at how they are used. First 80 integers has about 20 refs, with much bigger number for the initial couple of numbers. Then the refcount is usually less than 10. For some reason, there is a whole lot of references to the number 8.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_HenNjwjePPw/SV1OexvXelI/AAAAAAAACss/GpvuaLNHN8o/s1600-h/Obrazek+1.png"><img style="display:block; margin:0px auto 10px; text-align:center;cursor:pointer; cursor:hand;width: 320px; height: 241px;" src="http://2.bp.blogspot.com/_HenNjwjePPw/SV1OexvXelI/AAAAAAAACss/GpvuaLNHN8o/s320/Obrazek+1.png" border="0" alt=""id="BLOGGER_PHOTO_ID_5286467828257880658" /></a><br /><br />EDIT: I didnt' notice the 8's leap on debian machine. It's reproducible though on my mac.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com1tag:blogger.com,1999:blog-7789059876927500089.post-32236397479271430822008-11-27T05:49:00.000-08:002008-11-27T05:53:02.382-08:00Rails tip - how to see clearly the templates rendered for a query?Not *that* inventive, but I somehow didn't think of it before:<br /><pre><code>tail -f log/development.log | grep Rendered</code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-24591131914814711732008-10-08T00:04:00.000-07:002008-10-10T02:54:21.653-07:00Getting svn:externals to work with git-svn and... LeopardThere IS a simple solution to work conveniently in git-svn with repository that has svn:externals set. You can clone all repositories to seperate trees, and instead of watching svn:externals, make hard links to the trees you need (remember to ignore them in git). The problem is you can't have hard links to directories... unless you're using OS X Leopard (they implemented it in order to get Time Machine to work).<br /><br />This ability isn't exposed to the command line (ln makes an explicit check to prevent you from doing that), but the <a href="http://www.mactech.com/articles/mactech/Vol.23/23.11/ExploringLeopardwithDTrace/index.html">code to have it working(scroll down to listing 4)</a> is trivial.<br /><br />Update: When working with hard-linked directories, remember to be careful when unlinking them: rm wouldn't unlink the directory unless provided with "-r" option, which deletes everything recursively, leaving other hard links to that directory empty.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-92068495137809006182008-09-23T01:36:00.000-07:002008-09-23T01:46:19.430-07:00Harness the power of wgetI finally invested enough time in reading wget's man page to be able to download file hierarchies exposed through http file server. The magic command is:<br /><code>wget -r -np -nH --cut-dirs=2 http://example.com/some/nested/directory</code><br />The switches mean:<dl><dt>-r</dt><dd>turns of recursive download</dd><dt>-np</dt><dd>no-parent - only links to files below in the hierarchy are followed</dd><dt>-nH</dt><dd>no-host - doesn't dump the files into directory named after host name</dd><dt>--cut-dirs=2</dt><dd>in addition to skipping host name skips also 2 topmost directories. In the example instead of <i>some/nestes/director</i>y the files are put straight into <i>directory</i></dd></dl>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-34684139009827963382008-05-21T10:42:00.000-07:002009-01-19T15:29:19.146-08:00Getting fresh datastore while testing GAE appsIf you want to find out how to test you GAE apps, check out these two URLs first:<br /><a href="http://groups.google.com/group/google-appengine/msg/9132b44026040498">http://groups.google.com/group/google-appengine/msg/9132b44026040498</a><br /><a href="http://farmdev.com/thoughts/45/testing-google-app-engine-sites/">http://farmdev.com/thoughts/45/testing-google-app-engine-sites/</a><br /><br />I want to show you how to clear the datastore quickly (yep, only that :>):<br /><pre><code><br />from google.appengine.api import apiproxy_stub_map<br /><br />def clear_datastore():<br /> datastore = apiproxy_stub_map.apiproxy.GetStub('datastore_v3')<br /> datastore.Clear()<br /></code></pre>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com1tag:blogger.com,1999:blog-7789059876927500089.post-52828101640456946802008-04-29T05:40:00.000-07:002008-04-29T05:58:43.096-07:00Outputting pdfs with google app engineIt's child easy, but not googleable yet. I used reportlab - a handy, pure-python pdf library. Here are the steps:<ol><li>Checkout reportlab into your app directory<pre><code>svn co http://www.reportlab.co.uk/svn/public/reportlab/trunk \<br />reportlab</code></pre></li><li>Write some pdf-generation code, like<pre><code>import wsgiref.handlers<br /><br />from google.appengine.ext import webapp<br /><br />from reportlab.pdfgen import canvas<br /><br /><br />class MainPage(webapp.RequestHandler):<br /><br /> def get(self):<br /> self.response.headers['Content-Type'] = 'application/pdf'<br /> p = canvas.Canvas(self.response.out)<br /> p.drawString(100, 750, "Hey, it's easy!.")<br /><br /> p.showPage()<br /> p.save()<br /><br /><br /><br />def main():<br /> application = webapp.WSGIApplication([('/', MainPage)], debug=True)<br /> wsgiref.handlers.CGIHandler().run(application)<br /><br />if __name__ == "__main__":<br /> main()<br /></code></pre>The key is the canvas instantiation. The constructor takes a file-like object and writes into it. You might have noticed that the code is heavily inspired by <a href="http://www.djangoproject.com/documentation/outputting_pdf/">this tutorial</a>. The only difference is the creation of canvas.</li><li>That's it. See? I said it was easy. You can run the dev_appserver or upload it to Google in order to see the results.</li></ol>Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com8tag:blogger.com,1999:blog-7789059876927500089.post-30186848444230477732008-03-13T17:28:00.001-07:002008-12-10T03:42:09.518-08:00At PyCon<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://1.bp.blogspot.com/_HenNjwjePPw/R9nG0rhvKiI/AAAAAAAABFM/LmhERCVWBXE/s1600-h/Zdj%C4%99cie+12.jpg"><img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;" src="http://1.bp.blogspot.com/_HenNjwjePPw/R9nG0rhvKiI/AAAAAAAABFM/LmhERCVWBXE/s320/Zdj%C4%99cie+12.jpg" border="0" alt=""id="BLOGGER_PHOTO_ID_5177387854978820642" /></a><br />Yaaaay, after 9h flight, finally here.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com0tag:blogger.com,1999:blog-7789059876927500089.post-56917238821256408752008-02-16T11:27:00.000-08:002008-02-16T11:40:03.761-08:00Brand new blog engine without a single line of codeHaving investigated the new Google D&S forms feature, I decided to create a proof-of-concept blog application (OK - calling it an application is a *bit* an overstatement). It has some basic features, like form for entering entries and a feed. I doesn't have a site and is pretty much crippled. Still, quite good for 0 lines of code.<br /><br /><a href="feed://spreadsheets.google.com/feeds/cells/o11126070771619719026.3749542711380394284/od6/public/basic?range=B2:B90&alt=rss">Here's the feed</a><br /><br />And here's the <a href="http://spreadsheets.google.com/viewform?key=pcuc_GGu-Z5oswfvlOg0JNA">new entry form</a>. Feel free to post.Konradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com1tag:blogger.com,1999:blog-7789059876927500089.post-36856382233490052352008-02-03T05:00:00.000-08:002008-02-04T15:02:06.574-08:00Implementing Arc's function-negation operator in PythonHey look! <a href="http://4.flowsnake.org/archives/53">This guy</a> implemented Arc's function-negation operator in Chicken. And he claims to be a Pythoneer, traitor ;-) !<br /><br />I just investigated the possibilities of doing more or less the same thing in Python. Of course: you can't change the syntax, and you can't change default function's behaviour (which is not having "~" operator implemented). Or can't you?<br /><br />Let's start with something more explicit for a moment. Nothing prevents us from creating a callable decorator that implements __invert__ something like this:<pre><code>class invertible:<br /><br /> def __init__(self, foo):<br /> self.foo = foo<br /><br /> def __call__(self, *args, **kwargs):<br /> return self.foo(*args, **kwargs)<br /><br /> def __invert__(self):<br /> def wrapper(*args, **kwargs):<br /> return not self.foo(*args, **kwargs)<br /> return wrapper<br /></code></pre>Implementing __init__ enables us to call invertible() with an argument, and implementing __call__ makes the instances of our class callable. These two work together in a way that an ordinary decorator works: It's a callable, that takes a callable and returns another callable - easy, huh?<br />Implementing __invert__ lets us use the "~" operator on the instances of our class, like this:<pre><code>truth = invertible(lambda: True)<br />not_truth = ~truth<br />assert truth()<br />assert not not_truth()</code></pre>This implementation, even though it's easy, is not finished yet. The inverted value is a regular function, which means, that we couldn't invert it again. The easy solution to that is decorating the wrapper:<pre><code>class invertible:<br /> # ...<br /> def __invert__(self):<br /> @invertible<br /> def wrapper(*args, **kwargs):<br /> return not self.foo(*args, **kwargs)<br /> return wrapper</code></pre>But this would cause the wrappers to accumulate on inverting. We can also check explicitly if we get an invertible on input and set a switch on each instance<pre><code>class invertible:<br /><br /> def __init__(self, foo):<br /> self.foo = foo<br /> self.inverted = False<br /><br /> def __call__(self, *args, **kwargs):<br /> if self.inverted:<br /> return not self.foo(*args, **kwargs)<br /> else:<br /> return self.foo(*args, **kwargs)<br /><br /> def __invert__(self):<br /> ret = invertible(self.foo)<br /> ret.inverted = (not self.inverted)<br /> return ret</code></pre>Now, with the decorator ready, you can do some *evil* magic with frames and namespaces in order to decorate everything in scope, but I'm leaving it as an exercise to readers ;PKonradhttp://www.blogger.com/profile/14388784103552736162noreply@blogger.com5