Home » Odeon Blogs » Calvin, Weaver »

Pypy on virtualenv and a need for speed

25 Jul 0

Pypy on virtualenv and a need for speed

Pypy is a high performance python interpreter written in python (for Python 2.7.1 at this point in writing). In contrast, the original (standard) python interpreter is written in C, commonly referred to as CPython.

So why would we use Pypy instead of the standard CPython?

Performance and speed = faster Python apps!

Pypy works as a just-in-time compiler. It looks for some bits of code that are executed often and optimize those bits into assembly.

Ok. Sounds interesting, but just how fast is fast?

Let's take a look at the performance benchmark comparing django on pypy against django on standard python. The performance contrast is stunning - more than 0.5 seconds faster in absolute terms for this benchmark. For a more generic overview of pypy's performance, check out http://speed.pypy.org/.

Why is 0.5 seconds important to us as application developers or product owners? Do we really care about optimizing for 0.5 seconds?

In a talk at Web 2.0 Conference in 2006, Marissa Mayer, head of usability at Google, spoke at length about their own experiences relating to user experience and web application speed - http://www.zdnet.com/blog/btl/googles-marissa-mayer-speed-wins/3925.

Long story short, it is discovered that:
For Google an increase in page load time from 0.4 second to 0.9 seconds decreased traffic and ad revenues by 20%. For Amazon every 100 ms increase in load times decreased sales by 1%.

So yes, as software professionals working on large scale apps, we care about implementing (web) apps that are speedy from an end-user perspective. By the way, this is where some properly implemented native apps (apps built specifically for mobile devices like the iPhone or Android or for your Desktop computer) has the advantage (at the moment). But that's a story for another day.

Usability and User Psychology

To further emphasize usability issues in relation to app performance and speed, here's a 1993 article (o yea, ancient by internet standards :-) ) - http://www.useit.com/papers/responsetime.html. Quote:

0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result.
1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data.
10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect.

Now that you are convinced (I hope) that it is worthwhile to experiment with pypy purely from a speed/performance gain perspective and convince your pointy-haired boss to let you try out some of your Python apps on pypy, here's a quick run down on how we can get pypy running on your machine:-

1. Grab the pypy trunk (Yes, I like bleeding edge software) and build from source

$ cd ~/work
$ git clone https://github.com/pypy/pypy.git

2. Compile pypy on your machine to create the pypy-c binary

Be sure that you have Python 2.7 already installed in the first place. We will need Python 2.7 to run translate.py as seen below:-

$ cd ~/work/pypy/pypy/translator/goal
$ python translate.py --opt=jit targetpypystandalone.py

This is going to take a while (probably a lot more than 30 minutes). So go take a walk :-)

3. Use pypy in an isolated python environment with virtualenv

As pypy is a lot more bleeding edge and experimental and may not work with python libraries with c extensions, let's do ourselves a favour and keep pypy in an isolated environment where we can experiment to our heart's content without worrying about it messing with our python libraries already installed in our global site-packages.

If you do not have virtualenv installed, it's as easy as running "sudo port install py27-virtualenv" on your Mac (or equivalent commands on your preferred linux distro).

Once that's done, we can now run:-

$ cd ~/work/
$ mkdir env
$ cd env
$ virtualenv -p ~/work/pypy/pypy/translator/goal/pypy-c --no-site-packages --distribute pypy
Running virtualenv with interpreter /Users/calvin/work/pypy/pypy/translator/goal/pypy-c
New pypy executable in pypy/bin/pypy-c
Also creating executable in pypy/bin/pypy
Installing distribute............................................................................done.
Installing pip...............done.

4. And checking

$ source pypy/bin/activate
$ python
Python 2.7.1 (6f7e32a3d998, Jul 24 2011, 00:39:56)
[PyPy 1.5.0-alpha0 with GCC 4.0.1] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``topics are for the feeble
minded''
>>>> exit()

And now, we can experiment with any python apps we like in this isolated python environment using pypy!

And to get out of your pypy-c python environment at any point in time, we simply run "deactivate".

And in case you were wondering what the image above is all about, it is a picture depicting a snake eating its own tail - i.e. ouroboros. Since pypy is Python written in Python (Restricted Python - RPython - to be exact), that's the logo that the pypy developers use!

Category: Python

Tagged as: pypy

Pypy on virtualenv and a need for speed