Stefan Talpalaru, CTO

If you have a Canon IXUS 220HS (ELPH 300HS) camera with the 1.01G firmware version, this is your lucky day: now you can run CHDK on it.

Category: CHDK

We all know that Skype is a terrible piece of software, with the Linux version buggy and left behind the main code base, but lately it's been crashing like a Top Gear funny men in an overpowered car. Each time it starts and there are chat windows with new messages to be shown, the damn thing segfaults (at least on 64 bit, statically linked against Qt). While trying to find more details about the problem, I stumbled upon an unlikely fix: running skype under gdb seems to stabilize it. Is it the delay introduced by the debugger interfering with some concurrency bugs or some other mechanism at work? No idea. Just that this is how I run the rubbish right now:

LD_LIBRARY_PATH="/opt/skype" gdb /opt/skype/skype

and then type 'r' in the debugger. It hasn't crashed once under gdb so there's no stack trace available.

Category: Linux

If you enable the SandyBridge New Acceleration (SNA) in the xf86-video-intel driver on a GM965 chipset with a dual-monitor setup your GPU will hang with the kernel DRM driver crying like a little girl and both screens going blank. While the rest of xorg is still working and your current session is accessible through x11vnc, you will need to reboot to restore the video output.

Category: Linux

One of the first things you learn about optimizing web applications is how to cache processed data in memory. Just install memcached - the easy to use key-value store - and you're set. Until you reach its limits, that is...

Memcached does not allow keys larger than 250 bytes or values bigger than 1MB. What happens when you really need bigger keys/values? You move on to redis.

If you are using a proper framework (and you should) there's probably already a redis plugin for you (here's the Django one). Now all you have to do is install redis and configure it to suit your caching needs. As a rule of thumb, speed and low resource usage are more important than data persistence. Here's my configuration (only the relevant differences from the default redis.conf):

daemonize yes
bind 127.0.0.1
loglevel notice
logfile /var/log/redis/redis.log
dir /var/lib/redis/
maxmemory 67108864

Category: caching

Starting with django-1.3.1 you'll get redirect failures for reverse proxy setups (cherokee and cherrypy in my case). The reason is that relative paths that you feed to HttpResponseRedirect are converted into absolute URLs using META['HTTP_HOST'] which is actually the IP of the reverse proxy. To fix this add the following line to settings.py:

USE_X_FORWARDED_HOST = True

More details in the release notes.

Category: Django

I don't know how high ranking python is for automating system administration tasks but when I had to script remote ssh commands, I thought I'd give python a try. The main SSH library is paramiko and, while lacking in the documentation department, it's rather rich in features.

Besides the usual root login, I had a more difficult use case: using a regular user with full sudo rights instead of the superuser. These are the functions I came up with:

import paramiko
import getpass

def ssh_connection(user, host, port=22, password=None, key_filename=None):
    """
    with password='' you will be prompted for a password when the script runs
    """
    ssh = paramiko.SSHClient()
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
    if password == '':
        # ask for one on stdin
        password = getpass.getpass('Password for %s@%s: ' % (user, host))
    ssh.connect(host, port=port, username=user, password=password, key_filename=key_filename)
    # custom attributes
    ssh.user = user
    if user == 'root':
        ssh.homedir = '/root'
    else:
        ssh.homedir = '/home/%s' % user
    ssh.password = password
    ssh.use_sudo = False
    return ssh

def run_remote(ssh, cmd, check_exit_status=True, verbose=True):
    chan = ssh.get_transport().open_session()
    stdin = chan.makefile('wb')
    stdout = chan.makefile('rb')
    stderr = chan.makefile_stderr('rb')
    processed_cmd = cmd
    if ssh.use_sudo:
        processed_cmd = 'sudo -S bash -c "%s"' % cmd.replace('"', '\\"')
    chan.exec_command(processed_cmd)
    if stdout.channel.closed is False: # If stdout is still open then sudo is asking us for a password
        stdin.write('%s\n' % ssh.password)
        stdin.flush()
    result = {
        'stdout': [],
        'stderr': [],
    }
    exit_status = chan.recv_exit_status()
    result['exit_status'] = exit_status
    def print_output():
        for line in stdout:
            result['stdout'].append(line)
            print line,
        for line in stderr:
            result['stderr'].append(line)
            print line,
    if check_exit_status and exit_status != 0:
        print_output()
        print 'non-zero exit status (%d) when running "%s"' % (exit_status, cmd)
        exit(exit_status)
    if verbose:
        print processed_cmd
        print_output()
    return result

# set up a ssh connection for root and ask for the password interactively
myconn = ssh_connection('root', 'example.com', password='')
# run some commands with the default settings
run_remote(myconn,
          """
           pwd
           cd /var/log
           pwd
           ls -la
          """)

# set up a connection for a regular user with full sudo rights and use it for running commands with root privileges
sudoconn = ssh_connection('jim', 'example.com', password='imanewbieandileavepasswordsintextfiles')
sudoconn.use_sudo = True
run_remote(sudoconn,
          """
           whoami
           pwd
           cd /var/log
           pwd
           ls -la
          """)

Category: Python

Flickr's flash uploader has a little problem: it will fail to upload certain files. I can't find a pattern here - all the files come from the same digital camera. Anyway, the problem I was facing was finding out which of the photos in a certain directory need to be uploaded again (and do that by other means than the buggy flash object). The photos already uploaded where added to  certain set so getting that info shouldn't be complicated, right?

It seems that that folks writing software with upload functionality haven't taken this case into consideration (at least for the Linux software I've tried), so I had to write my own. Since python is my preferred language right now, I located a Flickr API for it (named flickrapi, oddly enough) and an ebuild for the latest version. Here's the quick and dirty result:

#!/usr/bin/env python

import flickrapi
import os
import sys

api_key = 'your own'
api_secret = 'your own'

flickr = flickrapi.FlickrAPI(api_key, api_secret)

(token, frob) = flickr.get_token_part_one(perms='write')
if not token:
    raw_input("Press ENTER after you authorized this program")
flickr.get_token_part_two((token, frob))

photosets = flickr.photosets_getList()

def update_photoset(photoset_id, photo_dir, max_uploads=None):
    """
    set max_uploads to 1 while debugging to upload only one photo per run
    """
    uploaded_photos = []
    disk_photos = []
    to_upload = []

    for photo in flickr.walk_set(photoset_id):
        uploaded_photos.append('%s.JPG' % photo.get('title'))

    for photo in os.listdir(photo_dir):
        disk_photos.append(photo)
    disk_photos.sort()

    for photo in disk_photos:
        if photo not in uploaded_photos:
            to_upload.append(photo)

    i = 0
    for photo in to_upload:
        photo_path = os.path.join(photo_dir, photo)
        print photo,
        sys.stdout.flush()
        result = flickr.upload(filename=photo_path, is_public=u'0', is_family=u'1', is_friend=u'1')
        print
        sys.stdout.flush()
        photo_id = result.find('photoid').text
        flickr.photosets_addPhoto(photoset_id=photoset_id, photo_id=photo_id)

        i += 1
        if max_uploads and i == max_uploads:
            break

if __name__ == '__main__':
    update_photoset(u'12345678901234', '/home/bart/photos/2011-07-24', max_uploads=None)

Category: Python

In order to use GeoDjango I had to install postgis on a Gentoo unstable system. This is how I did it (if I did it).

First, I had to fix the postgis-1.5.2 ebuild. You can get it from here. Corresponding bugzilla item here.

After installing the ebuild in my custom overlay, I ran the following commands in a root shell:

emerge postgis
echo "template_postgis" >> /etc/conf.d/postgis_dbs # default GeoDjango template. used for tests
echo "my_db" >> /etc/conf.d/postgis_dbs # the existing db used by my Django project
emerge --config postgis

Categories: Django Gentoo

If you've noticed that the bash that comes with OS X is outdated and you're bothered by it, this post is for you. First, install macports if you don't have it already. Open a terminal (I prefer iTerm but the default one will do just fine) and get a root shell:

$ sudo -s

# port -cv install bash

# echo /opt/local/bin/bash >> /etc/shells

chsh -s /opt/local/bin/bash

# chmod u+w /etc/bashrc

[ -f "$HOME"/.bashrc ] && . "$HOME"/.bashrc

Category: OS X

If your browsing habits are anything like mine, then you've collected a lot of open tabs in Firefox. The browser starts slower because it needs to load them all, runs a bit slower and uses a lot of RAM. Wouldn't it be nice to postpone the tab loading until you click on it? BarTab does just that. Besides the default settings, it also allows you to delay the loading of background tabs (useful if you middle-click a bunch of links and don't want the CPU/RAM usage to spike) and to unload background tabs after a certain interval (with a white-list of excepted sites).

Category: Firefox