Unpacking "some" list elements in Python

Programming is not only about efficiency or correctness but also about style. Well, it is for me! Now "my" style is really not that great but from time to time I try to improve it. I may want to make my code more readable, more mathematically oriented or even just more elegant.

Basic unpacking

This is why I was searching for an elegant way to extract some data from a list and leave the rest untouched. Python offers the possibility of unpacking data from a list or tuple. In the code below you'll find the basic unpacking use. For this to work you have to know how many elements there are in your list and assign a variable to each of those elements.

list = [1, 2, 3, 4]
a, b, c, d = list

I believe this is a lot better than the code below which has exactly the same effect but does not make use of the unpacking functionality.

list = [1, 2, 3, 4]
a, b, c, d = list[0], list[1], list[2], list[3]

However this does not solve my initial problem of unapacking only part of the list and leaving the rest as is. Below I'll show you how to do just that in Python 3.x and in Python 2.x.

Catch-all unpacking in Python 3.x

With Python 3.0 came the extended iterable unpacking which brought an elegant way to extract needed data into variables and also specify a "catch-all" variable that will take the rest.

Let's suppose we want to work with the first two elements of the list and leave the rest as a list for future use. In versions 3.x you can do that as shown in the two lines of code that follow.

list = [1, 2, 3, 4]
a, b, *rest = list

The variable a and b will point to the first and second elements respectively. While the variable rest will reference the rest of the list. The "catch-all" variable is marked with an asterisk. This is elegant and concise. Exactly what I was looking for.

However life is not always that easy. While Python 3.x brings lots of new and interesting features it is not backward compatible with python 2.x. Using the Django web development platform I am one of those poor souls stuck in version 2. So let's see what we can come up with in this version.

Catch-all unpacking in Python 2.x

The best way (as in more elegant) I found to replicate the snippet above in Python 2.x is to make use of slices. In the code sample that follows I'll show you how I do it when I want to unpack just the head of the list or more elements because the method is slightly different.

list = [1, 2, 3, 4]
(a, b), rest = list[:2], list[2:] # The first and second elements are interesting.
a, rest = list[0], list[1:]       # Only the head is interesting.

I have to admit I am disappointed with what I could come up with. However I can't think of any other way to do this that would be more elegant. the second case is just a normal assignment with the use of a slice in the right part of the assignment. While the first case is trickier and manages to actually use the unpacking functionality it still lacks clarity. Besides the use of slices only works with list and not with all iterable objects. I can't wait for Django to switch to Python 3.x!

I would be very interested to hear how you would implement the small code pattern above.

Python IDE: the Pydev plugin for Eclipse

Choices for a good open source Python IDE are not many! Today I'll show you how to quickly have a basic setup working with Pydev for Eclipse. Since the last post about SFTP and FTP support in Eclipse. You already know it is my platform of choice.

Pydev is a product by Aptana which also provides the Aptana Studio software for working with web development (Ajax, Ruby, PHP, etc...).  The plugin as all the usual features you would expect from a normal IDE and also some goodies like Django integration. Instead of a lengthy discussion on the pros and cons of this setup. Let's just go ahead with installing and configuring it so you can judge for yourself.

Installing the Pydev plugin.

To install install the plugin you obviously need to first have Eclipse. I generally start with Eclipse Classic but that will pretty much depend on your preferences... From Eclipse follow theses steps to get the plugin:

1. Go the Help menu and click Install New Software.
2. On the work with text box insert http://pydev.org/updates and click Add.
3. You  can insert a name for this update site and wait for Eclipse to check the available content.
4. Select Pydev and hit Next two times.
5. Accept the license agreement if that's ok with you and click Finish.

After restarting Eclipse the plugin should be installed and ready to be configured.

Configuring the Pydev plugin.

To get started on programming with Pydev you now need to tell it where to find the interpreter:

1. Open the Preferences window. Go to Window then preferences.
2. Next you expand Pydev and select Interpreter - Python.
3. On the right pane you can select Auto Config if your system path is properly set. Otherwise you might have to select your interpreter and libraries manually.

This is all that is needed for a basic setup. You can start exploring you newly installed Python IDE. Any suggestions are welcome.

FTP/SFTP support in eclipse with Remote System Explorer

I got tired of using an FTP/SFTP client and switching from it to my IDE. So I went on looking for some FTP and SFTP plugins for Eclipse.

I've seen the Aptana Studio or plugin for Eclipse being recommended and tried to use it. Works ok but they only support SFTP in the professional version. Because of that I tried Remote System Explorer. I was quite happy to discover that not only it supports FTP and SFTP but you actually can also open a ssh terminal or issue ssh commands from Eclipse directly.

Install the Remote System Explorer plugin

First you need to install the plugin. With version 3.5.2, you can follow these steps:

1. Go to Help -> Install new software
2. From the "work with" drop-down list select: "Galileo - http://download.eclipse.org/releases/galileo"
3. In the pane below, expand General Purpose Tools and select the Remote System Explorer end user runtime. 
4. Click Next and follow the instructions.

Create a SFTP connection

Once installed you can start creating connections. Just do as follows:

1. switch to Remote System Explorer perspective.
2. Right-click on the Remote Systems view on the left and choose new -> connection.
3. Choose FTP only if that's what you need or SSH only if you want SFTP support as I do.
4. Specify the Host Name and the Connection Name.
5. Finally keep clicking on next until finished.

Tadam! You should have your remote system on the left pane. From there you can start browsing your files or opening an ssh terminal to the remote system. Of course you don't need to be in the RSE perspective. You can switch to any perspective you like and and open the necessary views through Window -> Show View, etc.

Happy Coding!

Django whois using the subprocess module

Last month I wrote a post about writing a whois client. This time I want to investigate If it's possible to leverage your operating system for those kind of tasks or if you really need to reinvent the wheel.

So why did I wrote my own crappy whois client in the first place? As you may already have noticed from my blog it's because I want to use the whois client from within Django (web application framework) hence the need to have a python module that can do whois queries.

Ok so the need is pretty clear but there should be a way to call the whois client of my operating system from a python module instead. This would allow me to write less code and I'll probably end up with a more robust whois client. So what possibilities does Python offer to spawn a process and communicate with it?

Since version 2.4 there is the subprocess module. In essence this is exactly what I need. So let's see how we can use it from within Django to provide a web interface to the whois native client.

from django.shortcuts import render_to_response
from django.http import HttpResponse
from django import forms
import subprocess

def whois(domain):
    p = subprocess.Popen(['whois', domain], stdout=subprocess.PIPE)
    answer = p.commmunicate()
    return answer

class WhoisForm(forms.Form):
    domainname = forms.CharField(max_length=100)

if 'whois' in request.POST:
    whois_form = WhoisForm(request.POST, prefix='whois')
    if whois_form.is_valid():
        domainname = whois_form.cleaned_data['domainname']
        answer = whois(domainname)
else:
    whois_form = WhoisForm( prefix='whois')

return render_to_response('index.html', {'whois_form': whois_form,
                                         'answer': answer,})

What happens in the code above is that we spawn the whois subprocess and pipe its output. This enables us to communicate with it and retrieve it's standard output in a variable. We can then pass along the output to the template for the user to view in his browser. Quick and easy!

Ok, so at this point I should be a happy man! I did reach my initial goal of not reinventing the wheel. However  I'm really not sure this is the way to go. Next time we will use the timeit module to see how this one compares to the python whois client regarding performance.

Javascript debugging

This week-end I had a hard time debugging some javascript code and could hardly find any time for posting. While we wait for the next post I wanted to share a good article I found about javascript debugging:

    http://www.alistapart.com/articles/advanced-debugging-with-javascript/

Debugging javascript can be a real pain. Please feel free to share your tips, techniques and tools in the comments section.

Multicolor jQuery Accordion widget

I received a question about having different elements in a jQuery accordion with different colors. There are probably many ways to achieve this. Below I'll show you how to do it with just a little bit of javascript.

$(document).ready(function(){
    $("#accordion").accordion();
    var my_colors = ["aqua", "black", "blue", "fuchsia", "green", "lime"];
    $('.ui-accordion-header').each( function(i) {
        $(this).css("background-color",my_colors[i])
    });
});

What is important to note here is the use of the css classes added by jQuery to an accordion to retrieve all accordion headers. I hope I answered the question... If I can find the time I'll try to put a polished demo on the web with complete source code.

A collection of jQuery Modal Dialog Boxes plugins

Today I was searching for a good Modal dialog Boxes plugin for jQuery in an attempt to improve the overall look and feel of my current project. After some time I ended up on 19 jQuery Modal Boxes to improve your UI.

Since I know how frustrating this kind of search can become. I thought I'd better share this collection with you.

Multiple Django forms in the same view

When you want to handle multiple Django forms within the same view you need some extra coding as opposed to having a single form. This is needed because:

• You want to identify which form has been submitted to your view.
• And you don't want to confuse fields from one form with fields from another.

So let's start with identifying the form from your view. You'll first need to add some information to your template to distinguish between the two forms. This can be easily done by giving a unique name to each submit buttons. For the rest of this post we will assume that we have two forms: A and B.

<fieldset>
    <legend>Form A</legend>
    <form method="post">
        {{ A_form.as_p }}
        <input name="A" type="submit" value="Submit" />
    </form>
</fieldset>

<fieldset>
    <legend>Form B</legend>
    <form method="post">
        {{ B_form.as_p }}
        <input name="B" type="submit" value="Submit" />
    </form>
</fieldset>

Now that you uniquely identified both forms you can add some logic to your view to get to know which submit button as been pressed.

if 'A' in request.POST:
    # A form processing
elif 'B' in request.POST:
    # B_form processing

What happens above is that you search the submitted POST data dictionary for the name you gave to your submit buttons. That's all there is to it.

However if for some reason fields in the different forms you present to your users have the same names then you need to add a couple more lines of code to distinguish between them. From the django documentation you can borrow the strategy in using more than one formset in a view. The idea is to prefix each field name with a form identifier. I choose to use the same token for the prefix and for the submit button name.

The end result would be something like this:

class AForm(forms.Form):
    fieldname = forms.CharField(max_length=25)

class BForm(forms.Form):
    fieldname = forms.CharField(max_length=25)

if 'A' in request.POST:
    A_form = AForm(request.POST, prefix='A')
    B_form = BForm(prefix='B')
    # A form processing
elif 'B' in request.POST:
    B_form = BForm(request.POST, prefix='B')
    A_form = AForm(prefix='A')
    # B_form processing

Whois client in Python

The other day I had to implement a small whois client in python. After looking for a moment on the web I thought that this would be an easy task. Indeed the RFC is so small it's disturbing. So it didn't took long to get some sort of code running that could talk to a whois server:

import socket

def whois(domain):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((whois.dns.be, 43))
    sock.send(domain + "\r\n")
    response = ''
    while True:
        d = sock.recv(4096)
        response += d
        if d == '':
            break
    sock.close()
    return response

However I soon realized that the whois RFC is small for a reason: there is nothing in the specification and so problems start to appear...

The above snippet can talk to a server but will get a proper answer only for .be domains. To get usefull information one needs to know which server to talk to. Back on the web looking for some kind of whois server list. As it turns out whois-servers.net does provide such a list through DNS CNAME records. In order to know the whois server for .com one could do a DNS query for "com.whois-servers.net"

To make the above code more generic you could connect to the whois server with the following strategy:

tld = domain.split('.')[-1]
sock.connect(((tld + '.whois-servers.net'), 43))

While this is already alot better. Problem remains as not all domains use the same whois server even if they have the same TLD. To make things even easier not all whois servers have the same syntax or return the same results. If we take google.com we have to query .com.whois-servers.net for the string "=google.com"  just to be informed that their actual whois server is different. This information could be obtained by parsing the result in search of the following line "Whois Server: whois.markmonitor.com"

MJJX8TPXDHKM

Syntax Highlighting Test

Straightforward quicksort algorithm taken from wikipedia to test syntax highlighting within blogger:

def qsort1(lst):
    if len(lst) <= 1:
        return lst
    pivot = lst.pop(0)
    greater_eq = qsort1([i for i in lst if i >= pivot])
    lesser = qsort1([i for i in lst if i < pivot])
    return lesser + [pivot] + greater_eq

The idea has been borrowed from this blog post and SyntaxHighlighter can be found here.