Archive for the ‘Python’ category

Easy examples of redundancy

2008-07-29-Tue

The following are some examples of simple redundancies in procedural programming. They are given in Python. I typically run into these when I simplify my code after writing it for the first time or after making edits.

Using if-else to assign a Boolean:

if condition:
    x = True
else:
    x = False

Simplified:

x = condition

(Requires that condition is a Boolean.)

Returning immediately after an assignment:

y = value
return y

Simplified:

return value

Common code among branches:

if condition:
    group of statements A
    group of statements B
    group of statements D
else:
    group of statements A
    group of statements C
    group of statements D

Simplified:

group of statements A
if condition:
    group of statements B
else:
    group of statements C
group of statements D

Nested if-statements:

if condition0:
    if condition1:
        statements

Simplified:

if condition0 and condition1:
    statements

Requires that the outer if-statement does not have an else-clause.

If-statements with the same actions:

if condition0:
    statements (same)
if condition1:
    statements (same)

Simplified:

if condition0 or condition1:
    statements (same)

Requires condition0 and condition1 to be disjoint, or statements to be idempotent.

Advertisements

Department of redundancy department: Python lambdas

2008-07-29-Tue

A coworker recently pointed out the lambda (anonymous function) in this piece of code that I wrote was redundant:

project_key_regex = re.compile(r"^project(\d+)$")
for key in filter(lambda k: project_key_regex.match(k), req.args.keys()):
    ...

It would be improved like this:

project_key_regex = re.compile(r"^project(\d+)$")
for key in filter(project_key_regex.match, req.args.keys()):
    ...

Apparently, methods of an object can be thought of as functions that have one variable bound to the object.

A bit of info about the phrase “Department of Redundancy Department” on Everything2: http://everything2.com/title/Department%2520of%2520Redundancy%2520Department

My misinterpretation of Python decorators

2008-06-22-Sun

DrProject is written in Python, but I had only a little experience with Python when I first started. So I had to learn many things along the way, sometimes in the hard way.

Today’s topic is the semantics of decorators in Python (PEP 318). I used to think I knew how they worked, but I was wrong. Let’s start with a simple example:

def decorate(func):
    def decorated(self):
        print "Before"
        func(self)
        print "After"
    return decorated

class Foo(object):
    @decorate
    def bar(self):
        print "Hello, world!"

This is equivalent to the following code in the old style (I won’t repeat decorate because it remains unchanged):

class Foo(object):
    def bar(self):
        print "Hello, world!"
    bar = decorate(bar)

In either case (they’re equivalent), if we create a Foo and invoke bar, we get:

Before
Hello, world!
After

So far so good? This is the kind of code I found in DrProject:

def action(name, template=None, stylesheets=None):
    def _decorate(func):
        def _execute(controller, req):
            (...)
        (...)
        return _execute
    (...)
    return _decorate

class AdminController(Controller):
    @action('list_projects',
        template='admin_list_projects.html',
        stylesheets=['admin.css'])
    def _process_list_projects(self, req, order='name', desc=''):
        (... omitted ...)

Somehow, I naïvely expected the decorator invocation to be equivalent to something like _process_list_projects = action(_process_list_projects, 'list_projects', template='admin_list_projects.html', stylesheets=['admin.css']), with the function as the zeroth argument. But that would be impossible, because action only takes 3 arguments, and has no provisions for extra arguments – so Python would err loudly if action is called with 4 arguments.

The key to understanding this situation lies in PEP 318, but the code itself is very suggestive too. Essentially, _process_list_projects is decorated by the function returned by action('list_projects', template=..., stylesheets=...). (See how action returns a nested _decorate function?) So overall, the explicit form of this decoration would be as follows:

class AdminController(Controller):
    def _process_list_projects(self, req, order='name', desc=''):
        (... omitted ...)
    _process_list_projects = action('list_projects',
        template='admin_list_projects.html',
        stylesheets=['admin.css'])(_process_list_projects)

Notice the double call, i.e. action(...)(...).

To recap, a decorator usage like

@decorate
def foo():
    (...)

expands to the form

def foo():
    (...)
foo = decorate(foo)

where decorate is an arbitrary expression that evaluates to a decorator, which can be a simple function variable or the result of a function maker (e.g. make_decorator(1,2,3)).

I have nothing against the decorator syntax; in fact, I think it makes a lot of sense now that I understand it better. I just don’t want anyone else to repeat the same naïve misinterpretation.

Default behaviour of comparisons in Python

2008-06-09-Mon

Browsing DrProject’s code base a moment ago, I came across this piece of code in drproject.project.Project: (I abbreviated the long expression a bit)

    def get_members(self):
        members = [m.user for m in Membership.query.filter_by(...)]
        members.sort()
        return members

I was curious about what the sort actually did. So, members is a list of drproject.project.User objects. The class User extends Elixir’s Entity class. Neither of these classes appeared to have overridden the comparison operators. EntityMeta didn’t seem to do anything either. And the sort method was being called with no comparator function passed in. What on earth is going on?

The Python language reference page on comparisons was not very helpful. So I had to make some guesses. Since User objects are database rows, maybe they are sorted by their primary keys? But the aforementioned lack of code suggested against this possibility. Based on my experience with object-oriented languages, the only other thing I could think of was the IDs of objects — their memory addresses. It was a good candidate, since IDs are comparable for all objects.

A quick experiment in a Python session gives evidence for this behaviour:

In [1]: import random
In [2]: randcmp = lambda x, y: random.choice([-1, 0, +1])

In [3]: things = [object() for i in range(10)]
In [4]: things  # The IDs are in ascending order merely by coincidence.
Out[4]:
[<object object at 0xb7d6d598>,
 <object object at 0xb7d6d5a0>,
 <object object at 0xb7d6d5a8>,
 <object object at 0xb7d6d5b0>,
 <object object at 0xb7d6d5b8>,
 <object object at 0xb7d6d5c0>,
 <object object at 0xb7d6d5c8>,
 <object object at 0xb7d6d5d0>,
 <object object at 0xb7d6d5d8>,
 <object object at 0xb7d6d5e0>]

In [5]: things.sort(randcmp)
In [6]: things  # Now the IDs are in random order.
Out[6]:
[<object object at 0xb7d6d5c8>,
 <object object at 0xb7d6d5a8>,
 <object object at 0xb7d6d5d0>,
 <object object at 0xb7d6d598>,
 <object object at 0xb7d6d5a0>,
 <object object at 0xb7d6d5b0>,
 <object object at 0xb7d6d5d8>,
 <object object at 0xb7d6d5b8>,
 <object object at 0xb7d6d5c0>,
 <object object at 0xb7d6d5e0>]

In [7]: things.sort()
In [8]: things  # My gosh, we have ascending IDs again!
Out[8]:
[<object object at 0xb7d6d598>,
 <object object at 0xb7d6d5a0>,
 <object object at 0xb7d6d5a8>,
 <object object at 0xb7d6d5b0>,
 <object object at 0xb7d6d5b8>,
 <object object at 0xb7d6d5c0>,
 <object object at 0xb7d6d5c8>,
 <object object at 0xb7d6d5d0>,
 <object object at 0xb7d6d5d8>,
 <object object at 0xb7d6d5e0>]

Is this behaviour unintuitive, and does it let you shoot yourself in the foot easily? You be the judge.

And going back to the original DrProject code, the sort indeed sorts by IDs, which is probably not very useful.

Singleton metaclass example

2008-05-21-Wed

Here’s a metaprogramming example that I conceived, involving the use of a metaclass to implement the singleton property.

class Counter(object):

    global_count = 0
    count_by_class = {}

    @staticmethod
    def count(obj):
        cls = obj.__class__
        class_count = Counter.count_by_class.get(cls, 0)
        result = (Counter.global_count, class_count)
        Counter.global_count += 1
        Counter.count_by_class[cls] = class_count + 1
        return result

class CountedObject(object):

    def __init__(self):
        print "Creating instance of class %s" % self.__class__.__name__
        (self.big_count, self.small_count) = Counter.count(self)

    def __str__(self):
        return self.__repr__()

    def __repr__(self):
        return "<%s instance. Global ID: %d. %s ID: %d.>" % (self.__class__.__name__, self.big_count, self.__class__.__name__, self.small_count)

class SingletonMeta(type):

    def __init__(cls, name, bases, d):
        print "Creating class %s" % name
        type.__init__(cls, name, bases, d)

    def __call__(cls):
        print "Attempting to create instance of class %s" % cls.__name__
        if not hasattr(cls, "SINGLETON"):
            instance = type.__call__(cls)
            cls.SINGLETON = instance
        else:
            print "Returning singleton of class %s" % cls.__name__
            instance = cls.SINGLETON
        return instance

class SingletonBase(CountedObject):
    __metaclass__ = SingletonMeta

class FooSingleton(SingletonBase):
    pass

class BarSingleton(SingletonBase):
    pass

class NotSingleton(CountedObject):
    pass

Now, time for some queries:

Creating class SingletonBase
Creating class FooSingleton
Creating class BarSingleton

>>> FooSingleton()
Attempting to create instance of class FooSingleton
Creating instance of class FooSingleton
<FooSingleton instance. Global ID: 0. FooSingleton ID: 0.>

>>> FooSingleton()
Attempting to create instance of class FooSingleton
Returning singleton of class FooSingleton
<FooSingleton instance. Global ID: 0. FooSingleton ID: 0.>

>>> NotSingleton()
Creating instance of class NotSingleton
<NotSingleton instance. Global ID: 1. NotSingleton ID: 0.>

>>> NotSingleton()
Creating instance of class NotSingleton
<NotSingleton instance. Global ID: 2. NotSingleton ID: 1.>

>>> NotSingleton()
Creating instance of class NotSingleton
<NotSingleton instance. Global ID: 3. NotSingleton ID: 2.>

>>> SingletonBase()
Attempting to create instance of class SingletonBase
Creating instance of class SingletonBase
<SingletonBase instance. Global ID: 4. SingletonBase ID: 0.>

>>> SingletonBase()
Attempting to create instance of class SingletonBase
Returning singleton of class SingletonBase
<SingletonBase instance. Global ID: 4. SingletonBase ID: 0.>

>>> BarSingleton()
Attempting to create instance of class BarSingleton
Returning singleton of class BarSingleton
<SingletonBase instance. Global ID: 4. SingletonBase ID: 0.>

>>> BarSingleton()
Attempting to create instance of class BarSingleton
Returning singleton of class BarSingleton
<SingletonBase instance. Global ID: 4. SingletonBase ID: 0.>

As you can see, classes that extend Singleton (or more precisely, classes that are metaclassed by SingletonMeta) can only be instantiated once. This can be seen by the control flow implied by the printed messages, and can also be seen by the IDs of the objects returned.
One complication, though – instantiating BarSingleton seems to return the wrong result because its superclass already has its SINGLETON field set.