Python Returns Inside With

The Python with statement, introduced in 2.5, is awesome.  I just discovered, perhaps a little naively, how awesome it really is.

For those that don't know, the with statement allows you to use an object inside a new context.  You can't just use any only Python object, like a string or a dictionary.  Any object used in a with statement needs a context manager.  For more on using context managers (and the with statement in general), see Fredrik Lundh's guide.

The additional piece of with magic I've been playing with is returning data from inside a with statement.  For example, consider the following getcount() function that returns that line count of a file...

def getcount(fobj):
    with fobj as f:
        return len(f.readlines())

fobj = open('tmp.txt')
print getcount(fobj)
print fobj.closed

This example is quite simple — open a file object and print the number of lines.  The getcount() function uses the with statement to open and close the file.  But wait a minute, how was the file object closed if we returned from the function inside the with statement?  This is the beauty of the context manager — it is guaranteed to carry out all clean-up tasks no matter what.  This is why we're printing out fobj.closed — to verify that this did indeed work.

The same methodology can be used with lock context managers.  For example...

import time
from threading import Thread, Lock

class MyThread(Thread):

    lock = Lock()

    def __init__(self):
        super(MyThread, self).__init__()

    def set_name(self):
        with self.lock:
            self.name = 'MyName'
            time.sleep(2)

    def get_name(self):
        print 'Waiting for name...'
        with self.lock:
            return self.name

    def run(self):
        self.set_name()

tobj = MyThread()
tobj.start()
print tobj.get_name()

Here we're explicitly making set_name() take longer than it should — causing the lock object to block.  The return statement in get_name() is paused until the name is set, since it's inside the with.  Cool stuff.

jQuery UI Widget Selectors

Widgets in jQuery UI have their own selectors.  These selectors allow the developer to select widgets by type.  For example, if you want to select all progressbar widgets, you can do this.  The selector for a specific widget type is similar to other element selectors you'd use with jQuery.  For example, you can say :button to retrieve button elements.  This same approach is used with jQuery widgets.

Each widget selector is constructed and added to the core jQuery set of expressions.  It does so by taking the name space, in this case, ui, and combining it with the widget name.  So we can make queries such as :ui-tabs or :ui-progressbar.  This is a great approach - we know that we're only retrieving elements that are real jQuery UI widgets.  There is a problem though.  It's slow and can't be used universally throughout all jQuery UI has to offer.

For example, I used the following to benchmark the widget type selector:

var iterations = 100;
var start = new Date().getTime();
for (i = 0; i < iterations; i++) {
    $(':ui-tabs');
}
var end = new Date().getTime();
console.log(end - start);

Notice we're using the : character in the selector - this invokes the selector built by the base widget.  However, to get a feel for how relatively slow this operation is, try replacing the : with . so you end up with $('.ui-tabs').  Notice the performance difference?  So why would anyone in their right mind use this widget selector?

The rationale behind such a selector is that you should be able to select all widget types base on, well, their type.  Think of a class method in Python or Java that will return all instances of that class.  It's an intuitive leap to transform "get me all accordion widgets" into :ui-accordion.  This way, it becomes a standard query format for widgets - namespace (ui) and widget name.

So what is wrong with using the class selector variant to retrieve all widgets belonging to a specific class?  Nothing aside from the fact that we can't be absolutely certain of widget class names.  For instance, if another UI developer on your team were to build a widget, and you now write some code that wants to grab them all.  You'd have to know ahead of time, or do some investigation work, to figure out which class is applied to every widget of that type.  Well, not necessarily.

You'll notice that you can use the class-based variant of the above query to retrieve all widgets of a specific type for every jQuery UI widget.  For example, .ui-tabs, .ui-progressbar, .ui-button, all these queries work fine and they're much faster than the widget selector that automatically extends the UI core for every widget - .ui-tabs is much faster than :ui-tabs.

You'll also notice that the naming in CSS classes applied to the widgets are all consistent.  This CSS class naming schema doesn't only have a role with themes - we can use this consistent convention to actually speed-up our code.

Social Network Technology

We all know what the real value of social networks is - the data.  The connections between individuals and other entities.  The micro updates, the kind that share little tidbits of information.  These things, aren't all that valuable on their own.  In fact, they produce a lot of noise.  The real value in social networks, is the aggregate data, the information we cannot see at a glance.  I think the ramifications of this information, the quest for it's possession, can be attributed to the current social bubble.  The prospect of mining this data is tantalizing - of course anyone with the necessary means will pay top dollar for it.  But is it just the data they're after?  There must be some valuable technology here, right?  Something is responsible for organizing all this information, for providing the lay user with tools to say what they want, when they want, and more importantly, as much as they want.  Data, data, and more data.  It's where it's at.

Why?  Why is the oodles of social information so important?  Statistics.  Competitive edge.  The collective opinions of consumers.  This is why we're looking for a better way to extract meaningful information from all this social noise.  Social noise.  I like that term.  It makes me think of riding home on the train and a group of over-galvanized teenagers hop on board.  Its like I signed into Facebook for three stops.  I filter it out, the best I can anyway, by reading, or listening to music.  This filtering is relatively easy to achieve on a subway car with few people - not a lot of noise.  But with social networks, and their big data, it'd be like standing in the middle of general admission at an Iron Maiden show with my headphones on.  Kinda difficult to filter that out.

It's technology like this - separating noise from meaning - that social networks have inspired.  It's like NASA, and the various spin-off thechnologies they've developed.  Going into outer-space is an enormous undertaking, requiring thousands if not millions of smaller technological solutions.  These smaller innovations probably never would have seen the light of day if not for the desire to visit distant worlds.  This is how I see the role of technological innovation in the social realm playing out.  We've got a pile of data, and we're only discovering now the interesting things we can do with it, other than the original use case of social interaction on the web.

Take Google for instance.  They've already been doing R & D for a long time now, despite the fact that labs is winding down.  They started doing R & D as a consequence of having more data than anyone previous.  The result?  Stuff like Docs and App Engine.  Could Google have hammered out these products had they not been the worlds largest, most relied-upon search engine?  Probably.  But would the idea to build such products have been prevalent if they didn't build such technology to support their web index?  See, data at this scale inspires new technology, innovative new tools that have wider applications than for what they were originally designed.

Obviously Google doesn't have the last word on innovative technology with regard to big data.  Big data is becoming the norm.  It seems that today you can't be taken seriously unless you own and operate multiple data centers.  Sure, the information were storing is interesting, I have no doubt about that.  Some insights about social information come naturally, without the aid of technology.  Even in the past three or four years, I'm sure there have been a few light-bulbs popping up while digging around in the ever-connected social landscape.  What I'm really looking forward to is not so much how much data the social domain can accumulate - they've already flexed their data center muscles and there is no end in sight.  I'm more interested in the spin-off technology these social networks are going to give to the world - hopefully in the form of open source software.

Django Class-Based Views: A Design Perspective

The release of Django 1.3 introduced the concept of class-based generic views.  Prior to 1.3, generic views were written as functions.  What's the big fuss over class-based views?  What can they possibly offer that functional views can't?  Convenience, less code, and elegant design - a property that extends down through Django's core.   From a design perspective, class-based views in Django are a better way to implement common view patters with marginal effort than generic function views.  For me, having only used class-based views for a few months, I'm very impressed.  When I compare generic functions with generic classes, I'm amazed at the difference in quality.  This is one of those small enhancements that I think will change Python web development for the better.

Maybe a little history first, shall we?  First of all, what makes Django views so useful and easy to work with in the first place?  We can tell Django how to map HTTP requests to our view functions using URL configurations.  This is useful because we have URL definitions separate from the functions themselves - multiple URL patterns can be handled by a single view function.  The URL path segments and query parameters are passed to the function as positional arguments and keyword arguments respectfully.

Having a Python function handle each HTTP request sent to the application is intuitive, no two ways about it.  The job of the view function is to take the client's request, perform some business logic, and return a rendering context for the template.  For common scenarios, generic view functions can be passed to the URL configuration.  For example, listing specific objects or for a detailed view of a single object.  For this, generic view functions are perfect.  We don't need to write repetitive code that differs only slightly from view to view.

However, we can't use generic views for everything and this is where the Django developer needs to step in and write some custom stuff.  Stuff that'll refine a query or pass additional context to the template.

Prior to Django 1.3, where generic class views were introduced, generic functions were customized by passing an info dictionary to the URL configuration.  This isn't an ideal coupling since the view now depends on additional data being passed to the URL definition.  For instance, if I have a URL where I want to return a list of objects, I can use a generic view that does just that.  However, to customize the query, I need to pass additional information to the URL definition that tells the view how to alter the query.  So if I wanted to reuse this same view, with the same query modifications, I'd have to duplicate the additional generic view info.  Not that this is painfully tedious, just error-prone.

Class-based generic views aims to solve the trouble of binding view-specific data to the URL, and to that end, I think they've succeeded hugely.  The new class-based approach introduces the concept of inheritance.  This method of extending views is less susceptible to problems with configuring URLs which depend on additional information.  Plus, extending generic views is simply elegant from a design perspective.

For example, we can extend the ListView class provided by Django to render a list of objects from our model.  At the most basic level, all we really need to tell ListView about is the model to select from.  Say we have a Book class.  We can define ourselves a new BookList class view for returning lists for books.  We're overriding the default model value of ListView.

I find this superior to the decorated customization approach.  With decorators, we're wrapping the view function with additional logic.  For instance, depending on the request context, HTTP headers, GET values, the URL, and so forth, we'll execute the logic of the generic view differently.  Nothing is wrong with this except for the fact that it doesn't sit well with doing things generically for a database view.  If I'm writing a list or detail view for things stored in the database, I want as little customization as possible if I'm using generic views.  This is a key point - the views are generic because you're supposed to give them only as little information as possible.  Just the bare minimum required to produce the appropriate template context.

With the new class-based approach, it's easier to leave the defaults alone, only overriding what's necessary.  The model name, for instance.  Or if you need more control over the resulting query set, overriding the get_queryset() method.  It's best to leave the common stuff to the generic view framework.  This is it's selling point, after all.  Things like automatically finding the template based on the model name.  If you absolutely must change something about a view, there is an attribute or a method you can override.  If you find that you're extending the Django generic views and they're becoming large, chances are you shouldn't be using them.  That is, if a method you've overridden on a Django generic class view looks nothing like a basic customization, you should think hard about why you're using a generic view in the first place.  Maybe you're better off writing a regular, non-generic view.  There is nothing wrong with this approach - that's why standard, function-based views exist - for writing business logic.

So from a design perspective, what do class-based generic views bring to the table?  A cleaner, more elegant mechanism for displaying database data.  In addition to decoupling the customizations that go along with generic views from the URL configurations, class-based generic views embrace inheritance as a means to use default behaviour and override only when necessary.  Often the generic behaviour is enough, just tell the view about the basic necessities, and you're off to the races.  Django offers a plentiful selection of built-in class-based views to choose from, each of which can be extended cleanly.