jQuery UI Widget Selectors

Widgets in jQuery UI have their own selectors.  These selectors allow the developer to select widgets by type.  For example, if you want to select all progressbar widgets, you can do this.  The selector for a specific widget type is similar to other element selectors you'd use with jQuery.  For example, you can say :button to retrieve button elements.  This same approach is used with jQuery widgets.

Each widget selector is constructed and added to the core jQuery set of expressions.  It does so by taking the name space, in this case, ui, and combining it with the widget name.  So we can make queries such as :ui-tabs or :ui-progressbar.  This is a great approach - we know that we're only retrieving elements that are real jQuery UI widgets.  There is a problem though.  It's slow and can't be used universally throughout all jQuery UI has to offer.

For example, I used the following to benchmark the widget type selector:

var iterations = 100;
var start = new Date().getTime();
for (i = 0; i < iterations; i++) {
    $(':ui-tabs');
}
var end = new Date().getTime();
console.log(end - start);

Notice we're using the : character in the selector - this invokes the selector built by the base widget.  However, to get a feel for how relatively slow this operation is, try replacing the : with . so you end up with $('.ui-tabs').  Notice the performance difference?  So why would anyone in their right mind use this widget selector?

The rationale behind such a selector is that you should be able to select all widget types base on, well, their type.  Think of a class method in Python or Java that will return all instances of that class.  It's an intuitive leap to transform "get me all accordion widgets" into :ui-accordion.  This way, it becomes a standard query format for widgets - namespace (ui) and widget name.

So what is wrong with using the class selector variant to retrieve all widgets belonging to a specific class?  Nothing aside from the fact that we can't be absolutely certain of widget class names.  For instance, if another UI developer on your team were to build a widget, and you now write some code that wants to grab them all.  You'd have to know ahead of time, or do some investigation work, to figure out which class is applied to every widget of that type.  Well, not necessarily.

You'll notice that you can use the class-based variant of the above query to retrieve all widgets of a specific type for every jQuery UI widget.  For example, .ui-tabs, .ui-progressbar, .ui-button, all these queries work fine and they're much faster than the widget selector that automatically extends the UI core for every widget - .ui-tabs is much faster than :ui-tabs.

You'll also notice that the naming in CSS classes applied to the widgets are all consistent.  This CSS class naming schema doesn't only have a role with themes - we can use this consistent convention to actually speed-up our code.

Query Superclass

Throughout the object-oriented software development process, you're likely to find new and better ways to query object properties. These properties aren't necessarily the defined attributes for the object's class. The methods used to query these attributes are often obvious enough. What about derived properties? These are computed properties and they sometimes go unnoticed until some iteration further down the line.

Is the best approach to adding these newly discovered query methods to add them to the class directly? Probably not because you are adding new responsibilities directly to the class. There are exceptions to not not doing this. For instance, there might be some aspect of the objects that you may want to query that are painfully obvious and were missed for some reason. Also, if the class is somewhat specialized already, the newly discovered query methods may only apply to instances of this class.

When more general query methods are discovered, we can use the inheritance mechanism to extend the functionality of our class. This can go in either direction. You can define a specialized version of your class that is a child of that class. This class would then define the new query methods. The problem with this approach is that rather than being more generalized, you are increasing the specificity level. This is fine if the newly discovered query method are specific to your class. More often than not, however, derived properties that are computed can often be applied to a group of disparate classes. If this is the case, spending the time to develop a highly-reusable query super class may be worthwhile.

Magic methods of the Django QuerySet

The QuerySet class defined in the Django Python web application framework is used to manage query results returned from the database. From the developer perspective, QuerySet instances provide a high-level interface for dealing with query results. Part of the reason for this is that instances of this class behave very similarly to other Python primitive types such as lists. This is accomplished by defining "magic" methods, or, operator overloading. If an operator for a specific instance is overloaded, it simply means that the default behavior for that operator has been replaced when the operand involves this instance. This means that we can do things such as use two QuerySet instances in an and expression that will invoke custom behavior rather than the default behavior. This useful because it is generally easier and more intuitive to use operators in certain contexts than invoking methods. In the case of using two instances in an and expression, it makes more sense to use the built-in and Python keyword than it does to invoke some and() method on the instance. The Django provides several "magic" methods that do just this and we will discuss some of them below.

Python instances that need to provide custom pickling behavior need to implement the __getstate__() method. The QuerySet class provides an implementation of this method. The Django QuerySet implementation removes any references to self by copying the __dict__ attribute. Here is an example of how this method might get invoked.

import pickle
pickle.dumps(query_set_obj)

The representation of Python instances is provided by the __repr__() method if it is defined. The Django QuerySet implementation will call the builtin list() function on itself. The size of the resulting list is based on the REPR_OUTPUT_SIZE variable which defaults to 20. The result of calling the builtin repr() function on this new list is then returned. Here is an example of how this method might get invoked.

print query_set_obj

The length of the QuerySet instance can be obtained by calling the builtin len() function while using the instance as the parameter. In order for this to work, a __len__() method must be defined by the instance. In the case of Django QuerySet instances, the first thing checked is the length of the result cache. The result cache is simply a Python list that exists in Python memory to try and save on database query costs. However, if the result cache is empty, it is filled by the QuerySet iterator. If the result cache is not empty, it is extended by the QuerySet iterator. This means that the result cache is updated with any results that should be in the result cache but are not at the time of the __len__() invocation. The builtin len() function is then called on the result cache and returned. Here is an example of how this method might get invoked.

print len(query_set_obj)

Python allows user-defined instance to participate in iterations. In order to do so, the class must define an __iter__() method. This method defines the behavior for how individual elements in a set are returned in an iteration. The QuerySet implementation of this method will first check if the result cache exists. If the QuerySet result cache is empty, the iterator defined for the QuerySet instance is returned. The iterator does the actual SQL querying and so in this scenario, the iteration looses out on the performance gained by having cached results. However, if a result cache does exist in the QuerySet instance, the builtin iter() function is called on the cache and this new iterator is returned. Here is an example of how the __iter__() method might be invoked.

for row in query_set_obj:
   print row

Python instances can also take part in if statements and invoke custom behavior defined by the __nonzero__() method. If an instance defines this method, it will be invoked if the instance is an operand in a truth test. The Django QuerySet implementation of this method first checks for a result cache, as does most of the other "magic" methods. If the result cache does exist, it will return the result of calling the builtin bool() function on the result cache. If there is no result cache yet, the method will attempt to retrieve the first item by performing an iteration on the QuerySet instance. If the first item cannot be found, false is returned. Here is an example of how the __nonzero__() might be invoked.

if query_set_obj:
   print "NOT EMPTY"

Finally, Python instance may be part or and an or Python expressions. The Django QuerySet instance defines both __and__() and __or__() methods. When these methods are invoked, they will change the underlying SQL query used by returning a new QuerySet instance. Here is an example of how both these methods may be used.

print query_set_obj1 and query_set_obj2
print query_set_obj1 or query_set_obj2

Inside Query.get_machines()

In ECP 2.2, there exists a new Query class that defines a few class methods for querying the ECP database. The rational behind creating this class was that there are several queries that are very similar but require slightly varying degrees of flexibility. Also, it makes logical sense to group query methods into a class called Query. We know that using this class will in no way affect the state of the database.

The most pressing need for this class are machine queries. For example, there is often a need to select machines based on a specific cluster. Also, we may need to order these results and combine the various filters on a per-query basis.

Here is what the method definition looks like:

#ECP - The Query.get_machines() method.

@classmethod
def get_machines(cls, *args, **kw):
   """Retrieve a list of machines.
    @status: Experimental"""
   filter=False
   join=[]
   if kw.has_key('order_by'):
       order_by=kw['order_by']
   else:
       order_by='machine_name'
   if kw.has_key('machine_uuid'):
       machine_uuid=kw['machine_uuid']
   else:
       machine_uuid=False
   if kw.has_key('cluster_uuid'):
       cluster_uuid=kw['cluster_uuid']
   else:
       cluster_uuid=False
   if kw.has_key('namefilter'):
       namefilter=kw['namefilter']
   else:
       namefilter=False
   if kw.has_key('children'):
       children=kw['children']
   else:
       children=False
   if cluster_uuid:
       try:
           cluster_obj=UUIDSearch.cluster(cluster_uuid)
       except E2ClusterNotFound, e:
           e.store_traceback()
           return []
       filter=[]
       machine_part=Machine.q.id==ClusterMachine.q.machine
       join_part=ClusterMachine.q.clusters==cluster_obj.id
       filter.append(machine_part)
       filter.append(join_part)
       if namefilter:
           name_part=LIKE(Machine.q.machine_name, "%s%%"%(namefilter))
           filter.append(name_part)
   elif machine_uuid:
       try:
           machine_obj=UUIDSearch.machine(machine_uuid)
       except E2MachineNotFound, e:
           e.store_traceback()
           return []
       if children:
           filter=[]
           machine_part=Machine.q.parent==Hypervisor.q.id
           join_part=Hypervisor.q.machine==machine_obj
           filter.append(machine_part)
           filter.append(join_part)
       else:
           return [machine_obj]
   elif namefilter:
       filter=LIKE(Machine.q.machine_name, "%s%%"%(namefilter))
   result=PermSelect.machine(perm='r',\
                             orderby=order_by,\
                             filter=filter,\
                             join=join)
   return result

The first to variables in the method, join and filter, are initialized first. They are always going to be sent to the final PermSelect.machine() invocation. Next, the keyword parameters are initialized. Here, the default values are assigned to the parameters if they are not provided.

If a cluster_uuid was specified, we retrieve the Cluster instance. We then construct the table-joining query component that joins the machine and cluster tables. If there is a name_filter specified, we construct a query component based on it.

If a machine_uuid was specified, we retrieve the machine instance. Then, we check if the children parameter was set to true. If so, we build the joining query components to select the children of the specified machine. Otherwise, we return the single machine instance.

Finally, we use the PermSelect class to execute the final query, which integrates permission checking in the selection. Another benefit of this method is that it will significantly cut down on the number of queries executed eventually.

Flexible SQLObject queries

Although SQLObject is considered an ORM (Object Relational Mapper), it can still be used for traditional SQL queries. The question is, why bother adding this ORM overhead when the queries can be executed directly by a SQL library?

For anything but the most trivial queries, SQL syntax is still the superior method to manipulate data sets. When using a SQL library in Python, without an ORM, the result sets need to be constructed. This boiler plate code can be handled by SQLObject while still having the ability to use more elegant, or extremely complex in some cases, SQL. For example, if you have defined a couple SQLObject classes, blogAccount, and blogEntry, we can do somthing similar to:

result=blogEntry.select("""blog_account.id=blog_entry.blog_account_id
                          AND blog_account.user_id=%s"""%getCurrentUserID(),\
                          clauseTables=['blog_account'])

This result set will contain a list of blogEntry instances. This is useful because no code was needed to construct the result set. This is obviously not the worlds most complex query, but it does illustrate the idea that fancy SQL queries can still be combined with the concept of an ORM.