Making Deadlines

Lets face it, in the software development world, there are unrealistic deadlines that need to be met. This entry talks about some dirty coding tricks that were necessary to meet a deadline. Although these types of tricks would make any rational developer recoil in disgust, they are sometimes necessary. In fact, most production software isn't ideal code wise. Ask any developer who has code running in the wild if they would like to re-factor at least a portion of that code. Odds are that you'll find many.

Is there an acceptable limit to exactly how dirty these tricks can be in order to meet a deadline? Some dirty coding tricks are so horrific that fixing the repercussions of them in the future will be a project on its own. However, all that matters is shipping the software on time. And that it works. So if crossing some kind of ugly code threshold results in shipping on time, do it. Otherwise, you may not get a chance to re-factor that code later on.

And, lessons can be learned about the project as a whole when resorting to dirty code tricks. These lessons can impose new constraints and new policies on future releases.

Building Python Lists

Building primitive Python lists is a common task in virtually any Python application. The Python list is not unlike a traditional array in many respects, only more powerful. A common scenario is building a Python list from an iterator that yields values. In this case, each iteration adds a new element to the list being built. One method used to build Python lists using an iteration construct is the append method. This approach invokes the list append() method to append a new list element. Another approach to building Python lists is to place the iteration itself within the new list. This approach can be considered inline list building. The benefit to using the inline approach is the performance gain. The benefit to using the append method is readability when logic is required within each iteration that appends elements to the list. Below is an example illustrating both approaches.

#Example; Building Python lists.

import timeit

#Build a list consisting of ten elements using the append method.
def build_append():
  list_obj=[]
  for i in range(0,10):
      list_obj.append(i)
    
#Build a list consisting of ten elements using the inline method.
def build_inline():
  list_obj=[i for i in range(0,10)]

if __name__=="__main__":
  #Setup timers.
  t_append=timeit.Timer("build_append()",\
                        "from __main__ import build_append")
  t_inline=timeit.Timer("build_inline()",\
                        "from __main__ import build_inline")

  #Execute timers.
  r_append=t_append.timeit()
  r_inline=t_inline.timeit()

  #Show results.
  print "APPEND:",r_append
  print "INLINE:",r_inline
  print "DIFF %:",(r_inline/r_append)*100

jQuery UI Dialog Buttons

The jQuery UI javascript toolkit provides UI developers with a nice dialog widget. The dialog widget inside web application user interfaces is a great way to achieve optimal use of the space provided. These dialog widgets can be configured with dialog action buttons. These are automatically-generated buttons that are placed at the footer of the dialog when it is displayed. This is the preferred method of displaying buttons in the jQuery UI dialogs as they look more like part of the dialog than something that was placed inside the dialog.

The button configuration for the jQuery UI dialog widget is specified as a javascript object containing all the buttons to be displayed in the dialog. Each attribute of the object maps to the text displayed inside each dialog button. The value of these attributes are the callbacks that are executed when the button is clicked.

However, there are a few problems to this approach. Firstly, the text of these buttons is difficult to update without replacing the entire button set. This is not only computationally expensive, but is also not an ideal way to couple code since the same operation has two responsibilities. Updating the dialog button text of a particular button and replacing all buttons entirely should be two distinct operations. Secondly, this also relates to the first problem, the dialog buttons cannot be uniquely identified easily.

A better configuration format might be to specify a DOM ID for each button in the main button configuration object. So the button would then have a unique handle. Underneath each DOM button ID would be another object specifying the button label and the button callback event.

Stand Alone Python Servers

The Python programming language provides a full-featured suite of both high and low-level networking capabilities. This allows for stand-alone web servers to be written in Python. For instance, CherryPy is an object publishing framework for Python with a very powerful built-in HTTP server.

In production environments, Python is often run behind a more production-ready web server such as Apache. The main reason for doing so is performance. Web servers such as Apache are written low-level languages and thus have a raw computing advantage. They are generally mature as software packages. The have been tried and tested for much longer than most Python web servers.

Another reason may be architecture. If there are several other production services that already use Apache, then it would make sense for consistency. It is much easier to maintain a single web server than many, often very different web servers.

If one were to deploy a single Python web application with a single web server, is deploying to a production-ready HTTP server really necessary? Are the performance gains really all that noticeable in the grand end-user scheme of things? As for stability, this is the kind of thing that should be rigorously tested before even considering placing the stable label on the server to begin with.

Javascript Sockets

Is this idea even possible? Can javascript sockets actually exist inside javascript applications? According to this entry, this reality isn't too far-fetched after all. One of key problems this will solve is the current polling requirement of most javascript applications. These applications need to poll the server for state changes so that these changes, if they have taken place, can be reflected in the UI. This is an expensive operation that often yields nothing in return. So, we have this expensive process that is need just in case of a server state change. Below is an illustration of a polling javascript application.



Both Flash and Java Applets are components external to the browser that allow connection sockets to communicate with the server. It seems counter-intuitive to be using a web browser that doesn't work with most of the web without a separate software installation process. The ideal situation would be to provide the javascript application with a socket. This would eliminate the need for both a library external to the browser as well as the need for polling. Below is an example of what a future javascript application without the polling requirement might look like.

Configuring Trac Ticket Workflow

Trac is a project management system written in Python. Trac has a powerful issue tracking system in which new issues are created as tickets. The system is flexible enough to allow for custom ticket types. This means that Trac tickets not only allow for issue tracking, but any conceivable software development task as well. There are many other customizations that can be made to Trac tickets such as custom fields and custom work-flow. Tickets in Trac go through a series of states during their lifetime. The current state of a given ticket determines the actions available on that ticket. The state of a ticket in Trac terminology is the ticket status. The ticket work-flow that determines the actions available, the permissions required, and the state the ticket must be in in order to transition are all configurable. In other words, if a given ticket is closed, the only available action for the ticket is to reopen it. This is because of the default Trac ticket work-flow configuration.

In the Trac configuration, ticket actions can be configured to allow for certain ticket states to be active in order to execute the action. So, if you wanted to allow more actions other than reopen on closed tickets, this can be specified in the Trac configuration. Multiple current states may be specified as criteria. Only a single target state may be specified. Below is a simplistic example of a ticket work-flow configuration element.

[ticket-workflow]

action = current_state1, current_state2, current_state3 -> target_state

action.permission = TICKET_MODIFY

action.operation = ticket_operation_name

Platform Independence In Dynamic Languages

In today's modern computing world, more and more applications are built in dynamic languages. These languages offer more platform independence than compiled languages do. This is because dynamic languages run inside a virtual machine that was compiled for the target platform. The end result is applications that offer more platform independence. Dynamic language virtual machines are becoming more prevalent on end-user systems these days. For instance you'll find a Java virtual machine just about everywhere. In cases where the virtual machine does not exist on the target platform, the virtual machine can be distributed along with the application. The main benefit of course is that once a dynamic language virtual machine is installed, the machine handles the majority of the platform dependent operations.

There are obvious limits to how independent a given application written in a dynamic language can be. This mostly depends on what the application does. Applications that use only the most fundamental features of the language have any chance of being truly platform independent. Realistically, a given application is going to want to take advantage of libraries or modules offered by the language in order to provide better functionality. For instance, most dynamic languages will provide a operating system module for low-level system operations. Not all functionality of this module will be supported on all platforms and there may be subtle behavioral differences in the operations supported across all platforms. It is the responsibility of the application to handle these scenarios.

The best way to handle these platform dependent anomalies is to define platform specific abstractions. This abstraction is illustrated below.



The base Controller class is abstract. It should never be instantiated. Only one of its children, WinController or LinuxController, should ever be instantiated. Therefore, the base and its children should only define functionality that differs among supported platforms.

Django Boundary Iterators

The Django Python web application framework provides tools to parse multi-part form data as any web framework should. In fact, the developer responsible for creating Django web applications need not concern themselves with the underlying parsing machinery. However, it is still there and very accessible.

The Django boundary iterators are used to help parse multi-part form data. These iterators are also general enough to be used in different contexts. The BoundaryIter class collaborates with other iterator classes such as ChunkIter and LazyStream. An example of these classes collaborating are illustrated below.

#Example; Using the Django boundary iterator.

#Imports.
from StringIO import StringIO
from django.http.multipartparser import ChunkIter, BoundaryIter, LazyStream
                                    
if __name__=="__main__":
  #Boundary data.
  b_boundary="-B-"
  c_boundary="-C-"

  #Example message with two boundaries.
  _message="bdata%scdata%s"%(b_boundary, c_boundary)

  #Instantiate a chunking iterator for file-like objects.
  _chunk_iter=ChunkIter(StringIO(_message))

  #Instantiate a lazy data stream using the chunking iterator.
  _lazy_stream=LazyStream(_chunk_iter)

  #Instantiate two boundary iterators.
  _bboundary_iter=BoundaryIter(_lazy_stream, b_boundary)
  _cboundary_iter=BoundaryIter(_lazy_stream, c_boundary)

  #Display the parsed boundary data.
  for data in _bboundary_iter:
      print "%s:      %s"%(b_boundary, data)

  for data in _cboundary_iter:
      print "%s:      %s"%(c_boundary, data)

In this example, we have a sample message containing two boundaries that is to be parsed. In order to achieve this, we create three iterators, and a stream. The _chunk_iter iterator is a ChunkIter instance that is a very general iterator used to read a single chunk of data at a time. This iterator expects a file-like object to iterate over. The two boundary iterators, _bboundary_iter and _cboundary_iter, are instances of the BoundaryIter class. This iterators expect both a data stream and a boundary. In this case, a LazyStream instance is passed to the boundary iterators.

We finally display the results of iterating over the boundary iterators. By the time the second iterator is reached, the data stream is now shorter in length.

Trac Database Connection Wrapping

Web applications written in Python tend to use some kind of object-relational mapping technology such as SQLObject or SQLAlchemy. Trac, however, has not yet introduced such dependencies nor do they define an object-relational mapper of their own. Trac uses the more traditional database connection and database cursor abstractions. This isn't to say that there aren't any higher-level abstractions defined. There are and they are really designed quite well and are therefore easy to comprehend.

Of course, object-relational mapping technology isn't the be-all and end-all of all web applications. In fact, if you can do without the overhead involved you may be better off as long as you can clearly define a separation of concerns.

For instance, Trac defines a database drivers for MySQL as well as others. This is typically the selling point for object-relational technology. Trac's approach is to define two base classes for the core database abstractions; IterableCursor and ConnectionWrapper. These classes provide the generic functionality shared by all database drivers. A brief idea of the design is illustrated below.

Enhanced Code Completion

Most modern development environments for virtually any programming language come complete with code completion. What exactly is code completion? Code completion makes developers life easier by giving them names to select from when typing the beginning of the name. So, for instance, if you were to start typing the name of a class "My", the editor would then give you a list of names that begin with "My", allowing you to select the name you want to use. It makes their life easier by recognizing that they are attempting to use a name that is defined in the current context such as a class or function name. Code completion also works when developers attempt to use members of a class such as attributes or methods. Once the developers engages the dot notation, the code completion engine is able to present the developer with the available members of that class or instance.

So is this as productive as it gets in regards to code completion? I think code completion in open source development environments could be enhanced. Geany serves as an example of how code completion could be enhanced. If a developer already has a function or method name is use, then the developer should be able to edit sections of the name and be presented with a list of available names. A common scenario is to change the prefix of a name. This however, will not give the developer a list of choices in most editors.

Social Network Paranoia

The production of content on modern social networking sites is produced almost entirely by it's users. Additionally, this content accounts for a large portion of the content on the web. Jeff Atkins talks about the sharecroppers of the web. It is both interesting and terrifying to think about. People are working for free to benefit the corporations that enable this content to be published. Although the scary implications of leaving your data in the hands of others is nothing new, the reasons for doing so is interesting. People generally use social networking sites because they enjoy doing it, not because they want to occupy some tiny portion of the content available on the web.

But why does it matter that these big corporations are gaining from other peoples free labor? what else do you have to lose if all your data one day disappears? The time investment that could have been spent on yourself. Although, it is highly unlikely that any of these mainstream social networks are going anywhere soon. So, the odds of losing everything you have invested are currently not all that large.

jQuery Context

How many times have you wished there was an easier way to find DOM elements in a given HTML document? The jQuery javascript toolkit offers a powerful set of CSS selectors that enable developers to do just that; easily query DOM elements. The heart of the jQuery toolkit relies on these selectors. If you are using jQuery in your javascript applications to manipulate the DOM tree, you probably haven't noticed any performance issues while issuing queries. This is because jQuery is highly optimized.

However, the performance can still be improved. The jQuery() function is most often used to query the entire DOM tree. That is, if you issue a query for a DOM id, the selector is applied to every element in the document. More often than not, there is an opportunity to specify a context. For instance, when issuing a query, the developer may know that the given id exists somewhere within another element. Luckily, this element can be specified as a query context when issuing the query.

The benefit to using the context parameter is that the search for the specified DOM elements is narrowed. Possibly by a significant margin. There is another benefit to using the context parameter. Maintainability. Developers can easily deduce why a given query isn't yielding expected results for instance. Finally, the context can be used pragmatically to affect behavior. Below is an example of how to apply the context parameter to DOM queries using the core jQuery() function.

//Example; jQuery context.

var my_div=jQuery("#my_div");
var my_elements=jQuery(".my_elements", my_div);

Transactional Javascript API Interaction

Any meaningful javascript application is going to make use of some kind of application API that lives off on a server somewhere. Generally, some user interface triggers some event which, in turn, will send a request to a remote API. These requests are generally asynchronous. That is, the request happens in the background while the user interface is still responding to events. The number of API connections and requests can add up rather quickly. Even if the data sent over the network for each request and corresponding response are relatively small, there is still the overhead involved with assembling the request.

Using a transactional javascript API access pattern can help cut down on network bandwidth. This is done by minimizing the work done by the API on a given request. Additionally, we also cut down the response size for a given request. The bandwidth consumption will grow to a much larger size if each API response is sent to the javascript application in a single HTTP response. It would be nice if a single HTTP response could contain multiple API responses. One potential solution is to only return a transaction id for each transaction request. The javascript can then store this transaction id. At the core of the javascript application would be an interval that would check the API for completed transactions and their corresponding responses. This approach is illustrated below.

Suing Open Source

In an interesting entry, Bill Snyder talks about a new set of guidelines written by the American Law Institute. These guidelines state that the developers of software should be held responsible for "knowingly" shipping buggy software. Needless to say, something like this is bound to raise some controversy in both the proprietary and the open source software worlds. As the article states, there is some ambiguity in the chosen language of these guidelines. For instance, what constitutes "knowingly". Additionally, what constitutes "buggy"? When it comes to software, both these turns are open for interpretation. f there were a standard set of criteria in which to evaluate software in order to determine its "buggyness", this way be feasible. However, the end user is responsible for reporting bugs. That is, no matter how much quality assurance a given software package goes through, there is bound to be something "wrong" with the software. Now, this comes back to the notion of what constitutes "buggy" or "wrong" or "incorrect". If there were a contract in place that had the written requirements in stone, there might be something substantial here. But these upfront requirements are unrealistic. There is going to be changes needed and it is really up to the developers to decide if they fall into the bug category.

In the open source world, it is stated in virtually any end user agreement that this software should be used at your own risk. If you are an open source end user, file bug reports, be helpful about it and you will be happier in the long run than if you were to take the complaining road.

jQuery each

Performing iteration activities in javascript is actually slightly more complicated a task than it should be. This is brought about by the cross-browser inconsistencies. Developers can't implement standard javascript looping constructs and expect them to work the same way across multiple browsers. This is simply the reality of web application development. The other reality is that if any developer expects to be productive, they will use a javascript toolkit such as jQuery. Thankfully, the jQuery.each() function provides a cross-browser iteration facility. All the developer needs to supply is the sequence to be iterated over and the callback for each element in the sequence. In addition to the cross-browser iteration support, the jQuery.each() function provides flexibility in javascript applications. The function can accept both arrays and objects as the sequence to iterate over.

Here is an example illustrating the jQuery.each() function iterating over an array.

//Example; jQuery.each()

var arr=["One", "Two", "Three"];

jQuery(arr).each(function(){
console.log(this);
});

Also, during iterations, there will more often than not be a need to terminate the loop. This is achieved by returning false from the iteration callback as is illustrated below.

//Example; jQuery.each()

var arr=["One", "Two", "Three"];

jQuery(arr).each(function(){
   console.log(this);
   if(this=="Two"){
       return false;
   }
});