Enomalism 2.1B3

The third beta of Enomalism2.1 has been released. No new features. Just bug fixes and general code refactoring.

The next Enomalism release should be the first release candidate of Enomalism 2.1.

The future of PYPI

I'm actually wondering what the future has in store for the Python Package Index. Can package developers really rely on PYPI for distributing their packages? What happens if PYPI goes away?

For the foreseeable future, I would say that PYPI is a dependable repository for Python package distribution. It is the standard method of installing Python packages and as long as it remains popular, I can't really see it going anywhere. Especially since this helps spread the usage of a growing language.

Another question is how will the Python Enterprise Application Toolkit evolve to incorporate new functionality such as checking for updates on PYPI for all my installed Python packages? If something like this is ever solidified, this will be a major popularity boost for Python.

Python publish subscribe

In keeping with the theme of my previous post, I'd like to mention that the publish-subscribe event pattern can also be used in Python. The zope.event package provides this functionality. For example, If I want to publish a topic when Blog objects in my application are updated, I could do something like:

import zope.event

class Blog:
 def update(self):
     self._update()
     zope.event.notify('blogupdate')

This will send a notification to anyone in my application that has a subscription to blogupdate. To subscribe to this topic, we could do something like:

import zope.event

def sub_blogupdate():
  log('Blog updated.')
zope.event.subscribers.append(sub_blogupdate)

The subscription would obviously do something more important but this serves well to illustrate the general concept. Publish-subscribe comes in handy when there is a chain of complex events. The zope.event package, although very simplistic, allows us to produce higher quality code.

Dojo publish subscribe

The dojotoolkit provides a powerful publish-subscribe event system. In fact, I have yet to see a javascript toolkit that offers this functionality.

In dojo, this system allows developers to write objects that can publish or broadcast events. You can also write objects that subscribe to events. The reason this is so powerful is that it allows you to write code that is maximally decoupled. What does maximally decoupled mean? It means that your separation of concerns is largely taken care of.

For example, suppose I have three components in my javascript application (yes, a javascript application), model, view, controller. In view, I have a dialog widget that I want to reuse in several cases. I don't want to create 25 dialog widgets that only have subtle differences. If this were the case, I would need to also create 25 different events for the OK action of the dialog.

Using the publish-subscribe pattern with dojo, I can pass a topic to the dialog when it is displayed. The dialog then only has one event. This event publishes the topic. Then, any objects that subscribe to the topic in model, view, or controller, can then fulfill their responsibilities.

Another reason to employ this dojo system in your javascript application is for extensibility considerations. Future components simply need to subscribe to the existing topics accordingly.

Enomalism permissions

The current development version of Enomalism (Beta 3), has some interesting permission enhancements in sight. First and foremost, the correct permission validation is almost completely implemented. Second, selecting sets of objects based on permission, as well as other constraints, has been implemented and is being fitted within the application as I write this. This will no doubt result in some performance enhancements which is just a side effect. The main reason it was introduced was to simplify the retrieval of objects that satisfy specified permissions.

Gaphor as a sketching tool

Gaphor is a little UML sketching tool written in Python. It is small, but powerful. While it doesn't support all the fancy enterprise UML modeling features you will find in commercial products (which you would hardly ever use in the open source world), it provides one of the more flexible drawing canvases available.

It is the perfect tool for quickly sketching some object-oriented concepts without concerning yourself with UML "correctness". I think many software people avoid modeling so much because of the common notion that the UML is this big, ugly, complicated mess. It really isn't. Gaphor is not only useful for every day sketches in conjunction with producing production-ready software. It is useful as a primer for getting used to the act of modeling ideas. Not playing around with Gaphor before learning the UML is like writing an operating system before doing "hello.c".

Here is some Gaphor output. Nothing fancy, just easy, readable, sketches.

JSON or XML?

Which is the better format for data representation? JSON? or XML? That is a tough one because there are several dimensions here that constitute better. Some easy differences:

• XML is easier to understand, and therefore better for humans to read.
• JSON is more lightweight, and therefor better for software to read.

So how do you go about deciding which format to go with in the context of your application? If the data is constantly manipulated by humans, the answer is easy; XML. In this scenario, the lightweight of JSON simply doesn't pay off. If the data is seldom interpreted by humans but the data format is transferred across a network (which it most likely will), JSON would be the way to go.

Can the two data formats exist within the same context? Yes. Would it make sense to use both formats in the same application? Not likely. One exception I can think of is when using third-party libraries or tools that require one format or the other but you have already invested heavily in the other format. The two formats are isomorphic enough that there is no magic needed to convert between the two. It is just unnecessary if it can be avoided.

If I were to start developing an application from scratch today, and the choice between the two formats needed to be made, I would most likely choose JSON. There are few reasons to not use it. There exist JSON libraries for all the major programming languages. The only question is human interpretation. It would be pretty easy to build a JSON, tree-style, viewer/editor (I've never seen anything out there that does this, and only this). This JSON editor/viewer will be a topic of further discussion in later posts.

A data pattern

A useful data pattern one can implement is to extract all behavioral features from the abstraction in question, and into a new behavioral super class. This trivial pattern will allow several data implementations of the class inherit the same behavior interface.

For example, here is a simplistic view of this pattern in the context of a blogging system.




Here, we have three classes; BlogBehavior, BlogDB, and BlogMemory. BlogBehavior defines the methods involved with what a blog does. BlogDB defines data access methods for interacting with the database. BlogMemory defines data access methods that will store and retrieve attributes in memory.

Methods defined by instances of BlogDB and BlogMemory may be invoked by methods in BlogBehavior. This would be ideal, because which of the two data instances is being used in the context of the blog system doesn't matter.

The concept of separating abstrations based on storage location, data access technonlogy (same database, different API), is not new. I have witnessed several ocasions, however, where the same behavior is not generalized. I think this is an important practice to keep in touch with when using object technology.

Rich clients are still better than the browser

It seems to me that the user experience one gets when using a browser-based front-end, leaves something to be desired. That's not to say that the browser front-end hasn't come a long way, I just don't think its quite there yet.

The main issue I've come across when building or using web browser interfaces, it the lack of cross-browser support. The browser is a lot like an operating system in that there will never be a common ground between your users. There will always be Windows users and there will always be Linux users and there might always be Mac users (there aren't any die-hard Mac users anymore are there ;)).

The difference being, of course, that with rich clients, the main difference is in how the GUI library of choice is built. The implementation generally stays the same. Even if two browsers support the same feature/syntax, it is never guaranteed that the feature/syntax will remain functional. Just because your rich client is not delivered over HTTP and rendered by a browser, doesn't mean it can't function as a web application. As you probably know, there are countless rich web applications in existence today.

On top of it all, rich clients are much more reactive to user input. This is because there is no javascript to execute which is terribly slow (although it has come a long way). The user experience could potentially be bench marked by test users. Your application could have a web front-end, as well as a rich client front-end. Your test users could then be used to inform you of which experience delivers the better experience.

I thing the browser will eventually be able to deliver a user experience at the same level of quality as a rich client can today. Just not yet. And I'm not going to attempt a guess at when that time will be because these estimates are generally wrong in this business.

Enomalism 2.1B2 Released Early

I wasn't expecting Enomalism 2.1B2 to be released until Monday. Testing went well, so here it is! Enjoy.

Enomalism 2.1B2 Getting Close

The second beta release of Enomalism 2.1 is coming soon. Several minor fixes and enhancements have been made throughout the system. Several improvements have been made in the REST API including new user control functionality. Permission handling has been greatly improved in several core modules.

Although the stability fixes have been minimal, a couple noteable features have been added to the second beta. First, during the Enomalism installation process, existing libvirt domains will be imported into Enomalism. This means that any existing machines, created by some other means, can be managed by Enomalism.

Second, if the host Enomalism machine is shut down, the VMs running on that host will now remember their state and automatically start when Enomalism starts again.

Flexible SQLObject queries

Although SQLObject is considered an ORM (Object Relational Mapper), it can still be used for traditional SQL queries. The question is, why bother adding this ORM overhead when the queries can be executed directly by a SQL library?

For anything but the most trivial queries, SQL syntax is still the superior method to manipulate data sets. When using a SQL library in Python, without an ORM, the result sets need to be constructed. This boiler plate code can be handled by SQLObject while still having the ability to use more elegant, or extremely complex in some cases, SQL. For example, if you have defined a couple SQLObject classes, blogAccount, and blogEntry, we can do somthing similar to:

result=blogEntry.select("""blog_account.id=blog_entry.blog_account_id
                          AND blog_account.user_id=%s"""%getCurrentUserID(),\
                          clauseTables=['blog_account'])

This result set will contain a list of blogEntry instances. This is useful because no code was needed to construct the result set. This is obviously not the worlds most complex query, but it does illustrate the idea that fancy SQL queries can still be combined with the concept of an ORM.

Exception driven development

In software, an exception is an event that occurs unexpectedly.  An exception is exceptional behavior.  In any modern object-oriented programming language, developers can define their on exceptions that inherit from the languages' base exception class.  Defining your own exceptions and using them as flow control mechanisms in your code, if used cautiously, can yield very robust code.

The main use case for using this pattern is to simplify large branching flows of control.  For example, say you have defined some method m1 that is invoked in another method you currently implementing, m2.  The control flow in m2 is determined by the outcome of m1.  There could potentially be several new branching flows in m2 each with subsequent levels of branching.  This would eventually make m2 unmaintainable.  An alternative is to implement m2 to only handle the basic success case.  Any other cases are handled by custom exceptions.  In this case, the invocation of m1 would have to handle everything that m1 can raise.  Other than that, it will carry on as normal.  m2 shouldn't care so much about m1s' failures.

The exceptions handle all the exceptional flows in you application.  That is the basic idea.  You have to use your imagination to build an exceptional vocabulary in the domain in which you are working.  The exercise of building a vocabulary of what can go wrong in your system is a useful tool in its own right.  

Happy birthday

Happy birthday Melissa! I hope your birthday (month) is something truly memorable.

Love Adam

Everything should be iterative.

In software, everything should be iterative. Not only that, but the size of the set being iterated over should never be statically set in code. Functions and methods should never return a multiplicity of one. The exception in this case would be if the function simply performs some action and we are not interested in the result.

This is the backbone of extensibility in software development. As soon as you impose multiplicity limits on objects, you are essentially eliminating any chance for extension in that component. If you have some method of some object that returns another software object that is not iterated-over, the chance for that method to return a dynamically determined number of is gone.

What if the behavior caused by the result of some method invocation should differ depending on the length of the number of objects returned? This is seldom the case and you can always have alternative flows of control depending on the length of the result. This extra bit of code in extreme cases is a small price to pay for the extensibility this patter offers.

Try something new or use what you know?

Good question. What would a software object do? Well, it will do what its creator tells it to do. And a software object knows only what its creator tells it. This doesn't seem right. What about the system in which the object lives? The system is alive and constantly changing state. Shouldn't the software object know about these changes?

Well, in some cases, they do. You pass external variables into your object by sending your object messages. This is the fancy term for "method invocation". Your object then has knowledge about its environment.

But what about your software object's attribute that are initialized during construction? Sometimes these values are hard-coded and can only be changed further down the lifeline of the object. Is this the best way to initialize attribute values? Or are we better off allowing for the possibility of dynamic attribute value initialization?

TurboGears allows developers to retrieve configuration values using a key. It also allows the developer to specify a default value should the key not exist. Using this feature, or a similar pattern for that matter, we can initialize attribute values by checking if there is something new that our software object didn't previously know about. If not, it will always use the specified default.

Enomalism 2.1B1

Enomalism 2.1 Beta 1 is now available for download. We are currently building system packages and they will be available on sourceforge shortly.

The first beta of Enomalism 2.1 brings many improvements to the Enomalism core as well as several usability enhancements. There is also much improved documentation available.

Rather than go through the entire list of changes, I'll highlight some of the most important improvements. The exception-handling has been greatly improved since the last release. Enomalism defines its' own exceptions which are then stored in the database so they can be viewed at a later time. Storing exceptions can be turned off. There is also an exception viewer extension module which allows exceptions to be viewed in the Enomalism front-end.

Several GUI elements have been improved from a usability perspective and general cleanup. The Enomalism core itself has been cleaned up too. Finally, the REST API has also been improved. Although it is not quite there yet in terms of consistancy, it is much better than the previous release. The REST API is a high-priority feature that will continually improve as the project progresses.

That being said, try it out. Give us your feedback and we will do our best to resolve any issues you may encounter in the next release.

Python Entry Points

What are entry points? Entry points are a way to expose Python objects to other Python packages on a given system. See the PEAK documentation for a detailed reference. Other Python packages can iterate through various groups of entry points, and if your package has defined an entry point within this group, the entry point is dealt with polymorphically.

This is a good way to "glue" Python applications together. If your Python package consumes entry points, you should document the interfaces that are required by all the entry points your package may encounter. This way, other developers can implement packages with entry points that conform to your entry point requirements. Once this is complete, all they need to do is install their package. Thats it. The consuming package knows where to look.

Whats missing is a GUI front-end or a command line utility that will query and display the various entry points on any given system. Another, and more importantly, missing piece of the puzzle is a front-end Python package manager.

Identifying with uuid.

I've noticed that a good way to uniquely identify software objects is using uuid. Python has a built-in uuid module that allows developers to generate unique ids when needed. The notion of uniquely identifying software objects is old news and hence not really news at all.

However, when keeping in mind that you way want a given software object to be uniquely identifiable in environments remote to the environment in which the object was constructed, you may want every object in your system to have a uuid.

This is easy to achieve in Python, and any other language with uuid support for that matter. Simply create a base class that will act as the unique indentifier generator. This class should hold an id attribute which stores the generated uuid. Any class you implement for this point on can inherit form this base class if they need a uuid.

REST is better than all the REST

Representational State Transfer (REST) is by far the superior architecture when building web-based applications. What exactly is REST? Put simply, it is a means in which disparate software objects communicate state information to one another. REST can be used by practically any programming environment, and it is fast because it is lightweight.

The most powerful aspect of REST is that it is extremely easy to understand and use. By contrast, using SOAP (Simple Object Access Protocol) is anything but simple to use and understand. REST is most commonly used to access remote objects, similar to what SOAP is intended to do. Each object can be accessed using a URI for the object. An example REST URI might look something like:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Here, the unique aspect of the URL is the trailing user identifier. Using REST, we can tell the URI what action we want to take place by specifying the HTTP method. The HTTP method represents the action we want to perform on the object (the URI). And we can pass parameters to these actions using the typical HTTP request approach.

Here are some examples of how you can use the HTTP method to manipulate remote objects using REST.

Using GET to retrieve an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Using POST to update an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0?username=newname

Using PUT to create an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0?username=newname

Using DELETE to delete an object:

/rest/user/2b0a369e-e537-11dc-aa0f-0019d2b28af0/

Enomalism has a fully-featured REST API that will allow any client that has the necessary permissions defined, to access the entire Enomalism functionality.