Value In The Cloud

Cloud computing is definitely one of the top buzzwords in technology today. Yet, there doesn't seem to be any consensus on what it all means. Yes, many people have there own opinions or definitions of the topic, each slightly different from one another. There aren't many agreed-upon components that make up cloud computing technology. At least one thing remains unclear; where is the value for the end user in all of this?

To better understand the some of the more obscure value propositions offered by cloud computing technology, we need to understand the shortcomings of the current technology available. That is, the current technology that allows people to make applications available over the web. There is no shortage of hosting providers out there that make it easy for both businesses and individuals to deploy an application over the web. In fact, these providers often have the application pre-built and ready to go. It really depends on what you are trying to do on the web and if it falls within the realm of the commonplace, you're in luck.

Once you want to start requiring customizations that these pre-built applications simply cannot provide, it is time to build something. If you're not a developer, you need a team of them to build you a web application that does what you need. This means that you need to figure out which framework to use, what platform it will run on, etc. These are all details that just don't interest non-technical folks. Nevertheless, it is a reality in order to survive these days.

Whether you use a pre-built, or a custom-built web application, having a presence on the web is the ultimate goal. The Internet has an ever growing user base and without if you're not there, you'll go unnoticed. The web is a big part of cloud computing but that isn't the whole story. What about desktop applications? Do they simply have no place in cloud computing? Like it or not, desktop applications are still in heavy use today. Some development efforts involving re-creating the same desktop applications that will run in a web browser. This is nice to have but it would also be nice if we could move desktop applications into the cloud without re-inventing the wheel.

Virtualization allows us to do just that. We can run our desktop applications inside a virtual machine without having to re-write the entire program. We can also access these virtual machines remotely so we don't necessarily need to see the user interface inside a web browser, even though it is possible to do so. Virtualization is another key component of cloud computing. In fact, it is probably the key to differentiating cloud computing from a more traditional web application deployment. Virtual machines can be created an destroyed upon request. They can also be moved around to different physical nodes without interruption to the running application contained within the virtual machine.

There are several well known virtualzation technologies available today that service providers can use to their advantage. If they have the hardware, they can make the these cloud computing resources available to their customers. One shortfall to using these virtualization platforms is that they are missing several key components necessary in order to provide a cloud offering. For instance, we're doing some interesting things here at Enomaly with our cloud computing platform, ECP. ECP offers features that are essential to service providers such as multi-tenancy and a highly-customizable user portal.

Having said all this, what is it about cloud computing that really sets it apart from a more traditional web application deployment? That is, where is the value? I believe the value of cloud computing is enabled by virtualization. This gives us a level of freedom to do what we want with our deployed applications previously unheard of. What can be now be done with cloud computing can also be done with a more traditional deployment, it will just cost a lot more. When you loose the ability to create you're own environment precisely as you need it, it suddenly becomes much more difficult to do things.

Virtual machines are their own self contained environment so they can also be copied. That is, once you have an environment setup the way it needs to be setup, that same work is never done twice. You simply clone the virtual machine if you need more capacity, or for some other reason. What this means is that you never have to do the same thing twice and that translates to much time saved.

This customization work can even be performed by the cloud providers themselves. This gives them an opportunity to not only provide the cloud computing services, but to also add value by providing appliances to their customers. These appliances are virtual machines that have been built for a common purpose. Typically, they will have some commonly used application stack installed and minimally configured.

In summation, there is value in cloud computing that is often overlooked because it is obscured by the more technical aspects of trying to define what exactly cloud computing is. The hidden added value cloud computing offers is sometimes hard to see unless you're using the technology that makes it possible. Mind you, this technology is still in its' infancy but the end results all look very promising.

Using KVM with ECP on Ubuntu

A great step by step guide on how to use KVM virtualization with the Enomaly Elastic Computing Platform.

Registering configuration values in ECP

The Enomaly Elastic Computing Platform has an extension module API that allows developers to register new ECP components. These components include new web controllers, new RESTful API controllers, and so on. One component that cannot be registered are configuration values. Extension modules can be viewed as smaller applications that are executed within ECP. Therefore, these smaller extension module applications will need to be configured. There are always going to be values that should be configurable within any application such as storage locations. Currently, extension modules must implement their own settings abstractions. This functionality already exists in the ECP core and the way configuration values are accessed and stored should be consistent and hence the need for the custom settings class. It would make sense for extension modules in ECP to have the ability to register their own configuration values. This way, configuration values would be accessed and stored in the exact same way across the platform. An additional complication arises when trying to use the configuration editor. The configuration editor is tightly-coupled with TurboGears widgets and thus requires that all extension modules be tightly-coupled with TurboGears widgets. Ideally, when configuration values are registered, which currently is not possible, additional meta data suitable for generating a display widget for the configuration value could also be registered.

The current implementation of the ECP Settings class uses managed Python attributes to seamlessly save and load configuration values. Every time a managed Settings attribute is accessed, the Variable class will attempt to load the variable. Likewise, when a managed Settings attribute is altered, the Variable class will attempt to store the configuration value. It is easier to use managed attributes for simple storage and retrieval operations. The alternative is to use the Variable class directly. In fact, earlier implementations of ECP did exactly that. Every time a configuration value was needed, we had to invoke Variable.load() while specifying a default value in case the configuration value didn't exist. The new Settings class was introduced to help alleviate some of this troubled configuration access. A single instance of the Settings class is created in the configuration.py module. This instance can then be used throughout the ECP application, including extension modules. Configuration categories are also incorporated into the Settings class. This is done by using the same concept as the Settings class for each category. This category class is then set as an attribute of Settings. This allows us to access configuration values in the form of settings.kvm.bridge. This syntax offers much more readable code when used in context. However, the problem with this method of managing configuration values was soon after realized. There will always be a need to add new configuration values. Most noteably, extension modules are going to need this capability since developers are going to want to access configuration values in the same way as the rest of ECP. There is a need to be able to register new configuration values. This eliminates the extensibility problem of adding new configuration values. If every time a new configuration value needed by an extension module, or the core application for that matter, needs to be added to the Settings class, it will grow exponentially and become very challenging to maintain. Additionally, the configuration editor is very tightly coupled to TurboGears widgets because extension modules need to display these configuration values in the configuration editor. This is done by the extension module defining a hook that passes in TurboGears widgets used to display the configuration values for the extension module in the configuration editor. This isn't the ideal method since this also couples the extension modules to ECP dependencies (TurboGears). Ideally, the widgets for displaying configuration values should be generated by the configuration editor based on minimal meta-data provided by the extension module at registration time.

The new approach to ECP configuration value management is to have configuration values registered in the Settings class. The same approach of using managed attributes to access and store configuration values is still used. What is different is the ability to register a value and have these managed attributes automatically built for the developer. This is accomplished by introducing a new MetaSettings class. The purpose behind this new class is to dynamically construct new categorization classes and methods that will become attributes of the settings instance. There is also a new settings.register() method that can be used to register new configuration values. The end result of using settings.register() to register a new configuration value is the same syntax as before when using the configuration values. The name of the module passed to settings.register() will become an attribute of the settings instance. There are also meta-data parameters in the settings.register() method that allow developers to specify a title and description of the configuration value. In the current ECP configuration management implementation, this information must be specified in the TurboGears widget. With the new implementation, the managed attribute functionality found in the ECP core no longer needs to be duplicated. There is now a much more uniform interface.

With this new configuration registration functionality in place, there is now an opportunity for great improvements in the configuration editor. We could now potentially eliminate the coupling to TurboGears widgets and have each configuration widget generated automatically. Grouping by extension module is now also possible in the configuration editor.

ECP three-level machine abstraction

The Enomaly Elastic Computing Platform is a platform for managing distributed virtual machines. Therefore, we need some type of abstract representation of this concept. This requirement isn't really any different from any other software problem. There is a problem domain which contain concepts unique to that domain. Developers will then try to capture what that concept represents in that domain by creating an abstraction. By creating abstractions in this manor, we lower the representational gap between the domain and how concepts in that domain are realized in the solution. In the case of ECP, there is a real need to represent the idea of machines.

In any given software solution, the abstraction created by developers may be a very simple, single layer abstraction architecture or there could potentially be several layers within the architecture, yielding an extremely complex architecture. In the latter case, without a well thought out design, we start to lose the value that creating an abstraction brings in the first place. Sometimes, when dealing with a large abstraction, further dividing this abstraction into layers can help to better understand what you as a developer are actually implementing. Often, the abstraction design is further complicated by constraints imposed by the system or framework within which we are developing. Rationale, interfaces, and consistency in general, need to be taken into consideration when constructing a layered abstraction architecture.

To implement the machine abstraction, ECP uses a three-level approach to realizing this abstraction. In this architecture, each level is a class that realizes a different level of the "machine" concept, and for different purposes than other layers. In this implementation, the three levels hierarchical. At the top level, we have a class called ActualMachine which implements several methods for invoking machine behavior. The next level contains a class called DummyMachine that inherits from ActualMachine and doesn't do much. Finally, we have a Machine class that can store persistent data to the database. Hierarchically, the DummyMachine and Machine classes are at the same level since they both inherit from ActualMachine. In this discussion, however, the levels aren't necessarily based on the class hierarchy but rather based on the rationale behind each class.

The ActualMachine class is meant to most closely represent the concept of "machine" in the context of ECP. The same symbolizes that this is the underlying machine, not a Python object. Obviously, instances of ActualMachine are Python objects but when using these objects, we are more interested in what the underlying technology. This class is where all the behavior for the machine concept is defined. This class doesn't define any data attributes.

The DummyMachine class is exactly what the name implies; a dummy. The class simply defines a constructor that allows attributes to be set. Also, the class inherits all the behavior from ActualMachine. Instances of DummyMachine can set attributes in the constructor and invoke behavior provided by ActualMachine.

The Machine class provides persistence for the machine abstraction in ECP. The class also inherits behavior from the ActualMachine class. Machine functions similar to DummyMachine in that they both provide the same behavior. The difference between the two is that DummyMachine stores attributes in memory while Machine stores attributes in the database.

The rationale behind this architecture is that we want to be able to instantiate machine instances while not affecting the database. The opposite is also true; we need to be able to instantiate machines that will have an immediate effect on the database. Within the context of the ECP RESTful API, machines that are not stored on the local machine (they are retrieved from another ECP host), will need to be instantiated. That is, we want to have an abstraction available to use once the remote machine data has arrived. This can be done by using some primitive data construct such as a list or a dictionary, but by doing this we lose the machine concept. The behavioral aspect of the machine concept is gone because you can't tell a dictionary to shutdown.

There are still several limitations to this approach. For instance, not all ActualMachine behavior will be supported by the DummyMachine instances that are created. This is simply a limitation of the three classes and their inter-relationships. It is still an improvement over representing domain concepts using primitive types. We give ourselves more control in the three-level architecture over what happens when requested behavior cannot be fulfilled. The DummyMachine layer is an example of mixing the problem domain with the solution domain. The class came into existence because the solution demanded it. But this design allows for the behavior provided by the machine instances to still behave like "machines" without conforming too much to the solution constraints.

A similar approach is taken in ECP with other abstractions such as packages. The architecture hasn't been fully implemented for every abstraction within the platform. It will hopefully prove to add some balance between constraints and offered functionality.

I'm sure this approach prove useful in many other application areas. As objects become more and more distributed, we'll need a better way to represent their data when used locally while preserving the behavior of that object.

ECP and the future extension module architecture

Developers that have been using the Enomaly Elastic Computing Platform over the past year, myself included, have encountered some bottlenecks in the ECP extension module architecture. These aren't show-stoppers in every case but sometimes, they are. It is mostly an issue of architectural design such as "what is the rationale behind this API method?" and "if I build it this way, what is the impact of change in other areas going to bring?" Once we took a step back to think about such questions, we came to the conclusion that these questions should be apparent to any developer using the platform or at least easily answered by Enomaly. Right now, much of the ECP extension module framework isn't apparent how to use and we even have a hard time explaining it.

So, this has led the ECP development team to address some of the issues highlighted here.

One of the first major problems is a problem of uniformity and consistency among the core extension modules that ship with ECP. These extension modules aren't exactly consistent with one another. Some modules will use sections of the API as intended while others will use different sections and others, still, don't use the API at all. In fact, some extension module logic in ECP is coded directly in the core system. This doesn't necessarily cause any harm to anyone who wants to install the base system because these modules are "part" of the base system. They are simply constructed as extension modules. Earlier on in ECP's lifespan, we needed to construct an extension module API and these core modules were the perfect way to test out our ideas. So, either way, these core extension modules could have been built in to the core code base. But, it would be nice if they weren't so irreplaceable.

The first step is to introduce a new level of consistency among the extension module that are distributed with ECP. If nothing else, they can serve as useful examples for extension module developers.

The next major defective area within the ECP extension module API is the API itself. Or, rather, lack thereof. What I mean here is that there are plenty of smaller areas of ECP that should be extensible but aren't. For example, if there is some thing small in the ECP front-end GUI that a developer wants to extend, they must replace the entire template, duplicating many already existing elements. This means that many of the core elements, including GUI widgets, would need to become part of a bigger extension module framework. That is, we would need to move them to extension modules. And that is fine with me. A smaller core is easier to maintain and thus more stable.

There have already been some changes introduced to the ECP extension module framework in the past year. We identified the need for extension modules to store their own static data such as javascript and CSS files. To address this, we introduced methods to register these static components.

There is still a decent amount of work to do in order to realize these changes. They have been identified and that is a very good thing. Also, this is by no means a closed list of issues with the extension module framework that need fixing. This is a good starting point. I've already begun fixing the consistency problem with the core extension modules.

ECP 2.2.3 released

The latest stable version of the Enomaly Elastic Computing Platform has been released. Changes include:

• New approach to handling an index error caused by SQLObject.
• Better handling of ECP clustering when the host machine record does not exist in the database.
• Fixed a defect in the way remote eggs are installed with the vmfeed extension module.
• Better exception handling when importing existing Libvirt domains.
• Fixed several invalid javascript references.
• Small CSS fix.

ECP and SQLObject

It seems that everyday we have a new reason to move away from using SQLObject as an object-relational mapper in ECP. The latest issue with SQLObject has been rather challenging to work-around. The problem comes from the ErrorMessage class defined in SQLObject. Here is what the class looks like.

#SQLObject ErrorMessage class.

class ErrorMessage(str):
  def __new__(cls, e, append_msg=''):
      obj = str.__new__(cls, e[1] + append_msg)
      obj.code = int(e[0])
      obj.module = e.__module__
      obj.exception = e.__class__.__name__
      return obj

The problem we are experiencing with ECP is the fact that this class always raises an IndexError. The reason being, the ErrorMessage.__new__() method makes the assumption that the 0 and 1 indices will always be available in the e parameter. The e parameter is supposed to be an instance of Exception.

The question that now arises is how do we handle this? In this case, we have exceptions being raised by other exceptions. The ErrorMessage class could simply be fixed by adding exception handling for IndexError exceptions. However, now that the error is fixed, how do we ship this fix along with our application? ECP will currently install SQLObject from pypi. One solution would be to build our own SQLObject package and point the ECP setup to a custom repository that contains this patched-version. One problem I find with this solution is that it could potentially introduce a myriad of other deployment problems.

Another solution is to perform the patch inside of ECP. In this scenario, we don't actually patch the SQLObject package. The SQLObject package would remain as is on the system so that other Python applications using SQLObject wouldn't experience any side-effects as a result of ECP providing a different SQLObject. And this is the approach we are taking. Once ECP has started up, we import the mysqlconnection module and replace ErrorMessage entirely. Here is how it is done.

#ECP approach to patching SQLObject.

from sqlobject.mysql import mysqlconnection

class NewErrorMessage(str):
   def __new__(cls, e):
       if not isinstance(e, Exception):
           e = e.args[0]
       else:
           try:
               dummy = e[1]
           except IndexError:
               e = e.args[0]
                                      
       obj = str.__new__(cls, e[1])
       obj.code = int(e[0])
       obj.module = e.__module__
       obj.exception = e.__class__.__name__
       return obj
 
mysqlconnection.ErrorMessage = NewErrorMessage

What is shown here is a new implementation of the ErrorMessage class; NewErrorMessage. The interface of the original class is kept in tact. What has changed is the exception handling inside the exception. We first test if the e parameter is in fact an Exception instance. Next, we test for IndexError exceptions and rebuild the e parameter if necessary. The method then continues on as in the original implementation. Finally, we then replace the ErrorMessage class with NewErrorMessage. This all happens in enomalism2d so that the new error message class is available right away, before it is actually needed.

As an afterthought, I'm wondering what led SQLObject to this issue to begin with. That is, how can a class so tightly associated with exception handling be the culprit for bigger problems such as this one? Is it that this class is taking on too many responsibilities and thus adding to the risk of raising unforeseen exceptions itself? I wouldn't think so. The ErrorMessage.__new__() method isn't exactly overwhelmed with code. Besides, the exceptions defined in ECP do a fair amount of work when instantiated (including interacting with SQLObject). When the ECP exceptions are raised, they never raise inadvertent exceptions.

Perhaps special care needs to be taken when defining exceptions that do any work. If nothing else, SQLObject provides us with a lesson learned. As developers, we need to be absolutely certain that any given exception we have defined ourselves can be raised under any circumstances. They cannot fail. It would obviously be nice of no code failed at all throughout an entire application. That is obviously not a reality though. The code will fail at some point and having stable exception to deal with will make your code one step closer to being fail safe.

The dynamic configuration problem

Most applications, especially web applications require some type of configuration management. The configuration data is usually specified in a .cfg file in the application directory. Most languages and web frameworks provide utilities to parse and read these configuration files. These than become variables, possibly altering the control flow, in the application itself.

In many cases, users of these applications do not want to edit configuration files. I know I don't want to half the time. The solution? Integrate the configuration into the GUI. Perfect. So now we have a pretty interface that users (often administrators) can use to update these configuration values. One problem here is where do these values get stored once they have been updated? It is probably not a good idea to alter the configuration file. Or is it?

In ECP the approach is to store the altered configuration values from the web front-end into the database. This is good because we don't need to mess around with writing code in the application to manipulate the configuration file every time a configuration value is changed. When a configuration value is needed somewhere in the application code, ECP will check the database for the configuration value. If the value is found, this value is used in the application.

The problem here may be obvious but I'll mention it anyway. What happens when a configuration value in the configuration file is updated and that same value already exists in the database? Well, as it turns out, that change will have no effect.

What if we were to change the order precedence of the configuration value lookup; if the configuration file were checked before the database? In this case, we have lost the front-end GUI capability. The reason I don't like storing updated configuration values in the configuration files is because that isn't why they are there. The configuration files exist for humans to edit and they are generally quite annoyed when a machine starts messing with the configuration.

So how do we fix this configuration race condition? We have a value that is stored in the database and that same value is stored in the configuration file. Ideally, it would make more to support both configuration storage locations; the file and the database. At the most basic level, we are trying to solve the problem of the most recent value. The most recent configuration value should be used in the application code. One step to take on the database side is to store the modification date for each configuration value. At least then we can determine how fresh the database value is. On the configuration file side, this isn't really feasible since the file is edited by humans. We can't expect them to enter the current timestamp every time the update a value just so their application will behave as expected.

We can't use the modification date of the configuration file because this date doesn't reflect which value was updated. What we could do is introduce some new configuration fields to the database. The table might look like the following.



Here, we have four fields; db_value, db_tstamp, file_value, and file_tstamp. The db_value field stores the value the is sent from the application front-end. The db_tstamp value is updated with the current timestamp when the db_value field is updated. When the application is started for the first time, all file configuration values would then be copied into the database in the file_value and file_tstamp fields. At this point, we would have several file values stored in the database but no front-end editor values.

The reason we would need these file_value and file_tstamp fields is to compare them with the db_value and db_tstamp fields. When a lookup happens, the lookup functionality would read the value from the configuration file. If the value is different, we update the file_value and file_tstamp fields. We would then return either the db_value or file_value based on whichever is greater; db_tstamp or file_tstamp.

ECP and IP address retrieval

The Enomaly Elastic Computing Platform has to retrieve the IP address of the host in which it is running. There are several reasons for this but this isn't the reason for this entry. The concern here is how the IP address is actually obtained from the host machine. This must be accomplished in Python since that is what ECP is written in.

There are several Python recipies available for obtaining the IP address of a given machine. However, on Linux, you must know the which network interface you are retrieving the address from. In the current release of ECP, the default network interface to check is virbr0 since this is the interface that is setup when ECP is started. There are a few problems with this approach.

First, I noticed that there is no real way to change which network interface ECP will use as a basis for IP address retrieval. However, this has been addressed in the ECP trunk. Users may now specify a specific interface in the configuration by providing a enomalism2.ip_interface value. If this value is provided, ECP will check this interface for the machine IP address. One thing that hasn't changed about the host IP address retrieval in ECP is the fact that when all else fails, 127.0.0.1 is returned. This is a good thing since we can always identify a machine IP address as opposed to an empty string.

Secondly, ECP hosts may have several active network interfaces. What then? This deficiency was pointed out here. This is a very valid concern for some people. What would be needed here is a way to associate network interfaces with users in ECP. This results in a new database abstraction that could have some potential in a future ECP release. For the time being, users are restricted to a single network interface that is at least configurable now.

ECP and the ActualMachine

The ActualMachine class in the Enomaly Elastic Computing platform is the abstraction layer that interacts with Libvirt to control virtual machines. Not every method of this class interacts with Libvirt directly. In fact, most methods in the ActualMachine class don't directly interact with Libvirt. There are plenty of methods in this class. The ECP development team is in agreement that this number can certainly be brought down to something a little more cohesive.

To start, we'd like to focus on maintaining the handful of ActualMachine methods that interact directly with Libvirt. As an example, the shutdown() method directly interacts with Libvirt. This is necessary in order to shutdown the virtual machine.

The reason we are going to tackle these Libvirt-coupled methods first is because they are absolutely critical to ECP. Without a clear understanding of how these primitive abstractions behave, we hit a brick wall. I think the ActualMachine class gained a lot of weight in turns of the sheer number of methods because we didn't understand what was happening at the primitive level. By primitive, I mean the primitive reason for ActualMachine's existence. Once we are able to hammer out what it is that the ActualMachine is responsible for, we can lose a lot of the current functionality. Losing it is a good thing in this case. It will come back in a much nicer form.

For an idea of what is happening with the ActualMachine class, take a look at this.

ECP update

Over the last week or so, the ECP development team has been fixing several issues brought about by the ECP user community. One issue that has been resolved I'd like to point out.

The problem here is that during the ECP installation procedure, any existing Libvirt domains will be imported into the database. Obviously, the machine table must already exist in the database. When the user hits the "/install" url, the installer is run. The user may perform this action even after all the ECP database tables have been defined. This way, any Libvirt machines that have been created by some other machines may be imported.

One of the problems with this method is that sometimes, the machine import functionality is executed before the machine database table exists. Another requirement of the machine import functionality is that the local machine database record has been inserted into the machine database table. This is needed to determine what hypervisors are available on the local machine.

The fix made in this case was to check in the machine table exists. ECP cannot execute this functionality without it. Second, ECP will no longer assume that the local machine exists in the database. It will now check for both the machine table and the local machine record. If either is false, no machines will be imported. In this case, the "/install" url can always be reloaded once the required tables and records have been created.

So, how do we end up in a situation like this in the first place? Shouldn't the ECP installation functionality always ensure that the required data exists before it is needed? This is a perfectly valid concern and the installer does work this way. The table creation is the first task executed by the installer. Then, important records such as the local machine are inserted. The only way this ordering can be altered is if some extension functionality "hooks" into the installer. The hooks have a choice as to the order in which the "hooked" method is executed. The original invocation may happen before the new functionality or after. It is entirely the responsibility of the extension module to ensure that nothing is interrupted in the original behaviour.

That being said, there are several extension modules distributed along with ECP and we'll be keeping a close eye on those as usual. If anyone has noticed a potential defect in a core extension module, feel free to report it here.

New ECP exception and enhanced state restoring behaviour

Over the past few days, the ECP team has made some notable enhancements in the trunk. The first being the addition of a new exception called E2LibvirtError. As the title suggests, this exception is raised for Libvirt-related issues. The Python Libvirt library already defines an exception class. However, there are many types of errors that can happen from within Libvirt. The idea behind this new E2LibvirtError class is to provide better information when something bad happens in libvirt. For instance, in the Python Libvirt library, there is only one exception type. If this exception gets raised, a short message is displayed. This is the default message that gets initialized with the Python base exception class.

The problem here being that Libvirt can manage several different hypervisors on any given system. Thus, there are several layers within Libvirt in which something can go wrong. In ECP, the Libvirt exception is caught, and the generic message is recorded.

The new E2LibvirtError exception exploits additional exception information encapsulated within the basic Libvirt exception instance. I don't mean encapsulated in the traditional object-oriented sense. I mean the information is there, and ECP should use it for the benefit of the end user. The new ECP exception, when instantiated, will take several error codes from the original Libvirt exception and produce a much more meaningful error message.

This leads me to the changes made in the restore_machines_state() functionality. The rationale hasn't changed, only the implementation. We simply handle table existence and local machine existence detection much better than the current version. If the function finds a machine that is not running and it should be (because that was the state the machine was in when ECP last shut down), it will attempt to start it. We've already added the new E2LibvirtError exception handling to this function when attempting to start the machine since this is a Libvirt operation. I've already been seeing much more useful error messages in the logs. These new error messages should also be viewable in the web front-end via the error dialog box when something Libvirt-related goes wrong.

This does increase the Libvirt coupling in ECP a considerable amount. However, given the level of functionality that ECP would have without Libvirt, I think it is a fair trade off.

New ECP community site.

I'm pleased to announce that the new Enomaly ECP community site is up and running. Feel free to report bugs, request features, or check out the documentation.

ECP 2.2.2 released

The 2.2.2 maintenance release of ECP is now available. Changes include:

• SECURITY FIX: Removed the automatic installation of egg files from ECP start-up.
• Added more exception handling to failed ECP startups due to missing database tables.

ECP 2.2.1 released

The 2.2.1 release Enomaly Elastic Computing Platform is now available. This is a bugfix release. Changes and fixes include:

• Fixed a potential spoofing exploit in the enomalism startup process.
• Fixed an import error in the permission_control extension module.
• Removed non-functional GUI widgets for host machines.
• Fixed several invalid jQuery references.
• Fixed a unicode issue in the exceptions module.
• Fixed database exception handling in the exceptions module.
• Fixed database exception handling in the restore machine state functionality.

libvirt 0.6.0

Looks like libvirt 0.6.0 is now available. There are several new features, bug fixes, and improvements. I just tested this version with ECP 2.2 and everything works just fine.

ECP 2.2 released

ECP 2.2 has finally been released. Outlined below are the changes.

Core

• The ECP installer will now automatically generate a uuid for the host. Also, the installer will now synchronize with the local package repository if one exists. Several Xen fixes are now carried out by the installer that allow ECP to better manage Xen machines. The exception handling has also been drastically improved in the installation process.
• The core ECP data module has many new features as well as many bug fixes. Several subtle but detrimental object retrieval issues have been resolved. This alone fixed several issues that were thought to be GUI related in previous ECP versions. The new features include added flexibility to existing querying components and newer, higher level, components have been implemented. These newer components build on the existing components and will provide faster querying in ECP.
• The configuration system has gone through a major improvement. It is now much easier and efficient to both retrieve and store configuration data. This affects nearly any ECP component that requires configuration values.
• The extension module API now allows extension modules to register static directories as well as javascript. Some of the core extension modules are already taking advantage of this new offered capability. This helps balance the distribution of responsibilities and increases the separation of concerns among ECP components.

GUI

• There have been many template improvements that promote cross-browser compatibility. Many superfluous HTML elements have been removed and others now better conform to the HTML standard.
• A new jQuery dialog widget has been implemented. This widget is much more robust and visually appealing than the dialog used in previous ECP versions.
• General javascript enhancements will give the client a nice performance boost and improve on the overall client experience.

Testing

• With an emphasis on improving the ECP RESTful API design in this release, the requirement for automatically invoking various ECP resources came about. Included in this release is a new client testing facility that can run tests on any ECP installation. Although the tests are limited, they continue to be expanded with each new ECP release.

vmfeed (extension module)

• A big effort has been undertaken in analyzing the deficiencies with the previous versions of the vmfeed extension module in order to drastically improve its' design for this release. One of the major problems was the lack of consistency in the RESTful API provided by vmfeed. Some of the resource names within the API were ambiguous at best while some important resources were missing entirely. There has been a big improvement in both areas for this release.
• Another problem was the actual design of the code that actually drives the extension module. Much of the code in vmfeed has been re-factored in order to produce a coherent unit of functionality. As always, there is still room for improvement which will come much more easily in future iterations as a result of these changes.

machinecontrol (extension module)

• In previous ECP versions, when operating in clustered mode, removal of remote hosts was not possible. This has been corrected in this release.
• The machinecontrol extension module will now take advantage of the new ECP configuration functionality.
• When deleting machines, they are now actually undefined by libvirt.

static_networks (extension_module)

• The static_networks extension module will now use the newer ECP core functionality in determining the method of the HTTP request.
• Refactoring has taken place to remove the static_networks javascript from the core and into the actual extension module package. This improves the design of both the static_networks extension module while reducing the complexity of the ECP core.
• The static_networks extension module will now take advantage of the new ECP configuration functionality.

transactioncontrol (extension module)

• The transactioncontrol extension module will now use the newer ECP core functionality in determining the method of the HTTP request.

clustercontrol (extension module)

• Major improvement in the RESTful API design. Some invalid resources were removed while others were improved upon.
• The clustercontrol extension module will now use the newer ECP core functionality in determining the method of the HTTP request.
• The clustercontrol extension module will now use the newer ECP core functionality in determining the method of the HTTP request.

Understanding hooks in ECP.

Hooks in ECP are Python decorators that allow developers to replace existing methods by hooking into them. In fact, there is enough flexibility to decide at run time if the original invocation should be replaced by something new. Possibly depending on the state of some other object in the system.

The hook() decorator accepts two parameters: object, and method. The object parameter specifies the object in which defines the method to be hooked. The method parameter specifies the method to be hooked. The decorated function, the actual hook, must specify the same operation signature as the original method invocation.

For instance, lets say we want to hook the Package.get_name() method. We would define the hook as follows.

#ECP hook demonstration

from enomalism2.model import Package, hook

@hook(Package, Package.get_name)
def hook_get_name(fn, self):
  """Hook into the Package.get_name() method."""
  return 'My Package'

Obviously not the most useful hook in the world, we replace the package name that would have been retrieved with a static string. In fact, the original Package.get_name() call is never actually invoked. It is replaced entirely.

Hooks can also be modeled as event subscriptions, where in the system, each method invocation can be considered a published event. Each defined hook can be considered an event subscription. For example, lets modify the previous example to simulate a pre and post event situation.

#ECP hook pre/post event demonstration 

import logging
from enomalism2.model import Package, hook

@hook(Package, Package.get_name)
def hook_get_name(fn, self):
   """Hook into the Package.get_name() method."""
   logging.info("Attempting to get the package name...")
   result=fn(self)
   logging.info("Success.")
   return result

Here, our pre-event functionality logs the fact that we are attempting to retrieve the package name. Our post-event functionality logs the fact that the package name retrieval was successful. To contract the first example, the original method is invoked here. We are simply extending the functionality. What is interesting is the fact that we can do so in any direction. We can add pre-invocation functionality, post-invocation functionality; we can replace the invocation entirely if need be.

ECP e2_exception extension module.

The e2_exception extension module for ECP allows administrators to view exceptions that have been raised by ECP. By default, exceptions are stored in the database. This means that they can be viewed again at a later time. The e2_exception extension module, when installed, provides a table view of exceptions that have been stored in ECP.



You can also view the tracebacks for each individual exception.



The e2_exception extension module also adds exception resources to the ECP RESTful API. Clients can query for stored exceptions, and delete them. Future versions of the extension module will add more features to the API as well as visual enhancements to the GUI.

ECP 2.2 coming soon

ECP 2.2 has reached the final testing phase and the ECP development team is working hard to make this release a reality. It should be available early next week but I'll continue to post any release schedule changes.