There's A Web Page For that

Yep, there's a web page for that, a ton of them. The one thing you need, a currency calculator, a CSS generator, there are probably five different web pages fit for the job. How useful are these web pages on mobile devices? It seems that applications native to the mobile hardware replace the browser entirely. These apps don't work everywhere - you don't see the same code running on iPad, Android, and BlackBerry. Each application that runs on a phone, if you can still get away with calling it a phone, is tailored to the hardware. Is this really a big deal to the user? Is the device-specific experience really that much better than something running in the browser? Currently, yes. Web stuff for mobile doesn't quite make the grade comparatively. No, there isn't always a mobile web page for that. But there will be.

What are web pages for? Information, of course. Well, I'd say the vast majority of web pages are about sending a message to an audience. However, plenty of web sites are currently transforming their content - or at least thinking about it - into something that's more like the desktop application experience. This is what users want, and so us software creators must have a pretty good reason to not deliver.

Why the change? Our websites never used to have trouble engaging with users. Why all the fuss over user admonition? The short answer is because we can. If a company selling widgets has an opportunity to hear the assessments of current and prospective customers, they'll take it. In fact, they're going to encourage users to submit any feedback. There is no such thing as a customer giving bad information to a company about their product. I mean, yes, there will be the odd loudmouth who does nothing but criticize, but this is nothing new. We've got to deal with those types even if we don't operate on the web.

We've got a bidirectional channel for businesses – much more effective than television. Rather than pushing out information, hoping people are watching, it's always available your clients seek it. If you want immediate feedback, provide your consumers with the appropriate tools.

Applications that don't utilize the web as a data repository aren't modern in any sense. We need applications that access the web, otherwise, users can't provide their feedback. We'd be crazy not to, right? If an online store, selling say, music, isn't hearing about each customer's unique experience, why might that be? Maybe they simply don't feel enticed to respond, probably because it takes more effort than it's worth. The question is, how to we make these feedback tools seamless to the point of being irresistible to the viewer? This is interesting to think about, especially in a mobile context where things suddenly aren't so straightforward.

Web pages aren't universal. Not anymore. We're really good at building just about any kind of application for the web, just as long as you build it for desktop and laptop browsers. Even some tablets do a decent enough job of displaying your retro backwardness that is 1.0. The rest of your users don't have the luxury of a larger display. To them, viewing pages on your site is like trying to plot a route on a map while looking through a magnifying glass pressed against it. It's a little unpleasant.

How do these web pages become a little more tolerable for the mobile audience? After all, we want to capture everything they're willing to tell us – we've got to make this as simple as possible. The answer in the mobile domain is apps. Users love their devices, and filling them with software amends their utility. What good is a smart phone without hundreds of applications installed? It isn't actually easier for users to interact with your mobile application, relative to your mobile web page that does the same thing. The application, once installed, feels like part of the device. That might seem too obvious, all software controls the hardware. This is no different from software installed on our desktop computers – new software extends the capabilities of the hardware. And since its installed locally, we've just expanded the capability of our hardware. There is something about the intuitiveness of desktop software. This is where mobile applications have an edge – they truly become one with the device for which they're built.

Why can't web applications do the same? Why can't they mesh well with the hardware they're delivered to? They can, and do. The trouble is that there simply isn't enough of them. There is a strong argument for developers to build web user interfaces for mobile users. Its the same argument for building web applications for desktop users – supporting several platforms is a major headache. Supporting multiple desktop platforms and multiple mobile platforms is simply out of the question for most folks. We've got to overcome the obstacle of unfamiliarity. Users want to install things, they want a level of assurance, knowing the tool will reliably get the job done.

The question is, can we give users this same level of comfort, the intuitive feeling that the web user interface is somehow part of the device. I think so. As long as they're able to launch it as they would any other native application. A little icon, part of a start-up menu, whatever the environment dictates, it just needs to mesh well.

Over time, I'd say over the next few years, we'll see web user interfaces become the standard. Apps won't go away entirely, there will always be a need for something specialized that no browser can support. However, the roles will shift – the browser will become the default user interface environment, desktop software the fallback. So yes, it's important that users feel comfortable using their software no matter the device. Software that's intuitive always comes out on top. As little as five years ago, web applications for the desktop weren't anywhere close to native desktop applications. All that's changed now – there is still a need for specialized desktop applications, but browser-based user interfaces are the first choice for a lot of developers now. I think this same transformation will happen with the mobile market sooner rather than later.

Monitoring CPU Events With Python

I decided to carry out a little experiment in Python - watching for CPU events.  By CPU event, I'm referring to a change in utilization my application might find interesting - large spikes for example.  To do this, we need to monitor the CPU usage.  However, my earlier approach at doing this wasn't exactly optimal.  For one thing, it only works on Linux.  This excellent explanation pointed me in a new direction.  So I've adapted it for my purposes of monitoring CPU events.

If I'm able to monitor CPU usage, I'll need something that'll run periodically in the background, checking for changes.  My basic need is this - I have a main application class I want notified when the CPU load meets a given threshold.  This should be relatively straightforward, especially since we've got a handy times() function that'll give us everything we need.  Here is an example of what I came up with.

from threading import Thread
from time import sleep, time
from os import times

class CPUMonitor(Thread):
    
    def __init__(self, frequency=1, threshold=10):
        super(CPUMonitor, self).__init__()
        self.daemon = True
        self.frequency = frequency
        self.threshold = threshold
        self.used, self.elapsed = self.cputime
        self.cache = 0.0
        self.start()
        
    def __repr__(self):
        return '%.2f %%'%self.utilization
                   
    def run(self):
        while True:
            self.events()
            sleep(self.frequency)
            
    def events(self):
        if self.utilization >= self.threshold:
            self.jump()
            
    def jump(self):
        pass
            
    @property
    def cputime(self):
        cputime = times()
        return sum(cputime[0:4]), cputime[4]
        
    @property
    def utilization(self):
        used, elapsed = self.cputime
        try:
            result = (used-self.used) / (elapsed-self.elapsed) * 100
        except ZeroDivisionError:
            result = self.cache
        self.used = used
        self.elapsed = elapsed
        self.cache = result
        return result
        
class App(CPUMonitor):
    def __init__(self):
        super(App, self).__init__()
        self.power = 1000
        
        while True:
            try:
                print 'APP: Computing with %s...' % self.power
                10**self.power
                sleep(0.1)
                self.power += 1000
            except KeyboardInterrupt:
                break
                        
    def jump(self):
        print 'CPU: Jumped - %s' % self
        self.power = 1000
        
if __name__ == '__main__':
    
    app = App()
    

The basic idea is this - when the CPU utilization reaches 10%, my application is notified, and can adjust accordingly.  The CPUMonitor class is meant to extend any application class I might come up with.

CPUMonitor is a thread that runs in the background.  By default, it checks for CPU load changes every second.  If the threshold is matched, the application is notified by calling jump().  Obviously the application needs to provide a jump() implementation. 

In my very simple example scenario, App extends CPUMonitor.  So when the App class is first instantiated, the CPU monitor runs behind the scenes.  Jump is only called if the resources are being over-utilized.  The great thing about this is that I decide what over utilization is.  Maybe 25% is perfectly acceptable to the operating system, but maybe my application doesn't think so.  This value, along with the polling frequency can be altered on the fly.

Try giving this a go, it shouldn't get past 12% utilization or so.  You could also play around with the frequency and threshold settings.  I've only implemented one event.  It wouldn't be too difficult to extend this to, say, trigger a changed by X event.

Using jQuery UI Position With Mouse Events

I've been playing around with the position utility for jQuery UI. Its very handy giving page elements a fixed position, especially if you're trying to assemble visual elements of a widget. An alternative method to tell your widget where individual elements should be positioned is by using the CSS you create for your widget. In fact, this is the preferred method as it make for easy editing in themes. Theme authors can't aren't expected to edit Javascript files and change position() call values. There are certain uses for the position utility inside widgets when we know the theme isn't necessary going to change the alignment. If we can use position(), its syntax is intuitive enough that it makes for easy programming. Easy and simple, I like it. Sometimes combining the position utility with mouse events can give unexpected results. I'll give you an idea here of what I encountered and how I was able to get around it.

When do we want to use position()? When we have a subject element we want to position against a target element. For example, I have a paragraph of text, and a sidebar div. I want the div positioned to the left of the paragraph. Simple enough, I do sidebar.position({my: 'right', at:'left', of: paragraph}). Which means “take my right side and move it to the left this paragraph”. Instead of specifying pixel or percentage values, we're able to give the utility simple positions. More importantly, we can read and make sense of them. Sometimes the maze of floats and margins are difficult to grasp.

Let's get a little more elaborate with the position utility by creating a simple image widget. The widget will display a control to manipulate the image when the user hovers over it. We'll use the position utility to place the control. Here is the HTML markup:

<html>

 <head>
 
        <title>Using Position With Mouse Events</title>
        
        <link type="text/css" href="jqueryuitheme.css" rel="stylesheet"/>
        <script type="text/javascript" src="jqueryui.min.js"></script>
        <script type="text/javascript" src="jqueryui.min.js"></script>
        <script type="text/javascript" src="progresseffects.js"></script>
        <script type="text/javascript">
  
        $(document).ready(function(){
            $('img').image();
        });
  
        </script>
 
    </head>
 
    <body style="font-size: 10px;">
 
        <img src="myimg.png"/>
 
    </body>

</html>

And here is our widget that displays the control when the user hovers over the image:

$.widget('ui.image', {
    
    _init: function() {
        
        var image = this.element.addClass('ui-image'),
            control = $('<button></button>');
            
        control.text('Hide')
               .addClass('ui-image-button')
               .button()
               .insertAfter(image)
               .position({my:'center', at:'center', of:image})
               .hide();
               
        control.click(function(event) {
            $(event.currentTarget).fadeOut();
            $(event.currentTarget).siblings('img.ui-image').fadeOut();
        });
               
        image.mouseenter(function(event) {
            control.stop(true, true).fadeIn();
        });
        
        image.mouseleave(function(event) {
            control.stop(true, true).fadeOut();
        });

    }
 
});

That's all there is to it – you can open the example and try moving your mouse pointer over the image. You'll see the hide button in the middle of the image. We used the position utility to place the control in the center. Since we're dealing with an image element, we can't exactly insert the button as a nested image element, so it gets inserted after the image. This makes moving the button to the middle of our image painful if not for the position utility.

The hide button, when clicked, will fade the image and the control out of view. I know, not exactly practical, but I want to keep the irrelevant stuff miniscule. If you try clicking the hide button, you'll notice the problem with our widget. If you move the mouse pointer over the button, it fades back out again. Why does this happen? Moving the mouse pointer over our image control shouldn't cause this behaviour.

It turns out, that the mouseleave event handler we've defined for the image is triggered when hovering over the control. Remember, the control isn't a nested element of the image, so triggering the mouseleave event is actually correct behaviour. Is there any way we can salvage our implementation?

The way I got around this was by using the relatedTarget event property. This will tell us why the mouseleave event was triggered to begin with – the new element being entered. With our widget, we can use this property to cancel the mouseleave event handler if the mouse is entering the control.

$.widget('ui.image', {
    
    _init: function() {
        
        var image = this.element.addClass('ui-image'),
            control = $('<button></button>');
            
        control.text('Hide')
               .addClass('ui-image-button')
               .button()
               .insertAfter(image)
               .position({my:'center', at:'center', of:image})
               .hide();
               
        control.click(function(event) {
            $(event.currentTarget).fadeOut();
            $(event.currentTarget).siblings('img.ui-image').fadeOut();
        });
               
        image.mouseenter(function(event) {
            control.stop(true, true).fadeIn();
        });
        
        image.mouseleave(function(event) {
            if(!$(event.relatedTarget).is('button.ui-image-button')){
                control.stop(true, true).fadeOut();
            }
        });

    }
 
});

Server Side Embed

In this day and age, our web experience wouldn't be nearly as interactive without the objects embedded inside most pages. What is the purpose of embedding objects inside a web page? Shouldn't HTML take care of this for us? It should, especially now that HTML5 is ever more mainstream with each passing day. Flash, on the other hand, is the standard for video, or anything fancy user interactions - transitions and such. Are embeddable objects for the web, good or bad? They're so pervasive that we can't simply ignore them, even if we don't like them. From a design perspective, embedded objects seem like a good way to reduce coupling by offloading some resources such as video. Its easy to use resources from other services, like YouTube. With social networks abound, embedding objects is an important concept for building web pages in a social context. How can our stuff on the server benefit from the embed tactic?
 

Let's take a closer look at why we embed things in our web pages – videos, menus, widgets and the like. Videos are obvious – websites want to show videos and we can't do this with HTML alone. HTML5 has the video element, but even then, I think we're still embedding conceptually because the video data isn't markup language. Likewise, an iframe might be considered an embedded object – even though its data is HTML, we still need to fetch it from an external resource, one outside of the page. Social networking widgets, too, are loaded from outside the page. I consider stuff that isn't directly contained in the page an embedded object. This works for me logically, because I can classify everything else as external, even though it may reside on my own server. Everything else, the paragraphs, headings, and so on, are a tightly-integrated unit that depends on external data.
 

What does embedding objects in HTML pages have to do with low-coupling? Well, if we can think of our web pages as a skeleton structure with slots for objects, we can reuse a lot of our HTML, consequently, CSS too. These object placeholders can put whatever we need in them – flash, HTML, our own widgets. That's really all we really need to worry about with this design, fill the hole with a web object. This is different from traditional programming languages where we're filling in the blanks using objects and polymorphism. Here, you've got to make sure that each object has the interface the slot is expecting. Not so much with HTML. You can embed whatever you want, the worst outcome being a shoddy looking site. So can we take these ideas, these embeddable objects and apply them to our web applications that live on the server?
 

As an idea, of course we can. In fact we already do this, we just use different terminology. On the server, we've got components. This is how the functional aspects of our application interchange. Replaceable authentication components, graphics libraries, you name it. There is something that can step in to do the job of a misbehaving or under-performing server object fairly trivially. I say fairly trivially because it isn't always so easy. The server-side is very different from the browser, worlds apart.
 

Lets say we're designing a simple brochure site. It doesn't rely on the server in any way other than to deliver web pages. So the big requirement is that it displays some information, maybe has a mouse-over effect here and there. A project like this only requires HTML, CSS, and Javascript. We need a web server, but don't have to do anything with it aside from starting it – we don't have to program it. Our little web site, in the browser, has three things, three languages, to worry about. This can be handled easily by one person on a small scale. Even when it comes to replacing the embedded objects, there is no learning curve because there is nothing to objects in HTML. Usually just a URL. Or maybe some cut-and-paste object code with a parameter. Easily manageable.
 

The server part of an application is a different story because there is just so much involved with serving HTTP requests. We have to make sure the user is authenticated, make find the appropriate request handler, process the request in a timely manor, and perform database queries. All of this while making sure we're fault-tolerant because unlike the browser, an HTTP request has the potential to break the entire application for every other user. This is why we have web application frameworks, to handle the common stuff with serving dynamic content. So how do we embed objects, objects that extend the capabilities of our application, into the framework?
 

Django immediately comes to mind for me, when thinking about embedding on the server. Partly because I really enjoy using it and partly because it has the best example I can think of. I'm sure there are plenty others too. In Django, we've got middleware and applications, both ready to embed in our projects. Middleware is how we inject logic into each HTTP request served. Sorry, embed logic. Middleware inspects the request before it is handled and after the response has been generated by the handler. This relates to HTML embedding in that we're able to take third-party objects and use them for our purposes, just as we would copy and paste a YouTube video.

Embedding things into larger applications, be it an HTML user interface, or something on the server, is a powerful concept. Embedded objects are simply part of the larger component architecture, where stuff can be swapped out. I think the embed concept for severs has a close parallel with the kind of embedding we do in the browser. Its a powerful idea, it helps with designing loosely-coupled systems for the web. It helps with delegating the responsibilities of a feature to the appropriate party. And that's it, really. Its just a mode of thought when putting together an application.

Learning Computer Science

There is no doubt that in today's computer-based, technology-driven, network-connected, information-intensive environments, that computer science is an ever more pertinent discipline. Not just computer programming, not just software development for everyday components, but the underlying fundamentals that generates new research and insights. The challenges information technology presents are only going to get harder as our data grows larger. We need more computer scientists. Instead of seeing a growing trend in the computer science student population, it has fluctuated wildly over the past decade – enrolment jumps, enrolment dips. There is no single factor that explains rising or declining interest in computer science. I'm sure of the reasons are directly related to the the teaching institution, not the discipline itself. I wonder, is a degree the only way to master the most challenging theoretical and applied aspects of computer science?

I don't have a degree. This fact means different things to different people. Not having a top-notch education obviously doesn't help in finding employment. However, I also didn't want to spend half a decade learning without doing. This isn't to say that a degree is a bad thing. To the contrary. I've worked with several people in my life with computer science and math degrees who've changed my understanding of what I do every day. However, I was more motivated to join the workforce sooner rather than later. Accumulating student loan debt wasn't on my to-do list either, so I finished a 2 year program and started working.

Finding a job in information technology isn't easy – there are plenty more reasons not to hire someone than to hire them. The biggest roadblock for finding employment is lack of experience – I had zero at once point. Not only did I know nothing about working in the real world, I had a miniscule diploma while seeking out jobs that require a degree and experience. It felt like I had no chance of getting hired and would have to return to school. This went on for quite a while – over a year. That's a long time to list professional job hunter on your resume.

Things got interesting when I found Enomaly. Not only it seem like an awesome start-up to work for, but they did open-source consulting gigs! The main reason I got hired was because I started and worked on an open source project at the time – a little tool that parsed XMI files and ran activity UML activity diagrams. It wasn't the project that made an impression – it was the idea of taking on an open source project they valued. This got me thinking, and to this day I still think about it, maybe open source software development can serve as an education in computer science.

The same stuff you'd find in a computer science course of any kind, you'll find in an open source software project out there. Its all in the code. This is how you learn how to write your own code by studying someone else's. The lessons learned accumulate as you read more code and write more of your own.

However, if you're young and just getting started, this task isn't so straightforward – you need a starting point. How do I know which open source project to study? Well, maybe it is straightforward after all – find an open source project you enjoy using yourself. Don't read the source code of a image editing program if you can't see yourself using it. Reading code for applications you have no interest in won't help you learn the science behind it all. Instead, its more of a chore. And this is one area where an open source computer science education really shines – the sheer variety of what's out there to learn about.

Once you've found an application you enjoy using and find interesting, you're ready to start reading code. I find the best way to start is not by jumping down to the low-level stuff and working my way up. Instead, I like to find something specific the user can do with the application – import a foreign file format say – and drill down in the code structure using a top-down approach. There is plenty to learn about on the way down.

The best part about learning computer science by studying open source projects is that it often spawns a series of learning projects. If you learn all about Django, maybe you'll want learn about other Python web frameworks and how they tackle similar problems. Maybe you'll investigate further on how sockets work or buy a book on network theory. Maybe you'll start reading the Python source code and learning about programming languages. The possibilities are endless. As you learn the fundamentals, you'll hone in on the specific ideas of what you're really interested in.

Open source software as a learning tool is great because its open. There are no barriers hiding the conceptual bits and pieces for those interested in how they work. So who are your instructors in an open source software education? Your peers are. The same people who took an active interest in the project, spent the necessary time to understand and contribute to it. Contributions come in many forms – code, monetary donations, and knowledge itself. The aspiration of any open source project community is that it will grow, steadily and for the long run. The only way to make this happen is by educating those who are willing to learn about it.

If you're already enrolled in a computer science program, use an open source project as an opportunity to expand what you already know or as a way to devise the right questions. You might find something you wouldn't have otherwise.

jQuery UI Progressbar Effects

The jQuery UI progressbar is a nice way to show the completion of a task.  With each step, the progressbar is updated and subsequently, the user receives visual feedback.  The default implementation of the progressbar simply increases the width of the completed area with each updated.  So, for example, if the completion is at 30% and we update with another 10%, the width will be set to 40%.  It would be nice if we could make each progression a little fancier, perhaps with a little animation?

Thankfully, jQuery UI gives us the foundation with which we can extend the default behavior of any widget, including the progressbar.  The jQuery core gives us the tools necessary to do simple yet elegant CSS animations.  Here is what I came up with - first the HTML:

<html>
    <head>
        
        <title>jQuery UI Progressbar Effects</title>
        
        <link type="text/css" href="jqueryuitheme.css" rel="stylesheet"/>
        <script type="text/javascript" src="jqueryui.min.js"></script>
        <script type="text/javascript" src="jqueryui.min.js"></script>
        <script type="text/javascript" src="progresseffects.js"></script>
        
    </head>
    <body>
        
        <div id="progressbar"></div>
                    
    </body>
</html>

And here is what progresseffects.js looks like:

//Create the new progeffects widget by extending  the progressbar.
$.widget('ui.progeffects', $.extend({}, $.ui.progressbar.prototype, {
 
    //We need to redefine the progressbar options and add a duration.
    options: {
        value: 0,
        max: 100,
        duration: 250
    },
    
    //This is used to keep track of the refresh frequency.
    refreshed: 0,
      
    //Initialize our widget by initializing the base progressbar.
    _init: function() {
        $.ui.progressbar.prototype._init.call(this);
    },
    
    //Redefined in our widget to provide animations.
    _refreshValue: function() {
        
        //Some variables we need.
        var value = this.value();
        var percentage = this._percentage();
        var time = new Date().getTime();
        var duration = this.options.duration;
        var max = this.options.max;
        
        //This is part of the default implementation, but still required.
        if ( this.oldValue !== value ) {
            this.oldValue = value;
            this._trigger('change');
        }
        
        //Here, we're making sure the progressbar isn't refreshed more
        //often than the animation duration will allow.
        if (time - this.refreshed < duration && value < max) {
            return;
        }
        
        //Store the time for the next refresh.
        this.refreshed = time;

        //Perform the refresh, using an animation.
        this.valueDiv
            .toggle( value > this.min )
            .toggleClass('ui-corner-right', value === this.options.max)
            .stop()
            .animate({width:percentage.toFixed(0)+'%'}, duration);
            
        this.element.attr('aria-valuenow', value);
    }
        
}));

//Example usage.
$(document).ready(function(){
    
    //Periodic update function to set the value of the progressbar.
    var update = function(){
        var value = $('#progressbar').progeffects('value');
        var max = $('#progressbar').progeffects('option', 'max');
        if (value < max) {
            $('#progressbar').progeffects('value', value + 1);
            setTimeout(update, 100)
        }
    };
    
    //Create the widget and start updating the progress.
    $('#progressbar').progeffects();
    update();
    
});

This example will continually update the progressbar value until it reaches 100.  It works just like a regular progressbar widget with one minor difference - it accepts a duration parameter, used for the animation each time the value changes.  Here is how it works.

First, we create a progeffects widget that extends the progressbar widget.  We're only changing a couple things about it so we should reuse as much as possible.  When a jQuery UI core widget is extended, the options need to be redefined by the custom widget, otherwise, they won't be recognized.  We're also adding a new duration option that'll be used with the animation effect.

The bulk of the new progeffects widget is housed in _refreshValue().  This method is used by the original implementation to update the progress.  We use some of the original implementation, but we've modified it to use animate() to set the CSS width instead of using CSS.

The example code that uses our new widget value 10 times per second.  You may notice a problem here.  By default, the animation duration of progeffects is 250.  Yikes.  This means the progress bar will still be updating long after it has reached 100!  There are two ways to get around this problem.

The first way - pass progeffects a duration value reflective of the update frequency.  So for example, we could have passed it {duration:100}.  This approach, however, introduces a couple new problems.  First of all, how do you know ahead of time how fast the progress will be updated?  If you did, it would be nothing better than our static example.  Second, try actually passing 100 as the duration value - it can be choppy to the point that it negates all we've done.  I'd say this approach is out.

The solution I'm using in progeffects is to monitor the update frequency.  Before the animation effect is executed, it will check if enough time has elapsed since the last update to be meaningful and to look good.

There are several variations to this, I'm sure.  Its a minor enhancement, but it sure does look cool.

Features Don't Freeze Themselves

How do we know when its time to stop adding new features to our software? When do we take off our creative hats and put on our cautious stability testing hats? Once we meet the functional requirements of the project, that's when. The term used to describe this dividing line between the chaos of development and the order of a finished product is feature freeze. Yep, features are literally frozen in time. They don't change or evolve from this point forward, until the release is out. After releasing, we've got a clean slate, more or less. Its time to start adding new features, and expanding on those previously released. So my question is this: is the feature freeze phase really that straightforward to define? By this I mean, can way really say with confidence, based on the project requirements, that we'll know ahead of time when we're going to stop making new code? I would argue not, as would most developers because impudent predictions are discouraged in software development. This problem is more pronounced when we're building things that have ambiguous requirements, things like frameworks.

The rationale behind feature freezes is self-explanatory for the most part. There has to be a point during development were we devote our efforts to quality assurance. Of course, we do this even during development too – modern approaches to software development involving test harnesses for each feature. Even this is limited, speaking to the system as a whole, how it will be used in the wild by real users? The incremental develop and test approach cannot cover all cases. So we have to stop development, and try our best to destroy our code, automating as much of the process as possible. We pick and prod and poke, systematically dismantling what we've built until we can't find any weak points. This is the best way to build reliable software. This is also the ideal scenario - not exactly reflective of what we face in reality.

Think about software requirements for a client project. When have you ever built software based on a single set of unchanging requirements? That's what I thought, never. Requirements change and so does everything else. By everything else, I mean the driving force behind the requirements. The client wants a new widget because it'll make a new department at their organization function more efficiently. They want a new social widget because it is the brand new thing the simply cannot live without. They want the colours slightly different, due to nothing more than their mood that day. Who knows why? The important thing to remember is that initial requirements are nothing more than a general prototype of what the finished product might look like. Requests for change usually don't arrive until well after it would have been convenient for us to implement.

You'd think this whole implement the requirements thing would be a little easier when building a product that solves a more general problem. For example, shopping cart software, income tax reporting software, image editing software. The list goes on – things that are general enough that more than a small group of people might find it useful. Maybe a better example is the Linux kernel. You'd think that solving a general problem would make the requirements a little more concise. The opposite is when building stuff based on a contract for a company. Usually a web application of one flavour or another. These have more niche things that need building. This is why you were hired to build them in the first place – to execute a precise vision. Consumer software on the other hand, usually doesn't cater to the specific problems of any one organization. And this is the challenging part when it comes to devising requirements for such a thing.

There is no such thing as a universal software product that will solve every problem for every person on the planet. Not even for a specific problem domain is this true. There will always be a deficiency, a missing feature, or maybe the user interface is just not that friendly. This is why alternatives exist for every kind of consumer software system. Linux isn't for everyone, nor Windows, nor Mac. Users have their own reasons for choosing one over the other. Different software that strives to solve the same problem excels in different aspects. Some operating systems are more user-friendly than others, and some are more secure, and some are used for the sake of familiarity, why change? Again, no software caters to everyone. That doesn't always stop us from trying though.

Programmers are perfectionists. Not only does the code need to look beautiful, it must work as expected, with no bugs. More than that, we also care about the usefulness of our software, more than some might think. Nobody wants to write code that doesn't help somebody solve a problem. This is considered a failure, and this is never fun, we've all been there before. Successful projects have at least a handful, maybe a couple hundred people using it on a daily process. Ideally, users recommend it and praise its usefulness and how they simply could not live without it. I know I have a list of a dozen or so tools I'd be lost without. This is always great to hear. However, we also hear about those who struggle with certain aspects of the system or outright stopped using it for some reason or another. Hearing these things about code you've sweated over are hard to ignore. Maybe the last feature freeze wasn't such a good idea after all, I mean, there are people out there disappointed with our product!

And so it begins, we've now got to start the task of expanding our system's capabilities to accommodate. Built to please. This is the cycle, add new features, expand existing features. Accommodation. The problem is, the cycle never ends, and what you're left with is usually something bloated that looks nothing like the initial vision held so long ago. How do we break the cycle?

Loose-coupling is part of the solution to the feature bloating problem. Large systems that have grown due to the sheer number of features it provides aren't necessarily loosely-coupled. Can you download and install a smaller version of the software? For example, some applications let you configure the pieces you'll need before installing and using it. This isn't possible if your system's components are tightly-coupled. However, even this approach might not suit all users because they may need these negated components down the road. Can they add the missing pieces? Or do they need to repeat the entire download and install process? I think the only real solution is to develop independently installable components.

If your software can be broken down into individual components, there aren't so many dependencies that force features on users that don't need them. Building independently installable components, and designing loosely-coupled code, however, are two very different things. Loosely-coupled code is design discipline. Independently installable components is a deployment discipline. There is some degree of overlap though, because your design needs to support variable dependencies. For example, the application still needs to work if there is no sorting component found – it just wont sort stuff. As always, there needs to be some sort of core foundation, the essential code necessary to bootstrap the application. This is where the logic to determine what features are available live.

It might seem like overkill to divide your software into tiny installable components, one for every feature of the system. It might seem that way, even scary at times, what happens if the component installation fails for some reason? This isn't any different than installing the whole package as one big opaque box – buggy code is buggy code. Its better to think from the users perspective – maybe the majority of features we've implemented in the system are overkill to them. If they don't want them, they don't need them. So my advice is this – implement feature components as much as possible. Implement 400 of them if you need to, just don't ship them as one unit that cannot be decomposed.