Thursday, October 27, 2011

Software Luxury

Many things in this world can be considered luxurious — cars, homes, hotels, and yes, software too.  Attributes that establish an object as luxury include things like the material, the cost, and the brand.  It's difficult, sometimes, to state why exactly something is luxurious — at least in terms of visible properties.  We say things like, "it's a Rolex" or "it's a Mercedes" — the brand in it's own is luxurious.

Products that don't have the recognizable brand name to back them might be of outstanding craftsmanship and quality.  Yet, these things aren't considered luxurious items.  Perhaps they're used in industry to perform a specific task — instead of tooling around on a winding round or displayed as a decorative ornament.  Aside from the brand name, how can we identify luxurious things?  This is particularly interesting when it comes to software because we think of software as a tool more than a sumptuous want.

Tools and toys
To secure more insight into what makes something a luxury and what makes something a necessity, it's helpful to classify things more broadly.  A toy, be it a boat, a ski-doo, or a train set, isn't a requirement for day-to-day living.  They're a form of entertainment — something to keep us occupied when we're not busy other duties.  Yet, not all toys are luxurious.  When you're busy playing with rudimentary item, the desire to progress upward is unavoidable.

Tools are necessary for any trade.  They're how we're able to succeed at any task.  Tools get the job done, so even if we're not doing something enjoyable, tools make the job a little more bearable.  Tools are the screwdrivers, the text editors, the trucks — the things we need when we're not busy entertaining ourselves.  Once again, there is always room for improvement, a way to execute the task more efficiently or more comfortably.  If only there were a more luxurious shovel.

These broad categories — tools and toys — occupy the vast majority of our wakeful existence.  And they don't exist irrespective of one another either — they both share the same drive to improve, to become even more joyful, or perhaps less painful.  And that's just how the tools and toys of our lives evolve — through the constant pressure to become more integrated with what we're accustomed to.  Or, perhaps we're not accustomed to something luxurious, but that doesn't stop us from observing what luxurious items do, the joy they bring, and try to mimic that in the next generation of lower-end products.

Software necessity
Software necessity is hard to define — at least in a broad sense.  If you're performing an accounting task, you're probably going to need a spreadsheet.  If your doing development, you're going to need an editor.  These things are absolutely necessary, but they don't address the more granular idiosyncrasies associated with them.  For example, most spreadsheet applications have the required functionality built-in — the formulas, the importing capabilities, and so forth.  But what about defining your own formulas — what if your organization has a standardized set of formulas to plug into the data?  This is a specific requirement the application must support.  A necessity.

Software tends to carry a more stringent definition for everyday work necessities.  It clearly isn't enough to say I need a text editor to do development work.  You're probably used to features of a specific editor or IDE.  So this doesn't mean you can't do your work, it just means you can't do it effectively without contrived workarounds.

The necessity of software goes beyond simple features — checklists that determine the usefulness, the productivity one can expect from a given software package.  There is the matter of where software runs, what protocols can it support — things of that nature.  If an organization is determined to stick with legacy software that isn't web-friendly, they risk being left behind.  Staying current with the times is a necessity not just because it's allows new innovations, but because the competition will no doubt be on top of it.

There are two dimensions of software necessity — the bare minimum fundamental stuff that is requirement to do the job.  Then there is the the more specific aspects that allow you to do the job better.  These latter additions to your systems, small incremental changes and adaptations that transform a monotonous job involving a dozen steps and hide that complexity behind an intuitive interface — these are luxurious relative to the first iteration of the product.

Developing luxury
All software, if it is to survive, must be continuously developed, continuously improved and fleshed-out.  Are we developing luxurious software by making these improvements?  Or are we simply raising the bar of necessity?  Because, as time goes by, what was once thought of as a luxury, turns into a diminutive, core part.

If luxury is a definitive property of something whereby it goes above and beyond what the thing was designed to do, maybe luxury has no place in software development.  Remember, software is about simplicity — without simple solutions, we're stuck in an even repeating loop of frustrating experiences where we're trying to do something that goes beyond project scope.  Superfluous features, and conversely code, should be pruned.

But who's to say that creating luxurious software can't be reduced to simple improvements that aren't intrusive to the user or the developer?  We might even go so far as to say that luxury in software is created by removing things — by taking complexity and replacing it with simplicity.  Maybe instead of focusing on feature checklists, software projects should think about how the user will perceive it in terms of necessity.  A reductionist approach, removing superfluous constructs from the system, yielding something small in size and high in value.  The must luxurious software is that the fulfills the necessities, unlike other items in the world where luxury is the nice-to-haves.

Tuesday, October 18, 2011

Cloud Goods And Services

What exactly does the cloud offer to consumers?  Is it goods?  Is it services?  Is it some combination of the two?  The cloud is a unique market — we're seeing compute resources offered as goods but, by their very nature, they're also a service.  Does there exist a means to cleanly separate the two?

Maybe there is no method by which the goods are divorced from the underlying service.  From a marketplace perspective, this introduces a problem for cloud consumers — paying for goods that require a service to make them worthwhile.

External cloud services
Cloud services aren't necessarily tied to one particular cloud provider — they can be cloud provider agnostic.  This is called an external cloud service because it doesn't explicitly require the unique features or capabilities of any one cloud provider.  It's external to both the consumer's organization and the provider's organization.  An external cloud service is really a consulting gig performed by experts.

To get your application off the ground, and into the cloud, you need a team of cloud experts.  The type of companies who employ a permanent cloud virtuoso unit probably haven't been around too long.  Starting a new company today means understanding that must useful technology you're going to want to use inhabits the cloud.  You might want to query cloud data, you might want to store your information there to make it globally available and redundant.  Regardless of what technology you're developing that makes your organization tick, new companies have the advantage of not being hampered by legacy technology that is difficult to migrate and otherwise adapt outside of where it was initially deployed.

Established companies are the inverse — using new cloud technology is difficult because of the investment in local computing power.  Making this stuff work in the cloud isn't easy — or perhaps not even necessary.  But even mature organizations with a rich technology history need to, at some point, develop new technologies that are going to appeal to newer audiences and newer devices and have a broader reach in general.  These companies didn't start off with cloud offerings at the forefront of their IT strategy.

The latter company type — the one that doesn't have cloud ingrained in their day-to-day development efforts are the one who benefit from external cloud services.  The market for such services is only made possible by companies that don't yet have a large stake in this new paradigm.

Migrating applications to the cloud requires knowledge of the cloud technology platform and knowledge of the application.  Without expertise in both, migrating to the cloud doesn't make sense because you're loosing efficiencies that you'd otherwise gain.  In fact, the exercise might even yield a loss if the required overhead in executing the migration runs for an extended period of time.  Additionally, the application that should be running, and the people responsible for it's continued operation can't focus their efforts if a migration is in progress. Seasoned experience is what we're looking for in migrating legacy applications to the cloud.

So what if you're able to take an existing application and successfully move it into the cloud.  Now that you're running on a cloud platform that allows you to scale out, allows you to pay for what you consume, allows you to integrate with new technologies seamlessly, do you still need these external cloud services?  You're in the cloud at this point — is there such a thing as internal cloud services, specific to the unique features your application now has at it's disposal?

Services and goods
You've muscled your way through the barriers of taking an application that's designed to run on legacy hardware.  Or, maybe less iniquitously, applications that weren't designed to take advantage of cloud features.  How do you take advantage of the services that your cloud provider has to offer?  Perhaps a good starting point is to separate the services you're paying for from the goods you're acquiring.

Services internal to the cloud provider are unique features they offer to customers.  As an example, a facility to manage and monitor your resources.  This might come in the form of a user interface, an API, or a combination of the two.  Regardless, this is a must-have feature of any cloud provider if they're going to operate your application.  The differentiation is the quality of the service — which provider has a better facility to maintain the ongoing functionality your your cloud efforts?

Even services that are exclusive to the given cloud provider are still external, to a degree, from the underlying goods.  The goods are the meat of the cloud — the computational resources that you'd otherwise have to purchase, store, and maintain yourself.  This is a whole other department of expertise that we're deferring to the cloud.  Sounds good doesn't it?  We've got our application running on someone else's hardware, they're providing us with the necessary services — their own monitoring and management tools — so that we're able to operate our cloud infrastructure.  And, we're able to consume computing resources as we need them.

So the separation of goods and services appears fairly clear-cut.  Goods — the cpu, the memory, the bandwidth, the storage.  Services — tools to manage and monitor the goods.  The checklist appears complete and we've now got an understanding of how we're saving money by operating in the cloud.  There is, however, one other consideration we should probably make note of — how are we ever going to survive if this provider were to disappear?  Or, maybe we're using a brand name provider that isn't likely to dissolve anytime soon.  But that introduces another potential issue for the cloud consumer — vendors that occupy a large portion of the market have more freedom to introduce changes.  These changes aren't exactly consumer friendly and they highlight how tightly coupled cloud goods and cloud services really are.

Goods are what matters
In the end, you're using cloud technology because it provides access to hardware that you wouldn't otherwise have.  You build your own applications that utilize this hardware, you build it in such a way that it'll scale out and take advantage of new hardware resources should they become necessary.  Everything else is secondary.  Like the services of monitoring and managing your cloud hardware resources — you don't need a fancy user interface, you don't need an API that'll outdo every competitor.  Just the essential services.

With essential services — methods to track what you're consuming and provisioning new resources — you're decoupling the services from the goods.  At the most fundamental level, controlling what hardware is available to your application and when, is what cloud computing is all about.  Everything else is superfluous — useful, maybe, but also limiting.  It might make more sense for organizations to focus in on what really matters instead of on the features the cloud provider offers to keep customers around.  Remember, the cloud is a cloud of clouds — a cloud of providers.

Even this practice — that of decoupling the service from the good is limited in cloud computing because the good being offered is still only virtual.  Underneath the virtual hardware you're purchasing is still an opaque service, responsible for carrying our provision requests.  Even though you can't see the mechanism responsible for generating the goods you need, it is helpful to understand that each provider implements these hidden services differently — some better than others.  So if you can manage to keep your cloud operations abstract enough that they focus on the goods, you're giving yourself enough freedom to explore the entire cloud — not just a subset.  Eventually, you're going to have your own technology, capable of grading goods for what they really are and what their real-wold implications are for your application.

Wednesday, October 12, 2011

Static Tests And Dynamic Tests

How do you automate the delivery of solid software?  Software without bugs, software that fulfills what the customer has envisioned?  It's difficult to determine how software is going to achieve it's role when dropped into problem domain battle field.  Adroit human beings — the software development team — collaborate and work closely with the customer to work all this out.

Typically, this process never ends — even long after the software has been deployed and has become a possession of the customer.  We'd like for our software to work well enough that it can handle subtle mistakes in configuration — in other words, the unexpected.  Most experienced computer users have had experiences where they know ahead of time the software is going to fail as a result of their actions.  The nice surprise these folks get is when the software unexpectedly adapts to the situation and might even offer the user some advice.

These advanced features — little nuggets of exceptionally usable software don't come naturally — not without a lot of testing.  Back to automating the development process — the repetitive pieces that are better performed by unit test bots.  We, as software developers, are constantly pushing the limits of what it means to be stable — how can we find our weak points and design a test that will cover them when we want to step through the testing procedure?

The question I have is this — can you devise creative automated unit tests without manually testing the software yourself?

Static software design
When I talk about static software design, what I'm actually referring to is not the design itself, but the methodology used to generate a design.  In the end, what we're looking for is something that satisfies the requirements of our customers.  The customers rarely glimpse into the process that produces their code.  So if a development team follows the waterfall approach — if we're to assume that the requirements are Newtonian and unchanging, that'd be fine by them.

As we all know, the waterfall approach has a pretense of completeness about it.  We're assuming, under this mode of operation, that nothing about the project will change.  I'm sure there have been a few instances of this in the real world — hello world programs, for example.

Static software design means that we're not incorporating the emergent system properties into the final product.  During development, we're bound to make discoveries about our approach, the problem domain, or even the technology itself.  We don't have robots that'll write code for us yet — this means that humans still play a pivotal role in the design of software.  And where there are humans, there are false assumptions.

Is this necessarily a bad thing?  No, because if there is something unique within our species, it's our ability to problem solve and adapt quickly.  There are simply too many variables in software design to fully automate the process — and even if we could take humans out of the driver seat when it comes to optimal programming language use, we've still got humans supplying the input.

The iterative approach to software development is really nothing more than a means to organize the natural learning that takes place during evolution of the system.  Each iteration has new input.  The first iteration has what the customer envisions — each subsequent iteration has our previous attempts at prototyping the system along with new problems we've introduced for ourselves.  This is as good as it gets — we not only improve the software system we're building, but as developers, we're building our own repertoire of software development knowledge.  You don't get this with a static software design methodology.

Dynamic software testing
Write the tests first, build software that pass those tests.  An elegant solution — perhaps an obvious solution to building quality software in as little time as possible.  It's like when you're figuring out a problem mentally — it helps if you have a set of smaller problems — questions you can explicitly ask yourself before answering.  The same goes with software — it's much easier to write code that'll pass small tests than it is to write software that'll answer some grand philosophical question.

This is indeed a great starting point — an atomic means to get your feet wet with each iteration of the product life cycle.  As a bonus, you've now got a suite of tests you can continue to apply throughout the development of the software.

But here's the thing — the tests you write for your software are like a little software project on their own. The tests are the nurturing parent that makes sure the child is able to make it in the real world without walking out onto the highway and getting run-over.  Thus, if our software is continuously evolving, than so must our unit tests.  As we write our tests, we're going to discover what works and what doesn't.  But as we learn and improve the software, so too must the tests be improved.

As the system grows, we'll need to introduce new tests that aren't necessarily related to the overall requirements of the system.  Perhaps there is a something a little less tangible — something related to a technological dependency that caused problems during development.  These are useful tests to pass and are impossible to collect up-front.

So if tests are also evolutionary, they should become an integral part of the software development process.  Incorporating automated unit tests into the project isn't anything new — what I'm suggesting is that they're treated as the parent of the software product and follow the same evolutionary course.  Just as static software development isn't a tool humans can readily use, neither are static up-front unit tests.  New unforeseen scenarios need tests.  These don't reveal themselves till several iterations into the project.  And, old tests should eventually be culled — much later on.  Mature software products should have sophisticated unit tests that supply unique input as a result of evolution.

Friday, October 7, 2011

Printing Dynamic Content With jQuery

Formatting content from the web for printing can be a challenge.  Some websites have the preconceived notion of a printer-friendly format for their content.  This is makeshift version of what the user normally sees reading directly from the browser.  Online versions of content — the default version — contain elements that serve no purpose in a printed document.  Things like embedded links, navigational items, and so forth.  Typically, sites that offer printable content present the user with a print version button, leading them to another page.  This version has a different set of styles applied to it — probably hiding things that shouldn't be included in the printed copy.

This approach works fine for content assembled on the server — the web application generates the HTML markup in it's final form before sending it to the browser.  Likewise, when the user hits the print version button, they're either going to see the same exact page with different styles applied to it, or be redirected to another URI — /page/print/ for instance.  It's difficult, however, to make use of this approach when we've got dynamic content being pushed from the server onto the page.  How can we format the page for printing in a universal way when the content doesn't have a strict URI associated with it?

Dynamic content and URIs
When we're talking about dynamic content in web applications, we're not talking about the application server that fills in the empty template slots.  True, this is dynamic, but not from the user's perspective.  In this sense, the user is associating the content with a single URI — a one-to-one mapping.  The URI is the reader's key to fetching a chunk of content.

For written content, news sites and bogs for example, this approach works well because you can share URIs.  For applications, this is a little trickier — especially for applications that follow more of an Ajaxy paradigm and aren't necessarily URI bound.  In these types of environments, the user is is more concerned with generated output — tables and other forms of information that convey the state of the application.

But the application needs to get this data from somewhere — the Javascript Ajax calls need to follow some path to the application that'll return something of value for the user interface to display.  This is usually an API that returns JSON data that the user interface library, jQuery for example, knows how to parse and understand.  The data these URIs return are perfect for user interfaces, unfitting for user looking to print that data.

Although some — perhaps most — of this application-style content isn't suitable for printing.  But some of it probably is.  A grid would be a good example of this.  Grids take on a specific look and feel inside the web application user interface.  Suppose they want to take a look at it on the train ride home in dead-tree format.  Is the grid widget optimized for printing?

Building new documents
We could, to help solve the problem of printing this grid widget, implement functionality that'll take any dynamic data rows, and the grid as a whole, and adjust it for printing.  That's fine for this particular use-case.  But it also makes two assumptions that are at best sub-optimal.  First, if our application uses a grid that is populated dynamically via an API, chances are we're going to have other printable content.  So this means any printing strategy we employ is going to need to support more than just the grid.  Anything less is prohibitive in value.

The second problem, perhaps not so obvious as we're used to printing static web content, is how does the user return to where they were in the application after printing?  We could always apply some style changes that'll make the content printable in the current browser window.  This introduces more complexity, however, because we've now got to roll back those changes once the user has printed their document.

Another approach, one that I think behaves more intuitively, is to have the user interface identify printable regions in the interface when they click the print button.  These regions are then copied over to a new document before being formatted to print.  This is like following a print friendly URI for static content — it doesn't change the original browser window.

Here is some basic markup.  Let's pretend that the content div has been inserted into the document as a result of an Ajax call...

<html>
    <body>
        <div class='content'>
            Some <a href="#">content</a> with <a href="#">links</a> inside.
        </div>
        <button class='print'>Print</button>
    </body>
</html>

 We've also got a print button in our user interface.  Let's implement some example printing functionality for Ajax content...

$(document).ready(function(){
    
    function replace(){$(this).after($(this).text()).remove()}
    
    $('button.print').live('click', function(event){
        var print_window = window.open(),
            print_document = $('div.content').clone();
        
        print_document.find('a')
                      .each(replace);
        
        print_window.document.open();
        print_window.document.write(print_document.html());
        print_window.document.close();
        print_window.print();
        print_window.close();
    });
    
});

And that's it.  This is a simple example of how we can locate and modify information inserted by Ajax API calls and create a new document suitable for printing.

All this example does is replace a elements with their text.  But before doing so, the div.content element is cloned — so as not to disrupt the user interface.  Finally, we can copy the new modified element into a new browser window for printing.  We're automatically going to print this for the user, and close the new browser window to make the whole experience as unobtrusive as possible.

Monday, October 3, 2011

Time To Refactor

Refactoring existing code makes better software.  Martin Fowler has championed this concept and as a result, developers look at refactoring code as an essential part of the process — not something to do on slow days.  Of course, idealizing over code design is one thing — making it a reality is another.  Unfortunately, refactoring doesn't happen for one reason or another — usually there isn't enough time.

But what about the programmers that have to stare this code in the face — day in and day out, watching unruliness assume control?  They're the ones who see the problematic consequences of letting ugly code grow and metastasize into other subsystems.  Programmers probably don't decide there is no time to for refactoring.

If there is no time to refactor code, when will it get done, if ever?  Understandably, features and enhancements are expected.  Expected yesterday.  Maybe there is a way for software developers to convince stakeholders that refactoring is the only option left.  Maybe we can treat, somehow, refactored code as a feature?

Code gains weight fast
Code gains weight at an alarming rate.  You might start off your hacking session with a small set of classes, maybe a couple utility functions.  It takes only a couple hours for this lightweight setup to transform itself into a bloated plate of spaghetti code.  Not because you don't know how to write elegant code, but because others are depending on you — it just needs to work.  If we wanted beautiful code the first time around, we're in the wrong business.  Refactoring is about improving existing code, not code that has yet to be written.


As programmers, we idealize solutions — before sitting down and typing it into the editor, we've at least, if only briefly, conceptualized a few alternative approaches to tackling the problem.  This isn't considered big, upfront design — it's more of a personal game plan to help spring us into action.  But, since this is only a quick mental note — not using real code and a real editor and not running real tests — chances are the some aspect of our plan is wrong.

The trouble is, we don't know that some aspect of our ideal implementation is invalid until it's concretely implemented and executed.  Sometimes our code doesn't even reach the point of execution before we've realized it doesn't fit.  Sometimes all it takes is a glance at the interfaces you're working with or the documentation of another component dependency to realize that there is a sudden change in plan.

This sudden realization means that the nice, natural flow that we had envisioned, the one that motivated us to get going in the first place, has been scrapped, ups the urgency of the situation.  Maybe the entire metal workflow isn't thrown out, but the calm confidence we had slowly dissipates.  We're probably not following a waterfall approach — so we need to adapt quickly — pound out something that'll work.  Otherwise, the forward motion of the entire project is jeopardized.

Fixes to sudden disruptions aren't difficult to implement, but they're not the most elegant or efficient solutions.  And so the code gains weight.  It works, but it's become heavier — in terms of raw size and in logical baggage — all within a few hours.

Features are released on time
Software projects have a brigade of developers, all with the attitude of making sure things get shipped when expected.  Even if that means not setting aside some refactoring time.  This process, this seemingly endless cycle of pounding out lines of code, is very rewarding.  Getting stuff done on time is a recipe for success in any software project as far as I'm concerned.

This is the habitual part of it for the programmers — who wouldn't want repeated success by continuously pleasing customers and other stakeholders by shipping early?  Don't fix what isn't broken, right?

And so life goes one — each cycle of the development process yielding a new and enhanced version of the software that exceeds customer expectations.  It's hard to change a good thing while it's good — making decisions to refactor code can seem unjustified.  Is there another way to squeeze in some code improvements that don't necessarily please the client directly?

Well, small incremental improvements can typically be tucked-into sections of the software system during the development process.  These are low-risk, low-effort improvements that don't have an immediate impact on code comprehensibility but instead have a concerted long-term impact.  The trick is — making these small incremental refactorings — is really only beneficial if they're a habitual task of each and every programmer working on the project — cumulative change for the better.  But this can be overridden, mentally, by that other little habit — satisfying the customer by shipping on time.

Forcing the issue
Two competing ideologies vetted against one another — the ship on time versus the refactor messy code — lead to one of two places.  Perhaps the worst thing that can happen is programmers get caught up in cleaning up code and as a result, deadlines for requested features tend to slip.  What we have in this scenario is some nice looking code and a frustrated group of stakeholders.  The product is better off now that the mess has been tidied — paving the way for future ingenuity — but how do you explain that to non-programmers?

The second outcome, and perhaps a good selling point for investing time in refactoring code, is a garbled mess that can't be updated with new features or enhancements to existing features.  There is an illusion that this will never happen because it takes a while to happen.  A misconception of those involved in software projects who don't write any code is that code is an immutable thing — once it's written, it's there and it should just work.  Ask any software developer and they'll tell you that any body of code is a living organism — one that must be fed and cared for.

So, forcing the need to refactor.  The best way to explain it to those involved in the project but do not understand the concept of refactoring code is to present it as a feature.  Or, sometimes even more effective, present it as a blocker for new, potentially interesting features.  This second approach takes some inventiveness, but is ultimately the most effective way to communicate the state of the code base to non programmers.  Think features they'd like to see and what would stop them from happening in terms of the current state of the code.

The reason this approach — let's call it proactive refactoring for features — works so well as a communication tool is that you're taking a deficiency in the fundamental code structure and presenting it in business objective.  Rather than trying to introduce the philosophy of refactoring code — because I can tell you right now they're not interested — you're taking a valid business concern and presenting it in a cohesive way.

One final suggestion on proactive refactoring for features — back your claims up with evidence.  Not a written words, but a quick little prototype that'll make very obvious the problem at hand.  Nothing says urgency like proof that a potential money making feature might not work in our product down the road.