Wednesday, August 31, 2011

Widgets And Frameworks

Web frameworks are important because today, the target user environment is the web.  Why is this?  Because the web means anyone can use your application.  There are no barriers to installing the software and no restrictions on the hardware.  All the user needs is a decent web browser.

The web application framework itself is relevant because it enables developers to write solid software by reusing common components.  Common patterns used in all web applications.  Without these tools supplied by the framework, nothing would ever get off the ground — a diffuse time to market for smaller development teams.

What patterns are used in web frameworks?  Database connectivity, URL mapping, authentication.  The list goes on, each framework providing their own flavor of these major components.  Each is considered a pattern because any web application needs to implement it.  This quickly burdens the developer.  The role of a web framework is to parameterize these patterns and make them accessible.

What role do widgets play in web application frameworks?  By widgets, I mean standalone Javascript toolkits that have a selection of cool user interface stuff.  This includes jQuery UI, Dojo, and ExtJS.  Making these Javascript widget tool kits — which are a framework on their own — integrate seamlessly into our web application frameworks is disjointed to say the least.

Not an easy fit
Web application frameworks don't give us all the front-end browser code necessary for a finished product.  If it were a matter of delivering only HTML and CSS to the browser, we'd be all set.  This isn't the case unfortunately.  Not if there is any kind of user interaction involved — which is almost certainly the case.  For the kind of experiences users have come to expect, developers need to employ Javascript or Flash technologies.

In theory, building these rich user experiences is fairly straightforward — alter the HTML templates and serve the additional resources — Javascript, Flash, etc.  This can be achieved without any headaches — granted the project is a small one.  There many resources to worry about, not as big a user base compared to something enterprise-grade.  Every aspect of the software is scaled back slightly — and so the cause for error is scaled back accordingly.  For example, having a very large Javascript file or dozens of smaller Javascript files can be hard to maintain.  Not on their own — but because Javascript is different — it depends heavily on the DOM.

So as far as the development process goes, we need sound approach to ensuring there is some degree of coupling between individual HTML templates and their corresponding Javascript modules.  The days of implementing a monolithic Javascript file are quickly leaving us behind.  Especially if given the challenges of managing external widget library dependencies.  Imagine an application with a hundred or more HTML templates — I can't think of an optimal method where a single Javascript module suffices.

Technically speaking, uniting widget frameworks and web application frameworks wouldn't be overly strenuous.  All this entails, and this is generally how they're put together now, is serving new resources and having your HTML content ask for those resources.  The place where widgets don't fit seamlessly into the web application framework is in the back-end the web application.  How does a given view or controller (whatever the framework terminology alludes) know about which widgets are used?  Are these widget configurations persistent?  For that matter, are they configurable at all from the web framework?  For me this is a problem — widgets are a big part of the overall picture — web application frameworks should accommodate widget frameworks.

Frameworks and forms
A crucial feature of any software with a web user interface is forms.  Forms collect user feedback.  They can be simple — a single input field and a submit button.  They can be complex — a registration form that nobody in their right mind would fill out.  Web application frameworks do a pretty good job with tools that both render and validate forms.

There is good reason for.  Web frameworks define the database schema for the application.  The database is populated by user input.  Forms.  There is an obvious connection between the two concepts — this is why frameworks take great care to supply the database with what it expects.

Take Django for example.  The Django framework has a forms module that'll assist with defining forms, the validation, and rendering the form widgets for the browser.  Not unlike how Flask uses WTForms to validate and present forms.  The Pylons project also provides developers with form processing capabilities — they require a little more effort than the other Python frameworks.

What really stands out is the connection forms and their individual input widgets have the framework itself.  They're an essential ingredient to the application — a formative piece of the database.  There is a mapping between the database schema and what is presented to the end user so they can populate it.  This fits perfectly with the ideal of the web application framework — to take a pattern and expose it as a cohesive API for the developer.  This concept is further exemplified by Django where we're able to point the form subsystem to a database model and let it figure out how the form should be rendered and validated.  This isn't so easy with more general widgets that aren't necessarily used for collecting input and storing it.

Widgets and data
If we're able to utilize the framework form processing tools to store data, how does the framework help us present data to the user?  Well, since a web application framework is only responsible for delivering the content to be rendered by the browser, all we've got is templates.  These assist in generating the markup necessary for displaying fancy Javascript widgets.

For instance, a common user interface pattern is to group logical categories using tabs.  Tabs are visually appealing and allow for intuitive navigation.  This is a top-level layout widget — meant to replace a typical sidebar navigation item.  Or consider a smaller widget — a progress bar.  This type of widget, unlike tabs, isn't a layout container.  It displays something specific about the application data.

What both layout container widgets and smaller application data widgets have in common is that they both depend on specific application details that reside on the server, within the framework.  The tabs widget needs to know what specific tabs to display and what content within each tab.  The progress bar widget probably depends on a computed number — derived from user data from the database.  So what is the preferred approach to displaying both widget types in any given application?

Navigational constructs are typically done in the template.  Maybe there is a nested div and a tag structure the Javascript widget is expecting — like the jQuery UI tabs widget.  However, the developer still needs to instantiate their widget when the page loads.  This means that the Javascript code needs to know about the template and what the rendered HTML output will look like.  This is somewhat error-prone as the developer will need to maintain dependencies between the Javascript widgets and the HTML structure they're expecting.

More challenging is the application user data presented by widgets — like a progress bar.  The progress bar needs the value to display — but where does it come from?  This might be a JSON response, returned from one of the application controllers.  Again, as with defining layout widgets, the problem lies in maintaining the connection between what the Javascript widget is expecting to see and what the framework delivers.  The problem isn't so much technical — it is more of a design problem.  One that leads to a loosely coupled system.  Too loose.

Solutions and challenges
Web frameworks already do a great job of taking common patterns on the server and making tools developers can use to efficiently develop code.  Might it be that web application frameworks have no business managing widgets?  Should this task be left to the templates and other static files?  I think there are patterns we're missing here that can be taken advantage of — it's still only an emergent thought.

One benefit to having the framework manage the connection between application data and widgets is the potential to be widget framework agnostic.  There are a lot of widget tool kits out there and a lot of similarities between them.  The commonalities across all toolkits — these are the interesting widgets.  The basic widgets that all web applications use in one form or another could potentially be integrated into web frameworks.

What might we do with the layout widgets to help standardize and improve the coupling between Javascript and framework data?  After all, layouts, if they can be standardized in some fashion, are a valuable asset.  You'll generally have a web application framework of choice and you'll almost certainly be creating more than one application.

I think layout widgets — stuff specific to the user interface and not necessarily what the application does — will be easier for frameworks to implement.  If the framework is able to introduce the concept of a widget, maybe storing it's configuration, then properly outputting the required markup and the required Javascript isn't a stretch.  For example, the configuration for a tabs widget would store what tabs are actually rendered, the contents of each tab, and should produce the necessary Javascript to instantiate it.

Widgets specific to application data — like a grid — aren't as easy.  This is because they're more toward what the application does.  Another challenge is that widgets generally depend on some data source like JSON that is generated from the user data.  This isn't so easy to stardardize on.  The framework would have to know about the data schema in addition to the visual configuration.  This means the application would have to store configuration that dictates what the widget will look like, where to get the data it needs, and how that data should be interpreted for presentation.  Not impossible, just more challenging.

Perhaps a good start to making web application frameworks a little more aware of patterns in how browser widgets interact with the framework itself is to discover patterns you're using.  How would you benefit if you're able to store some configuration that defines how and what the Javascript widgets do?  Every application is different — and so every combination of widgets used brings a new set of challenges.  This is how we make progress and is how the current features of frameworks came into being — by experimentation and pattern discovery.