Monday, January 14, 2008 

Being 10 Years Behind (part 1)

In the last two years I've been working quite closely with a company, designing the fourth generation of a successful product. Indeed, a few of my posts have been inspired by the work I did on that project.
What we have now is a small, flexible, fast web application where we definitely pushed the envelope using AOP-like techniques pervasively, although in .NET/C#.

Compared with the previous generation, our application has more features, is much easier to customize (a must), is much easier to use thanks to the task-oriented design of the HCI (the previous generation was more slanted toward the useless computer approach) and also about 10 times faster (thanks to better database design and a smarter business layer).
Guess what, the source code size is just about 1/30 of what we had before (yeap, 30 times less), excluding generated code to read/write some XML files. The previous application was written in a mixture of C++, VB6, Perl, Python, C#.

Now, the company is considering a strict partnership with an Asian corporation. They have a similar product, in ASP.NET / C# as well. It took them something like 30 times our man-months to write it, so the general feeling was that it should have been "more powerful". Time to look at the features, but hey, features are basically the same, although their product is not task-oriented. If anything, they lack a lot of our customization interface.

Time to look at the code, as the code never lies.
The code is probably 50 times bigger, with no flexibility whatsoever. If you need to attach one more field to a business concept, just to track some custom information, you probably have to change the database, 5-8 classes between the database and the GUI, and the GUI itself.
In most cases, in our application we just need to open the administration console, add the field, specify validation rules, and that's it.
If you have special needs, you write a custom class, decorate the class with attributes, and we take care of all the plumbing to instantiate / call your class in the critical moments (that's one of the several places where the AOP-like stuff kick in).

What struck me when I looked at that code was the (depressing :-) similarity with a lot of old-style Java code I've been seeing over the years, especially in banking environments.

There is an EntityDAO (data access object) package with basically one class for each business entity. That class is quite stupid, basically dealing with persistence (CRUD) and exposing properties. Those classes are used "at the bottom" of the system, that is, near to the database.

Then there is an Entity package where (again!) there is basically one class for each (business) entity. The class is completely stupid, offering only get/set methods. Those classes are used "at the top" of the system, that is, near to the GUI or external services.

There is a BusinessLogic package where Entities gets passed to various classes as parameters, and EntityDAO objects are used to carry out persistency-related tasks.
Actually, inside the BusinessLogic lies a lot of database-related code, sometimes with explicit manipulation of DataRow classes. The alternative would have been to create much more EntityDAO classes.

Here and there, the coupling between the BusinessLogic and the database must have seemed too strong, so an EntityReader package has been created, where more sophisticated (yet still stupid) entities (or collections) are built using EntityDAO classes.

Finally, you just need :-) something to call from your service or GUI layer. The ubiquitous BusinessFacade package is therefore introduced, implementing a very large, use-case driven interface (put in yet another package), taking Entity instances as parameters and using the BusinessLogic.

At that point, people invariably realize that services need much more logic than what is provided by the BusinessLogic package, and so go ahead and create a (very sad) BusinessHelper package, where they complement all the missing parts in the BusinessLogic, most often by direct database access.

Then we have other subsystems (cache, SQL, and so on) all built around XXXManager classes, which we can ignore.

Of course, in the end everything is so coupled with the database schema that just adding a field results in a nightmare. And you get a lot of code to maintain as well. Good luck. Meanwhile, the Ruby On Rails guys are creating (simple) applications faster than the other guys can spell BusinessHelper. Say good-bye to productivity.

We can blame it on static typing, but reality is much simpler. That architecture is wrong. Is at least 10 years behind from the state of practice, which means is probably 15 to 20 years behind from the state of the art.
The problem is, that ancient architecture was popularized years ago mostly by language and tools vendors, or by people who thinks Architects don't have to understand code or the real problem being solved, just to replicate a trivial-yet-humonguous structure everywhere. It's basically a decontextualized architecture (more on this next time).

Indeed, if you look at the Java literature, you can find good books dating back to the early decade (like "EJB Design Patterns" by Floyd Marinescu, year 2002), where the pros and cons of several choices adopted in that overblown yet fragile architectural model are discussed. When a Patterns book appears, a few years of practice are gone by. That was 2002; now, ten years are gone, and yet developers still fall into the same trap.

It gets worse. While even the most outdated Java applications are gradually moving away from that model (see Untangling Enterprise Java by Chris Richardson for a few ideas), Microsoft evangelists are so excited about it. They happily go around (re)selling an architecture that is remarkably similar to the 10-years-behind behemoth above.

Which brings me to "Being 10 Years Behind (part 2)". Stay tuned :-).

Labels: , , ,

Tuesday, June 19, 2007 

Client Side-Augmented Web Applications

In the last few posts I've been writing a lot about AOP, and very little about what I'm doing every other day. It's plain impossible to catch up, but here is something that has kept me busy for quite a few days lately: Client Side Augmented Web Applications (I should file for a trademark here :-). What I mean is a regular web application, that can be regularly used as a stand-alone application or, when you install some additional modules on the client, can also interact closely with other applications on the client side (e.g. the usual office suite, and so on).

Naturally, that means web pages must have a way to send data to client side application, and to obtain data from the client side application. For several (good) reasons, we wanted this data exchange to be as transparent as possible to the web app developers. Also, we didn't want to write different web applications (regular and augmented). That would have had a negative impact on development and maintenance times, and it could also have proven to be an inconvenience for the users. This had some impact on page navigation design, which could be an interesting subject for a future post.

Now, I can't get into the specifics of the project, or disclose much about the design of the whole infrastructure I've designed and built (yes, I still enjoy writing code :-). However, I can show you the final result. If you want your ASP.NET page to obtain (e.g.) the filename and title of your Word document, and send back to Word a corporate-wide document number, all you have to do is add a few decorated properties in your .aspx.cs source file, like:

public partial class DocumentProperties : System.Web.UI.Page
  {
  [AskClient("filename")]
  public string Filename
    {
    get
      {
      return filename;
      }
    set
      {
      filename = value;
      }
    }

  [AskClient( "title" )]
  public string Title
    {
    get 
      { 
      return title; 
      }
    set 
      { 
      title = value; 
      }
    }

  [SendClient("description")]  
  public string Description
    {
    get
      {
      return description;
      }
    set
      {
      description = value;
      }
    }

  // here goes the usual stuff (methods, data members)
  }

that's it. The attributes define the property name as known on the client side; the invisible infrastructure will take care of everything else. In a sense, communication between the web application and the client side application has been modeled as a virtual machine concern, and the attributes are used to tell the virtual machine where interception is needed. Of course, this is only the tip of the iceberg. There is a lot going on under the hood, also on the client side, as your browser and your favorite application are usually not good friends, and to be honest not even acquaintances.

Is my abstraction leaky? Sure it is. To make all that stuff working, you also have to drop a non-visual component into your page at design time. That component, if you follow the virtual machine metaphor, is indeed the implementation of the (server-side) virtual machine layer that will deal with the communication concern.
If you don't drop the component into the page, the virtual machine layer just won't kick in, and your attributes will stay there silently, stone cold. This is a functional leak that I'm aware of, and that as a designer I have accepted. In fact, all the alternatives I've considered to avoid this leak had some undesirable consequences, and keeping that leak brought the best overall balance. Besides, there are ways to somehow hide the need to drop the control into the page (like using page inheritance or a master page), so it's really no big deal.

A final reflection. I do not believe that something like this could come out of refactoring code that was simply meant "to work". It's relatively trivial to make a web application and a client-side application talk. It's quite different to make this talk transparent. Having prototyped the first option (just to make sure a few ideas could actually work) I can honestly say that without the necessary design effort (and skill), it's extremely unlikely to come close to the final result I got.

Time to drop a few numbers: overall, I've spent roughly 20% of total development time experimenting with ideas (throwaway code), 35% designing, 30% coding (this includes some unit testing), 10% doing end-to-end testing, 5% debugging. Given the relative novelty of several techniques I adopted, I should actually consider the 20% prototyping an inherent portion of the design activity: you can't effectively design much beyond the extent of your knowledge. Sometimes, the best way to gather more knowledge is to talk; sometimes, to read; sometimes, to write code. Of course, I was looking for knowledge, not for code to keep and cherish, so I happily scrapped it away to build much better code. In the end, that's often the difference between intentional and accidental architecture.

Labels: , , ,