Thursday, August 07, 2008 

Do we need a Theory of Wrapping?

I think it was about 10 years ago. I was teaching OOD principles to a .COM company, and a senior developer said we should really develop a theory of wrapping. We talked a little about it, but then we moved to explore other concepts. I guess I totally forgot the idea till a few weeks ago, when 3 different customers came up with thorny issues, and they all had something in common: some kind of wrapper.

Wrappers are routinely created during software development, yet not all forms of wrapping are benign. Some may look convenient in the short term, but will come back to haunt us in the long term.
Would a theory of wrapping help? Indeed, we already have some organized knowledge on wrapping: several patterns are built upon the idea of wrapping an object or a subsystem. Looking at the GoF patterns alone, Adapter is a wrapper, Facade is a wrapper, Proxy is a wrapper. Decorator, although somehow similar to proxy in implementation, isnt't really a wrapper, as it adds new responsibilities.
Looking at wrapping through patterns, however, doesn't provide much guidance about the long-term consequences of wrappers. So, let's move to anedoctal knowledge for a while.

My first troubled customer is trying to solve an architectural problem by wrapping a few functions, and perform some magic inside those functions. The magic involves threads, fibers, apartments, and stuff like that. As any true form of magic, it must not be revealed to the general public :-)), so I won't.
Magic isn't well-known for reliability, but there is chance that magic might indeed work, which in a sense is even worse. We know the real problem is elsewhere: it has been known for more than 10 years, but the troubled area has never been fixed, just hidden under several layers. The real fix would entail changing 100 to 150 classes. Personally, I would work toward making that change as painless as possible, and then do it. They would rather go the wrapping way.

My second, less troubled customer has a much simpler problem: a legacy database, which in turn is mostly a mirror of the in-memory data structures of a legacy application (still maintained and developed). We need to provide a web front end, and the general agreement is to outsource that development. We basically have three choices:
1) ask the contractor to write the web app against the existing, ugly db. That requires some documentation effort on the company side, as the db can't be used without a detailed knowledge about fields (and bit-fields inside fields).
2) clean the database (in-house), clean the legacy app (in-house), let the contractor write the web app against the clean db. Sounds nice, but requires a lof of in-house work, and even worse, it would delay the contractor. Cleaning the legacy app seems daunting, and sooner or later we want to scrap that code anyway.
3) keep the ugly db, but provide a wrapper, in the form of a facade object model. Ask the contractor to write the web app against the facade. Delay cleaning the db for a while (possibly forever) and hope that a quickly developed facade will withstand the test (or ravages) of time. Because yeah, well, we ain't got much time, or we would go with (2). By the way, we could write the facade in-house, or write the same kind of documents as in (1) and ask the contractor to write the facade.

I would love to recommend (2), but being reality-based, I recommended (3), with the facade written by the contractor. So (dare I say it!!) I actually recommended that we spend our valuable in-house time writing documentation. How non-agile is that :-)). But the thing is, there is only one guy who knows the db, and that's bad risk management. Also, if at some point we want to redesign and clean the db, a documentation of what is currently hidden inside bit-fields stored as integers would be quite valuable. Oh, by the way: we did a little experiment to estimate how long is gonna take to write that doc: about 10 man/days, easily broken in short tasks, which can partially overlap the calendar time consumed by the contractor to build the facade object model. Not so bad after all, except we still keep a crappy db and application.

My third customer doesn't really feel troubled :-). Well, in fact, he's not really troubled: I am troubled :-). Years ago, a small team ventured into building an application-specific framework. Although I often preached about creating a set of small, loosely coupled mini-frameworks, people on that team weren't so interested in listening. So they went on and created a large, tightly coupled framework (which is much easier to build, and much harder to learn, change, etc).
When the company decided to build the first (web) application based on that framework, the application developers found out (guess what :-) that the ambitious design of the large framework was quite hard to understand, to the point that their main goal was surprisingly hard to reach. I proposed that the framework developers could create a simple facade, closer to our problem and therefore easier to use. They did, the application was completed, and is still in use today. So far so good :-).
A few years went by, and more applications have been written against that framework. The original framework developers moved to application development (or leaved the company), actually leading application development. Recently, I discovered that all the subsequent applications have been written against "my" facade, which was never designed to be general-purpose. However, it was so much simpler to use than the framework that people opted to use it anyway. They tweaked the facade when necessary, so we now have multiple versions of the facade around.
Again, the "right" thing to do would have been to make the framework easy to use on the first place. The facade was my backup plan because I never managed to make a dent in the framework developers' minds. Sure, it was instrumental in writing the applications, but somehow, it backfired: it became a superstructure of its own, living well beyond my expectations. Unfortunately, it was never designed to be a general-purpose component.

Of course, not every form of wrapping is evil. For instance, I often add a small facade between my code and any third party components and libraries. That shields me from many instabilities in interfaces, and in many cases allows me to change supplier if needed. That's harder with GUI components making heavy use of design-time properties, but there are techniques to deal with that as well (if warranted). Again, some wisdom is required: it makes little sense to wrap a pervasive library or framework; it makes little sense to invest in wrapping when you're developing a short-lived corporate application, and so on. Still, as someone said (exact quote needed :-), every programming problem can be solved by adding one level of indirection, and in some cases a wrapper provides a useful level of indirection.

Can we learn a general lesson from experience? From the pattern perspective, my first customer is just trying to add some kind of proxy between subsystems. The second and third are using a plain old facade. Nothing wrong with that. Yet no one is making the "right" choice: the wrapper is used to avoid fixing the real problem. We often say words like "postpone", but in software development, "postpone" usually means "don't do it".

So, again, would a theory of wrapping help? And what should this theory tell us? As is often the case, a good theory of software design should help us to make informed decisions, and like it or not, that means informed economical decisions. All too often, sound engineering choices are discarded in favor of cheap solutions, because we have no real economic model, or even a rough economic model, to calculate the (possibly negative :-) ROI of what we propose.

The real, long-term economical consequences for my first customer are way too complex, and I have little hope that we could ever develop a theory that could even come close to help with that. However, I believe the bigger long-term cost factor of the proxy-based solution would be in the increased debugging effort that would fall on the shoulders of all the other programmers working on the same project (about 100 people, I guess). This problem, somehow, could be highlighted in a theory of wrapping. A proxy, especially when doing threads-fibers-apartments magic, makes it hard to get a clear picture of what's going on just by looking at the stack (which is a per-thread, per-fiber structure). Unfortunately, the impact of this problem seems very hard to quantify.

My second customer is facing a much simpler scenario. I think we could eventually build an economic theory that could model the maintenance cost of keeping the legacy code as-is, the risk of building a leaky abstraction as we quickly build a facade over the legacy db, thereby increasing the mass of code that we'll have to change later, as we finally get to create a better database, and compare that with the savings of not doing it. We should also be able to factor in the increased maintenance cost of keeping the facade in-synch with any changes to the db required by the evolving application. It's not rocket science. Or maybe it is, but we can actually send a rocket to Mars :-). A good economic theory should recommend, in this specific context, to go with (3). Why? Because that's what I did :-)).

The third problem, I think, is more of a process problem. There was nothing wrong in the original wrapper. However, it wasn't designed for reuse, and is should not have been reused as-is, or by tweaking. This is a direct consequence of a lack of control over design choices; the team is, well, wrapped :-)) on itself. Unfortunately, lack of control over design issues is exactly what the team wants. A good process is meant to keep that kind of disfunctional behaviour out. Note that it doesn't have to be an heavyweight process. Just a good process :-).

Unfortunately, I don't have Theory of Wrapping to offer. At least, not yet :-). So, for the time being, I'll keep dealing with wrapping using informal reasoning, mostly based on experience and intuition. Got some good idea? Leave me a comment here!

Labels: , , ,

a good theory of software design should help us to make informed decisions, and like it or not, that means informed economical decisions

Isn't this at odd with the highly conceptual arguments you discussed on the concept of form?
by the way, you never completed those arguments!
Hi Jin,
excellent question!

I think it's not really at odd. Design requires the ability to generate ideas, and then the ability to evaluate those ideas.

What I'm discussing under the "concept of form" umbrella is mostly concerned with how to generate ideas, although yes, some of the same concepts can be used to evaluate the technical merit of a given solution.

The generative process is hard. Many people, unfortunately, get stuck with their first idea, and spend most of their time defending that idea, instead of looking for alternatives. By the way, this is also why I'm so skeptical about using only simplistic design concepts like TDD or let the design "emerge" through coding. It doesn't give you enough alternatives.

In the end, however, the choice is always economic, and by that, I mean that we always have to evaluate the technical merits, the cost and effort, the company strategy for the product, the company structure and people, and so on. A good theory of software design should help us with this step too, as a necessary complement to the generative phase.

I'm not over with the concept of form! There is a lot more I've to say, just stay tuned :-).
Of course, not every form of wrapping is evil. For instance, I often add a small facade between my code and any third party components and libraries. That shields me from many instabilities in interfaces, and in many cases allows me to change supplier if needed.

I agree ;-))
Post a Comment

<< Home