Tuesday, May 27, 2008
On the concept of Form (3): the Force Field
Warning: :-) this post is going to be somehow conceptual. I'll soon move to some real-world, software-based example, but I really need to introduce some concepts first.
The notion of force field might be unfamiliar to some, so I'll borrow a great example from Alexander himself. Consider your first "requirement" (for a system yet to be built) as a permanent magnet of some size and shape. If you place a flat glass over that magnet, and drop some iron filings on it, the iron will naturally dispose along the magnetic field lines. That gives us an image of [a section of] the force field. Now add another magnet: the shape of the field will change, as the magnets are interacting, thereby shaping a more complex force field.
We can change the [shape of the] field in many ways: moving magnets around, changing their shape, their magnetization, or even adding some shields around magnets.
The great thing about the magnetic field is that we can somehow observe its shape. Indeed, if our goal was to create a form that can be put into effortless contact with the field, we'll just have to replicate the same form that the magnetic field is giving to the iron filings. As Alexander says (NoTSoF, page 21), "once we have a diagram of forces [...] this will in essence also describe the form as a complementary diagram of forces".
In the real world, and even more so in the software world, we are never so lucky: the force field is invisible and tends also to be highly unstable.
Usually, the force field of a software project starts with Requirements. Requirements are often categorized in some way, like "functional" and "nonfunctional", or "user requirements" and "system requirements. However, requirements of any kind are just like magnets: they contribute to shape the overall field.
Requirements are just one kind of force, that is, they are not alone in shaping the field. Many technological choices we make, sometimes very (or too) early, are also shaping the force field.
Consider a simple business application. Once you decide that you'll build a web application, you have added quite a few powerful magnets. If you're familiar with JSP and EJB, you are naturally tempted to choose those technologies early on. That's like adding quite a few powerful magnets again. Or maybe it's like adding a magnetic shield: it really depends on context.
Sometimes, technology makes the field simpler: the right infrastructure should simplify the field, that is, it should act more like a shield than like a magnet. In this sense, infrastructure should be chosen when the dominant forces are known, unlike what happens in many projects, where infrastructure (usually a superstructure in disguise) is chosen too early, thereby making the overall field even more complex.
We also shape the field, so to speak, by choosing what to ignore and what to postpone in any given release. Anything we ignore, like anything we postpone, won't be allowed to shape the field right now.
This is fine, as long as the corresponding magnets will be placed somehow distant from the others (good modularity), possibly with some kind of magnetic shielding in between (stable interfaces). It's also fine if we can ignore it forever. Any attempt to temporarily ignore a strictly interacting force will wreak havoc later on, as our form will no longer match the resulting force field. Refactoring can accommodate minor misfits with the ideal form, but won't help much when the force field changes radically (see also my notes on refactoring here).
Here lies one of the architect's fundamental abilities: the intuitive understanding that something can be beneficially postponed, while something else must be dealt with immediately, because its influence on the force field is so strong that doing otherwise will shift us toward the wrong kind of form.
It is important to understand the role of choice in exploiting instability. Too often, software developers tend to see requirements as "fixed". They don't like to negotiate: it's much easier to fight the compiler than the marketing guys.
A good architect, however, can't miss the opportunity to simplify the field by moving some magnets around. That requires the ability to see the overall picture and the fine details at the same time. Here is Alexander again (page 18): "this ability to deal with several layers of form-context boundaries in concert is an important part of what we often refer to as the designer's sense of organization. The internal coherence of an ensemble depends on a whole net of such adaptations".
That ain't an easy feat. It requires an understanding of the business, the users, and the technology. And even more important, it requires a willingness to act on that knowledge. The power of choice extends to the infrastructure: sometimes, by willingly postponing a technological choice until the force fields takes shape, we can make a better, more "natural" choice.
This can be hard for some developers: they want certainty, and they want it now. In my experience, that goes in pair with the willingness to adopt a sub-optimal, but repetitive and context free solution for a wide class of problems, instead of adopting several optimal, but reasoned and context-dependent solution for smaller classes of problems.
Unfortunately, choosing the "wrong" technology is very much like choosing the wrong shape or orientation for a building. To quote Alexander once again (page 29): "Instead of orienting the house carefully for sun and wind, the builder conceives its organization without concern for orientation, and light, heat, and ventilation are taken care of by fans, lamps, and other kinds of peripheral devices. Bedrooms are not separated from living rooms in plan, but are placed next to one another and the walls between them stuffed with acoustic insulation".
I think we can easily see a parallel with software here: a misfit technology is chosen early on. As a consequence, you find yourself adding more and more technology (fans, lamps, insulation) to satisfy the end-user needs. "Modern" web applications seem to have taken this path: faced with a difficult field, they're layering one technology on top the other, desperately trying to overcome the problems of the previous layer.
Next time, in no particular order: agility, unstable requirements, early coding, TDD, "seeing" the field, internal and external representations, is UML any useful, order within chaos (dominant forces), constructive force field and systematic techniques, and whatever else will come to my mind :-).
The notion of force field might be unfamiliar to some, so I'll borrow a great example from Alexander himself. Consider your first "requirement" (for a system yet to be built) as a permanent magnet of some size and shape. If you place a flat glass over that magnet, and drop some iron filings on it, the iron will naturally dispose along the magnetic field lines. That gives us an image of [a section of] the force field. Now add another magnet: the shape of the field will change, as the magnets are interacting, thereby shaping a more complex force field.
We can change the [shape of the] field in many ways: moving magnets around, changing their shape, their magnetization, or even adding some shields around magnets.
The great thing about the magnetic field is that we can somehow observe its shape. Indeed, if our goal was to create a form that can be put into effortless contact with the field, we'll just have to replicate the same form that the magnetic field is giving to the iron filings. As Alexander says (NoTSoF, page 21), "once we have a diagram of forces [...] this will in essence also describe the form as a complementary diagram of forces".
In the real world, and even more so in the software world, we are never so lucky: the force field is invisible and tends also to be highly unstable.
Usually, the force field of a software project starts with Requirements. Requirements are often categorized in some way, like "functional" and "nonfunctional", or "user requirements" and "system requirements. However, requirements of any kind are just like magnets: they contribute to shape the overall field.
Requirements are just one kind of force, that is, they are not alone in shaping the field. Many technological choices we make, sometimes very (or too) early, are also shaping the force field.
Consider a simple business application. Once you decide that you'll build a web application, you have added quite a few powerful magnets. If you're familiar with JSP and EJB, you are naturally tempted to choose those technologies early on. That's like adding quite a few powerful magnets again. Or maybe it's like adding a magnetic shield: it really depends on context.
Sometimes, technology makes the field simpler: the right infrastructure should simplify the field, that is, it should act more like a shield than like a magnet. In this sense, infrastructure should be chosen when the dominant forces are known, unlike what happens in many projects, where infrastructure (usually a superstructure in disguise) is chosen too early, thereby making the overall field even more complex.
We also shape the field, so to speak, by choosing what to ignore and what to postpone in any given release. Anything we ignore, like anything we postpone, won't be allowed to shape the field right now.
This is fine, as long as the corresponding magnets will be placed somehow distant from the others (good modularity), possibly with some kind of magnetic shielding in between (stable interfaces). It's also fine if we can ignore it forever. Any attempt to temporarily ignore a strictly interacting force will wreak havoc later on, as our form will no longer match the resulting force field. Refactoring can accommodate minor misfits with the ideal form, but won't help much when the force field changes radically (see also my notes on refactoring here).
Here lies one of the architect's fundamental abilities: the intuitive understanding that something can be beneficially postponed, while something else must be dealt with immediately, because its influence on the force field is so strong that doing otherwise will shift us toward the wrong kind of form.
It is important to understand the role of choice in exploiting instability. Too often, software developers tend to see requirements as "fixed". They don't like to negotiate: it's much easier to fight the compiler than the marketing guys.
A good architect, however, can't miss the opportunity to simplify the field by moving some magnets around. That requires the ability to see the overall picture and the fine details at the same time. Here is Alexander again (page 18): "this ability to deal with several layers of form-context boundaries in concert is an important part of what we often refer to as the designer's sense of organization. The internal coherence of an ensemble depends on a whole net of such adaptations".
That ain't an easy feat. It requires an understanding of the business, the users, and the technology. And even more important, it requires a willingness to act on that knowledge. The power of choice extends to the infrastructure: sometimes, by willingly postponing a technological choice until the force fields takes shape, we can make a better, more "natural" choice.
This can be hard for some developers: they want certainty, and they want it now. In my experience, that goes in pair with the willingness to adopt a sub-optimal, but repetitive and context free solution for a wide class of problems, instead of adopting several optimal, but reasoned and context-dependent solution for smaller classes of problems.
Unfortunately, choosing the "wrong" technology is very much like choosing the wrong shape or orientation for a building. To quote Alexander once again (page 29): "Instead of orienting the house carefully for sun and wind, the builder conceives its organization without concern for orientation, and light, heat, and ventilation are taken care of by fans, lamps, and other kinds of peripheral devices. Bedrooms are not separated from living rooms in plan, but are placed next to one another and the walls between them stuffed with acoustic insulation".
I think we can easily see a parallel with software here: a misfit technology is chosen early on. As a consequence, you find yourself adding more and more technology (fans, lamps, insulation) to satisfy the end-user needs. "Modern" web applications seem to have taken this path: faced with a difficult field, they're layering one technology on top the other, desperately trying to overcome the problems of the previous layer.
Next time, in no particular order: agility, unstable requirements, early coding, TDD, "seeing" the field, internal and external representations, is UML any useful, order within chaos (dominant forces), constructive force field and systematic techniques, and whatever else will come to my mind :-).
Labels: architecture, design, form, requirements
Friday, April 04, 2008
Asymmetry
I'm working on an interesting project, trying to squeeze all the available information from sampled data and make that information useful for non-technical users. I can't provide details, but in the end it boils down to reading a log file from a device (amounting to about 1 hour of sampled data from multiple channels), do the usual statistics, noise filtering, whatever :-), calculate some pretty useful stuff, and create a report that makes all that accessible to a business expert.
The log file is (guess what :-) in XML format, meaning it's really huge. However, thanks to modern technology, we just generated a bunch of classes from the XSD and let .NET do the rest. Parsing is actually pretty fast, and took basically no time to write.
In the end, we just get a huge collection of SamplingPoint objects. Each Sampling point is basically a structure-like class, synthesized from the XSD:
class SamplingPoint
{
public DateTime Timestamp { // get, set }
public double V1 { // get, set }
// ...
public double Vn { // get, set }
}
each value (V1...Vn) is coming from a different channel and may have a different unit of measurement. They're synchronously sampled, so it made sense for whoever developed the data acquisition module to group them together and dump them together in a single SamplingPoint tag.
We extract many interesting facts from those data, but for each Vi (i=1...N) we also show some "traditional" statistics, like average, standard deviation and so on.
Reasoning about average and standard deviation is not for everyone: I usually consider an histogram of the distribution much easier to understand (and to compare with other histograms):
Here we see the distribution of V1 over time: for instance, V1 had a value between 8 and 9 for about 6% of the time. Histograms are easy to read, and users quickly asked to see histograms for each V1..Vn over time. Actually, since one of the Vj is monotonically increasing with time, they also asked to see the histogram of the remaining Vi against Vj too. So far, so good.
Now, sometimes I hate writing code :-). It usually happens when my language doesn't allow me to write beautiful code. Writing a function to calculate the histogram of (e.g.) V1 against time is trivial: you end up with a short piece of code taking an array of SamplingPoints and using the V1 and Timestamp properties to calculate the histogram. No big deal.
However, that function is not reusable, exactly because it's using V1 and Timestamp. You can deal with this in at least 3 unpleasant :-) ways:
1) you don't care: you just copy/paste the whole thing over and over. If N = 10, you get 19 almost-identical functions (10 for time, 9 for Vj).
2) you restructure your data before processing. Grouping all the sampled data at a given time in a single SamplingPoint structure makes a lot of sense from a producer point of view, but it's not very handy from a consumer point of view. Having a structure of arrays (of double) instead of an array of structures would make everything so much simpler.
3) you write an "accessor" interface and N "accessors" classes, one for each Vi. You write your algorithms using accessors. Passing the right accessors (e.g. for time and V1) will get you the right histogram.
All these options have some pros and cons. In the end, I guess most people would go with (2), because that brings us into the familiar realm of array-based algorithms.
However, stated like this, it seems more like a "data impedance" problem between two subsystems than a language problem. Why did I say it's the language fault? Because the language is forcing me to access data members with compile-time names, and does not (immediately) allow me to access data members using run-time names.
Don't get me wrong: I like static typing, and I like compiled languages. I know from experience that I tend to make little stupid mistakes, like typing the wrong variable name and stuff like that. Static typing and compiled languages catch most of those stupid mistakes, and that makes my life easier.
Still, the fact that I like something doesn't mean I want to use that thing all the time. I want to have options. Especially when those options would be so simple to provide.
In a heavily reflective environment like .NET, every class can be easily considered an associative container, from the property/data member names to property/data member values. So I shold be able to write (if I wanted):
SamplingPoint sp = ... ;
double d1 = sp[ "V1" ] ;
which should be equivalent to
double d1 = sp.V1 ;
Of course, that would make my histogram code instantly reusable: I'll just pass the run-time names of the two axes. You can consider this equivalent to built-in accessors.
Now, I could implement something like that on my own, using reflection. It's not really difficult: you just have to gracefully handle collections, nested objects, and so on. Unfortunately, C# (.NET) do not allow a nice implementation of the concept, mostly for a bunch or constraints they added to conversion operators: no generic conversion operators (unlike C++), no conversion to/from Object, and so on. In the end you may need a few more casts that you'd like to, but it can be done.
I'll also have to evaluate the performance implications for this kind of application, but I know it would make my life much easier in other applications (like binding smart widgets to a variety of classes, removing the need for braindead "controller" classes). It's just a pity that we don't have this as built-in language feature: it would be much easier to get this right (and efficient) at the language level, not at the library level (at least, given C# as it is now).
Which brings me to the concept of symmetry. A few months ago I stumbled upon a paper by Jim Coplien and Zhao Liping (Understanding Symmetry in Object-Oriented Languages, published in Journal of Object Technology, an interesting, free publications that's filling the void left by the demise of JOOP). Given my interest on the concept of form in software, the paper caught my attention, but I postponed further thinking till I could read more on the subject (there are several papers on symmetry in Cope's bibliography, but I need a little more time than I have). A week ago or so, I've also found (and read) another paper from Zhao in Communications of ACM, March 2008: Patterns, Symmetry, and Symmetry breaking.
Some of their concepts sound odd to me. The concept of symmetry is just fine, and I think it may help to unravel some issues in language design.
However, right now the idea that patterns are a way to break symmetry doesn't feel so good. I would say exactly the opposite, but I really have to read their more mathematically-inclined papers before I say more, because words can be misleading, while theorems usually are not :-).
Still, the inability to have built-in, natural access to fields and properties through run-time names struck me as a lack of symmetry in the language. In this sense, the Accessor would simply be a way to deal with that lack of symmetry. Therefore it seems to me that patterns are a way to bring back symmetry, not to break symmetry. In fact, I can think of many cases where patterns "expose" some semantic symmetry that was not accessible because of (merely) syntactic asymmetry.
More on this as I dig deeper :-).
The log file is (guess what :-) in XML format, meaning it's really huge. However, thanks to modern technology, we just generated a bunch of classes from the XSD and let .NET do the rest. Parsing is actually pretty fast, and took basically no time to write.
In the end, we just get a huge collection of SamplingPoint objects. Each Sampling point is basically a structure-like class, synthesized from the XSD:
class SamplingPoint
{
public DateTime Timestamp { // get, set }
public double V1 { // get, set }
// ...
public double Vn { // get, set }
}
each value (V1...Vn) is coming from a different channel and may have a different unit of measurement. They're synchronously sampled, so it made sense for whoever developed the data acquisition module to group them together and dump them together in a single SamplingPoint tag.
We extract many interesting facts from those data, but for each Vi (i=1...N) we also show some "traditional" statistics, like average, standard deviation and so on.
Reasoning about average and standard deviation is not for everyone: I usually consider an histogram of the distribution much easier to understand (and to compare with other histograms):
Here we see the distribution of V1 over time: for instance, V1 had a value between 8 and 9 for about 6% of the time. Histograms are easy to read, and users quickly asked to see histograms for each V1..Vn over time. Actually, since one of the Vj is monotonically increasing with time, they also asked to see the histogram of the remaining Vi against Vj too. So far, so good.
Now, sometimes I hate writing code :-). It usually happens when my language doesn't allow me to write beautiful code. Writing a function to calculate the histogram of (e.g.) V1 against time is trivial: you end up with a short piece of code taking an array of SamplingPoints and using the V1 and Timestamp properties to calculate the histogram. No big deal.
However, that function is not reusable, exactly because it's using V1 and Timestamp. You can deal with this in at least 3 unpleasant :-) ways:
1) you don't care: you just copy/paste the whole thing over and over. If N = 10, you get 19 almost-identical functions (10 for time, 9 for Vj).
2) you restructure your data before processing. Grouping all the sampled data at a given time in a single SamplingPoint structure makes a lot of sense from a producer point of view, but it's not very handy from a consumer point of view. Having a structure of arrays (of double) instead of an array of structures would make everything so much simpler.
3) you write an "accessor" interface and N "accessors" classes, one for each Vi. You write your algorithms using accessors. Passing the right accessors (e.g. for time and V1) will get you the right histogram.
All these options have some pros and cons. In the end, I guess most people would go with (2), because that brings us into the familiar realm of array-based algorithms.
However, stated like this, it seems more like a "data impedance" problem between two subsystems than a language problem. Why did I say it's the language fault? Because the language is forcing me to access data members with compile-time names, and does not (immediately) allow me to access data members using run-time names.
Don't get me wrong: I like static typing, and I like compiled languages. I know from experience that I tend to make little stupid mistakes, like typing the wrong variable name and stuff like that. Static typing and compiled languages catch most of those stupid mistakes, and that makes my life easier.
Still, the fact that I like something doesn't mean I want to use that thing all the time. I want to have options. Especially when those options would be so simple to provide.
In a heavily reflective environment like .NET, every class can be easily considered an associative container, from the property/data member names to property/data member values. So I shold be able to write (if I wanted):
SamplingPoint sp = ... ;
double d1 = sp[ "V1" ] ;
which should be equivalent to
double d1 = sp.V1 ;
Of course, that would make my histogram code instantly reusable: I'll just pass the run-time names of the two axes. You can consider this equivalent to built-in accessors.
Now, I could implement something like that on my own, using reflection. It's not really difficult: you just have to gracefully handle collections, nested objects, and so on. Unfortunately, C# (.NET) do not allow a nice implementation of the concept, mostly for a bunch or constraints they added to conversion operators: no generic conversion operators (unlike C++), no conversion to/from Object, and so on. In the end you may need a few more casts that you'd like to, but it can be done.
I'll also have to evaluate the performance implications for this kind of application, but I know it would make my life much easier in other applications (like binding smart widgets to a variety of classes, removing the need for braindead "controller" classes). It's just a pity that we don't have this as built-in language feature: it would be much easier to get this right (and efficient) at the language level, not at the library level (at least, given C# as it is now).
Which brings me to the concept of symmetry. A few months ago I stumbled upon a paper by Jim Coplien and Zhao Liping (Understanding Symmetry in Object-Oriented Languages, published in Journal of Object Technology, an interesting, free publications that's filling the void left by the demise of JOOP). Given my interest on the concept of form in software, the paper caught my attention, but I postponed further thinking till I could read more on the subject (there are several papers on symmetry in Cope's bibliography, but I need a little more time than I have). A week ago or so, I've also found (and read) another paper from Zhao in Communications of ACM, March 2008: Patterns, Symmetry, and Symmetry breaking.
Some of their concepts sound odd to me. The concept of symmetry is just fine, and I think it may help to unravel some issues in language design.
However, right now the idea that patterns are a way to break symmetry doesn't feel so good. I would say exactly the opposite, but I really have to read their more mathematically-inclined papers before I say more, because words can be misleading, while theorems usually are not :-).
Still, the inability to have built-in, natural access to fields and properties through run-time names struck me as a lack of symmetry in the language. In this sense, the Accessor would simply be a way to deal with that lack of symmetry. Therefore it seems to me that patterns are a way to bring back symmetry, not to break symmetry. In fact, I can think of many cases where patterns "expose" some semantic symmetry that was not accessible because of (merely) syntactic asymmetry.
More on this as I dig deeper :-).
Labels: article reference, design, form, language design, link
Sunday, February 17, 2008
Cognitive Dimensions of Notations
Our material is knowledge, or information. We acquire, understand, filter, and structure information; and we encode it in a variety of formats: text, diagrams, code, and so on.[...]the diagram isn't a material, just a representation. We use a tool to represent the material, which is intangible knowledge. [...] What's peculiar with software is that, in many cases, the tools and the materials have the same nature.
Note that I'm using "tool" in a very broad sense there. UML is not our material, is a tool we use to encode our material (and yeah, we may also use a software tool to draw the UML). Source code is not our material, is a tool we use to encode our material.
The fact that we often confuse tools and materials, and that we can get along with that just fine, empirically proves that they are somehow similar in nature.
Now, If our tools and materials have similar nature, then perhaps by understanding better the properties of our tools we can also understand better the properties of our materials, and ultimately of our creations. Assuming, of course, that we can somehow define and classify some interesting properties of our tools (that is, notations).
Turns out that people have been doing so for quite a while now. There is an interesting body of knowledge about the so-called Cognitive Dimensions of notations. I'll quote some portions from a short paper (Cognitive Dimensions of Notations: Design Tools for Cognitive Technology), but if you're interested, I recommend you dig deeper by reading Cognitive Dimensions of Information Artefacts: a tutorial.
When we use a notation, we are both limited and enabled by its own peculiarities. Our own style further emphasizes some of those attributes. For instance, a class diagram is relatively easy to change: you can quickly reshape your design into a very different one. In the Cognitive Dimensions vocabulary, we would say that its viscosity is low. However, if you have also modeled some scenarios on that class diagram (using for instance a sequence diagram) you have to work much harder to make changes. Sequence diagram have high viscosity.
Now, I really like the choice of viscosity as an attribute. I have to confess: I like it because it's not nerdy. When we're dealing with intangible information, it's easy to find ourselves lacking the necessary words. In software development, there is then a wide tendency to adopt some weird, geeky term.
Perhaps that's why I've immediately appreciated the Cognitive Dimensions framework. They put some considerable effort in creating the vocabulary itself. Here is a quote from the aforementioned paper: We believe that this problem is best addressed by providing a vocabulary for discussing the design problems that might arise – a vocabulary informed by research in cognitive psychology, but oriented toward the understanding of a system developer. The Cognitive Dimensions of Notations are such a vocabulary. Bingo! :-)
If you spend some time familiarizing with the vocabulary, you'll see how useful it can be to settle down some long-standing debates, like code Vs. diagrams or even static typing Vs. dynamic typing. Consider, for instance:
Premature commitment: constraints on the order of doing things.
Hidden dependencies: important links between entities are not visible.
Error-proneness: the notation invites mistakes and the system gives little protection.
and so on. They provide a sound reference against which the pros and cons of different notations can be evaluated.
The real boon (for me) is: some of those concepts can be extended from the tool to the material! Actual code may or may not have Hidden Dependencies. Actual code may have different degrees of Viscosity. Actual code may or may not have Closeness of Mapping. And so on. Of course, the notation itself is providing a bottom line on some properties. But ultimately, the form you give to your material may move quite far from the bottom line.
Indeed, I believe the form itself will be heavily influenced by the process and notation you used when you conceived the form. If you conceive your form using diagrammatic reasoning, it will keep some of the inherent properties of that notation even when it's translated into another notation (like source code). If you start with source code, your form will be more heavily influenced by the properties of source code.
I also like some of the candidate dimensions. For instance, I'm especially fond of Creative Ambiguity. This, I would say, it's one of the properties that code is lacking most. It's also something academics are trying hard to remove :-) from UML. And in a sense, it's what makes some practices like TDD so limited.
Code doesn't afford much creative ambiguity, yet what we call "modeling" should probably be called "incubation". Shaping a great form ain't easy. We need low viscosity, a good degree of creative ambiguity, high provisionality, and so on. Code alone just won't cut it.
Of course, we might not be after great form, just run-of-the-mill form. Then anything sensible is gonna work: see also my recent post on old-fashioned architectures "designed" for mass adoption.
Labels: article reference, design, form
Tuesday, December 18, 2007
Problem Frames
I found the concept interesting, and a few years later I decided to dig deeper by reading another book from Jackson (Problem Frames: Analyzing and structuring software development problems). Quite interesting, although definitely not a light reading. I cannot say it changed my life or heavily influenced the way I work while I analyze requirements, but it added a few interesting concepts to my bag of tricks.
Lately, I've found an interesting paper by Rebecca Wirfs-Brock, Paul Taylor and James Noble: Problem Frame Patterns: An Exploration of Patterns in the Problem Space. I encourage you to read the paper, even if you're not familiar with the concept of problem frame: in fact, it's probably the best introduction on the subject you can get anywhere.
In the final section (Assessment and Conclusions) the authors compare Problem Frames and Patterns. I'll quote a few lines here:
A problem frame is a template that arranges and describes
phenomena in the problem space, whereas a pattern maps forces to a solution in the solution space.
Patterns are about designing things. The fact that we put problem frames into pattern form demonstrates
that when people write specifications, they are designing too—they are designing the overall system, not its
internal structure. And while problem frames are firmly rooted in the problem space, to us they also suggest
solutions.
If you read that in light of what I've discussed in my latest post on Form, you may recognize that Problem Frames are about structuring and discovering context, while Design Patterns helps us structure a fitting form.
When Problem Frames suggests solutions, there is a good chance that they're helping us in the elusive game of (to quote Alexander again) bringing harmony between two intangibles: a form which we have not yet designed, and a context which we cannot properly describe.
Back to the concept of Problem Frames, I certainly hope that restating them in a pattern form will foster their adoption. Indeed, the paper above describes what is probably the closest thing to true Analysis Patterns, and may help analysts look more closely at the problem before jumping into use cases and describe the external behaviour of the system.
Labels: analysis, article reference, book reference, form, pattern, requirements
Sunday, December 16, 2007
On the concept of Form (2)
Alexander's definition goes well beyond the old-school, classical dichotomy between form and function, and also beyond the old-school dichotomy between functional and non-functional requirements. In fact, function intended as purpose is part of the context: functional requirements are obviously putting demands on form. Non-functional requirements are part of the context as well, and they should be considered first-class citizens too.
It is important to understand the human role in the design process. It is first an act of selection, and then an act of shaping.
We decide to shape a part of the world and leave the rest as it is. These are powerful words. I've long contended that in most case we have a much higher degree of control over requirements (context) than we're inclined to believe.
I've also contended for a very long time (mostly inspired by Tom Peters, I guess) that efficiency demands requirements (that is, context) to be developed jointly by developers and marketing.
Move away from this, and you get an inefficient context: a set of requirements (mostly functional) which may put strenuous demands on the form, without a corresponding premium in the resulting value. If you wanna win the game, you have to set the right rules.
In software development, more than in any other discipline, we can choose the right context before we step up to build our product. Of course, it is much easier to get a "fixed" set of requirements, complain about marketing lack of understanding, and start coding. But that's just not efficient. Also, keeping the context fixed while we shape the form is inefficient, as we wouldn't be profiting from newly discovered opportunities.
Building form, therefore, is an incremental process of discovery, as we don't know the exact context, and even if we do, we don't know if that's the context we wanna play with (or against, if you're the competitive kind ;-).
Indeed, Alexander reminds us that "In a real design project [...] we are searching for some kind of harmony between two intangibles: a form which we have not yet designed, and a context which we cannot properly describe" (pg. 26). It is not hard to see a deep connection with the notion of "software development as knowledge acquisition" popularized by Armour.
It is also important to understand a simple, yet deep consequence of Alexander's definition of form. As we create a product, even a software product, we are shaping its form. Even if we ignore design altogether, or even if we just concentrate on making it work, we are shaping some kind of form.
In most piecemeal development cases, that form will be determined by a small subset of forces: functional requirements, and sometimes the viscosity of previously developed software as well. The resulting structure won't be in frictionless contact with the true context, and the resulting friction will make our product brittle.
There is still a lot to be said about the subtle relationship between the above and agility, about the concept of shaping software, the [constructive] force field, and about the ubiquitous real options as well. Some more software-oriented examples are probably badly due too. And I should really say something about inventing requirements! So many things, so little time :-). Stay tuned!
Labels: design, form, requirements
Monday, October 29, 2007
On the concept of Form (1)
The point is, doing so would take too much [free] time, delaying my writings for several more weeks or even months. This is exactly the opposite of what I want to do with this blog (see my post blogging as destructuring), so I'll commit the ultimate sin :-) and talk about form in a rather unstructured way. I'll start a random sequence of posting on form, writing down what I think is relevant, but without any particular order. I won't even define the concept of form in this first instalment.
Throughout these posts, I'll borrow extensively from a very interesting (and warmly suggested to any software designer) book from Christopher Alexander. Alexander is better known for his work on patterns, which ultimately inspired software designers and created the pattern movement. However, his most releavant work on form is Notes on the Synthesis of Form. When quoting, I'll try to remember page numbers, in which case, I'll refer to the paperback edition, which is easier to find.
Alexander defines two processes through which form can be obtained: the unselfconscious and the selfconscious process. I'll get back to these two concepts, but in a few words, the unselfconscious process is the way some ancient cultures proceeded, without an explicit set of principles, but relying instead on rigid tradition mediated by immediate, small scale adaptation upon failure. It's more complex than that, but let's keep it simple right now.
Tradition provides viscosity. Without tradition, and without explicit design principles, the byproducts of the unselfconscious process will quickly degenerate. However, merely repeating a form won't keep up with a changing environment. Yet change happens, maybe shortly after the form has been built. Here is where we need immediate action, correcting any failure using the materials at hand. Of course, small scale changes are sustainable only when the rate of change (or rate of failure) is slow.
Drawing parallels to software is easy, although subjective. Think of quick, continuous, small scale adaptation. The immediate software counterpart is, very naturally, refactoring. As soon as a bad smell emerge, you fix it. Refactoring is usually small scale, using the "materials at hand" (which I could roughly translate into "changing only a small fraction of code"). Refactoring, by definition, does not change the function of code. Therefore, it can only change its form.
Now, although some people in the XP/agile camp might disagree, refactoring is a viable solution only when the desired rate of change is slow, and only when the gap to fill is small. In other words, only when the overall architecture (or plain structure) is not challenged: maybe it's dictated by the J2EE way of doing things, or by the Company One True Way of doing things, or by the Model View Controller police, and so on. Truth is, without an overall architecture resisting change, a neverending sequence of small-scale refactoring may even have a negative large-scale impact.
I've recently said that we can't reasonably turn an old-style, client-server application into a modern web application by applying a sequence of small-scale changes. It would be, if not unfeasible, hardly economic, and the final architecture might be far from optimal. The gap is too big. We're expecting to complete a big change in a comparatively short time, hence the rate of change is too big. The viscosity of the previous solution will fight that change and prevent it from happening. We need to apply change at an higher granularity level, as the dynamics in the small are not the dynamics in the large.
Curiously enough (or maybe not :-), I'll be talking about refactoring the next two days. As usual, I'll try to strike a balance, and get often back to good design princples. After all, as we'll see, when the rate of change grows, and/or when the solution space grows, the unselfconscious process must be replaced by the selfconscious process.
Labels: book reference, design, form, profession
