Sunday, May 27, 2007
Metaprogramming, OOP, AOP (Part 2)
Assuming we're still free to modify the existing library classes (I'll get back to this later), we could easily approach the problem as follows:
- identify a root class, e.g. Control, and add an abstract Localize responsibility.
- identify a derived class (let's call it ContainerControl), that is, a control which is mainly used as a container for other controls. Here Localize could provide a default implementation, which is basically to iterate over all the inner controls and forward the Localize call.
- make sure Localize is called at the appropriate time, e.g. before the control is drawn, or when the current language changes. There are several performance issues here, quite interesting in practice, but largely irrelevant to the discussion.
- Localize must have access to some kind of Dictionary class which keeps track of translations for different languages. For sake of simplicity, we could assume Dictionary to be a singleton, so that we don't have to worry about passing parameters around.
In a sense, that could pretty much be it. Each concrete control (e.g. Label, Button, ListView, etc) will implement its own localization logic. Label and Button will get their own text from the dictionary; ListView may forward the call to each ColumnHeader (to keep a design similar to the .NET version). And so on.
There are obviously a few issues with this simple (almost simplistic) design:
- there is no much reuse between controls; Label and Button will implement Localize basically in the same way, in two different places. We could fix this by implementing the common part in Control, but still we're not pushing reuse much further.
- controls have no idea of their context (that is, the meaningful nesting of ContainerControls from their parent up to the main window) during localization. This could be useful to share localization strings and/or to localize strings while considering context. However, this can be fixed in various ways, e.g. by pushing contextual information from a ContainerControl to contained controls, in the Localize call, or by walking up to the parent, and so on.
- there is no separation of concerns. Actually, localization has just become a cross-cutting concern, which is being implemented in many different classes (although all of them belong to the same hierarchy).
- (this is by far the worst problem) if you don't own the type system, and the owner didn't put in the Localize responsibility, you're out of luck, unless your language allows you to add polymorphic behaviour to an existing hierarchy.
Let's play with the last issue for a while. An unfortunate consequence of OO thinking is that it's always hard to consider a class "complete". For instance, your favorite String class may not have that regular-expression-based Split function you love so much. In most cases, you can put that function into another class, usually with a sad name like StringUtilities :-), and live with that. If your language allows extension methods a-la LINQ, you can add methods to existing classes without breaking the natural syntax for method invocation. However, you need true open classes if you want to add polymorphic behaviour to an existing hierarchy. Extension methods won't cut it. Mixin, or mixin layers, could be used to some advantage.
Having said that, is there an OO way to deal with all the problems above, while staying within the realm of a "restrictive" language, without open classes, and without having to change the library source code?
Turns out you can push some responsibility outside the control and inside a newborn class. In fact, if you look at the implementation of your average control, you might find that it has to keep the unlocalized text somewhere (e.g. in a string that you can also set at design time). That string is then passed to the dictionary as a key at run-time (possibly with some context information) to get a translation in the current language. Now, if you've internalized OO thinking, you'll recognize that the control is managing the strings. That's because the string doesn't know any better. Now, suppose you change all instances of string inside controls (all those which requires localization anyway) with a smarter string, let's say LocalizedString. A LocalizedString would store the unlocalized version but return the localized version when you read it. It will take care of Dictionary lookup, caching, watching for changes in the current language, and so on, all in a single place. No more duplications. Better separation of concerns. The programmer will declaratively say (by using string or LocalizedString) if he/she needs a language-independent or language-dependent string. That would be it.
Again, there are a few issues with this design:
- it's quite harder for the string to find out about its context. While controls can usually navigate to their parent (another Control), it would be quite hard for a string to navigate to whatever object is storing the string. While a Control may have a Localize method which takes parameter, our LocalizedString should be able to return the localized text through a parameterless read (ideally, a conversion operator, so that we can blissfully ignore the whole localization issue).
We could approach this problem from a few angles, and for a simple problem like string localization, we could even succeed. Unfortunately, the technique doesn't work so well for other problems, like persistence. You may resort to reflection, but I'll get to this later.
- just like before, if you don't own the code (or can afford to change it anyway) you're in big troubles: adopting this design is a pervasive change inside the whole Control hierarchy. What is worse, open classes won't help you a bit.
What we're witnessing is a major issue with OOP. Thinking in objects is like creating a foundation. If you get it wrong, you're done. If it's your own code, this might not be a big deal - refactoring is always possible (not necessarily cheap, but possible). But if you're dealing with third-party code, you often suffer from an impedance mismatch between their design and your needs. I do not consider open source as a cure - even if (e.g.) the .NET framework was open source, I would not venture in any massive change, because that's maintenance hell.
Given this major issue with OOP, I find it rather myopic that some features, like open classes and structural conformance to interfaces, are not more widely adopted by language designers. For languages like Java and C#, it's also rather myopic that you can't declaratively say that whenever an object of type T1 is requested, an object of type T2 (usually derived from T1) must be created instead. Not building these features in an OOP language means ignoring real problems, which could be solved without compromising type safety or efficiency. There are also real-world languages (like Objective C) which have shown how to deal with some of these issues in practice. Too bad we, as a community, don't seem to learn enough from the past :-)
Back to our problem: so far, we tried to allocate behaviour inside the existing principal decomposition, or to push it inside a finer-grained class (which should then be used pervasively inside the principal decomposition). Is there a third approach? I didn't play the interface card yet, but that seems quite at odd with the problem. Coming up with a useful ILocalizable (bleah :-) interface, which does not require pervasive, cross-cutting changes inside the principal decomposition to be implemented, seems quite hard to say the least.
There is, of course, the reflective/introspective card yet to play. However, this brings us so close to the AOP way of thinking, that I'll discuss this option while looking at my toy problem from the AOP perspective.
Labels: design, language design, oop
Wednesday, May 16, 2007
Metaprogramming, OOP, AOP (Part 1)
In a previous post, I introduced the concept of [Layered] Virtual Machines a a useful metaphor for aspect oriented design. I've further elaborated the concept in a later post, but I understand it's slightly elusive :-), so a realistic example could be beneficial.
In this post, and in the following 2, I'll take a look at the same problem under the metaprogramming, OOP, and AOP perspective. While doing so, my aim won't be to evaluate the respective merits of each approach. Right now, I'm more concerned about how any given approach will influence the reasoning process. Also, for AOP, I'd like to show how the Virtual Machine metaphor could be useful to think at the appropriate abstraction level.
I've chosen a problem that I hope will be familiar to many programmers: localization. To keep the discussion focused, I've further restricted the problem to GUI localization, and within that, only to text localization (leaving aside, for instance, bitmaps that may need to be localized as well).
Metaprogramming
As Gregor Kiczales said, The key property that all metaprograms share is that they explicitly manipulate other programs or program representations. Examples of metaprogramming include a code-generating IDE, so let's take a look at how Visual Studio 2005 is dealing with localization in .NET (Windows Form).
When you add controls to a Windows Form application, Visual Studio generates code (that's, of course, metaprogramming). If you add (for instance), a button, a listbox with a few preloaded values, and a listview with a few column headers, you'll get code like:
this.button1 = new System.Windows.Forms.Button();
this.listBox1 = new System.Windows.Forms.ListBox();
this.listView1 = new System.Windows.Forms.ListView();
this.columnHeader1 = new System.Windows.Forms.ColumnHeader();
this.columnHeader2 = new System.Windows.Forms.ColumnHeader();
this.columnHeader3 = new System.Windows.Forms.ColumnHeader();
//[...]
// button1
this.button1.Text = "button1";
// listBox1
this.listBox1.Items.AddRange( new object[] {
"value1",
"value2",
"value3"} );
// column headers
this.columnHeader1.Text = "Column1";
this.columnHeader2.Text = "Column2";
this.columnHeader3.Text = "Column3";
String literals are placed directly into the code. That makes the code itself fast and lean, but won't help much with localization. In a while, we'll take a look at how the code changes if you set the Localizable property to true in the container (usually the form). The code changes, because in Visual Studio / Windows Forms, localization is being dealt with at the metaprogramming level. Is that a sound approach? As I said, that's out of scope, but to keep a long story short, metaprogramming at the IDE level has some nice properties: most notably, very high integration with the development tool itself. Assuming, of course, that you consider localization as an integral part of software development (I do not necessarily agree with that).
Leaving the soundness question aside, if you're a designer thiking about providing metaprogramming-based support for localization, you'll have to look into a few common metaprogramming issues:
- the programmer may inadvertendly tamper with the generated code. Microsoft answer was to introduce partial classes in .NET, which are obviously just a partial solution, but not that bad anyway.
- the programmer may have problems debugging some (possibly) obscure, machine-generated code. This was certainly an issue for ugly macro-based, forward-only code generators like the old style ATL COM wizards. For the relatively simple job of static GUI creation, the issue is rather moot.
- extensibility: what if the programmer rolls his/her own GUI controls? How will they participate in code generation, or ultimately, in string localization?
The latest issue is probably the single most important problem affecting many metaprogramming tools. When we design a metaprogramming environment, we're basically building a code generator. The input to the code generator is a language in itself, possibily a visual language. In this case, the input is the (visual) selection of a control from a palette of tools, plus a set of properties.
In the most naive approach, the code generator needs to "know" the GUI, that is, each control and its localizable properties. It needs to know that a button has a Text property that must be localized. It needs to know that a listbox may have design-time values that must be localized. It needs to know that a listview may have headers, and that they in turn have a text, which must be localized.
Under the naive approach, the designer of the metaprogramming environment is thinking about a closed world: the set of supported controls is fixed, because knowledge about the localization of each control is built inside the IDE itself. Needless to say, that's quite easy to do, but very limiting.
Once you open up to user-defined controls, you can basically follow one of the following approaches:
- The programmatic approach. Here the component takes part in code generation. Each component with localizable properties will have to implement some "code generation" interface, defined in what can be considered a metaprogramming framework. The IDE will call the appropriate functions, but the component will ultimately be responsible for code generation. This approach is very flexible, but uncomfortable. There is also the nontrivial issue of having code generated in a specific, high-level language (like C#; but what if I'm using the component in a VB.NET form?) or directly into a neutral, platform defined, low-level language (like MSIL).
- the declarative approach. Here the component will "tell" the metaprogram (through some mechanism yet to be defined) which properties need to be localized. The metaprogram will then work uniformously over all those properties, perhaps guided by type (e.g. a string Vs. a collection of strings). If you're familiar with .NET, attributes may come to mind as a way to "tell" the metaprogram about localizable properties.
Let's take a look at how Visual Studio 2005 deals with localized strings. Here is the former example, with localization turned on:
this.button1 = new System.Windows.Forms.Button();
this.listBox1 = new System.Windows.Forms.ListBox();
this.listView1 = new System.Windows.Forms.ListView();
this.columnHeader1 = new System.Windows.Forms.ColumnHeader();
this.columnHeader2 = new System.Windows.Forms.ColumnHeader();
this.columnHeader3 = new System.Windows.Forms.ColumnHeader();
// button1
resources.ApplyResources( this.button1, "button1" );
// listBox1
this.listBox1.Items.AddRange( new object[] {
resources.GetString("listBox1.Items"),
resources.GetString("listBox1.Items1"),
resources.GetString("listBox1.Items2")} );
resources.ApplyResources( this.listBox1, "listBox1" );
// listView1
resources.ApplyResources( this.listView1, "listView1" );
// column headers
resources.ApplyResources( this.columnHeader1, "columnHeader1" );
resources.ApplyResources( this.columnHeader2, "columnHeader2" );
resources.ApplyResources( this.columnHeader3, "columnHeader3" );
The underlying idea is a mixture of reflection and metaprogramming. Metaprogramming is applied as, for each control, a call to ApplyResources is generated. Also, through metaprogramming, strings are not stored inside code anymore. Without going into much details, strings will be placed in resource-only DLLs (known as satellite assemblies), one for each language. Note that the choice of storing the strings into DLLs has nothing to do with the metaprogramming approach - they could have chosen to store strings into an XML file (I would have appreciated that).
Reflection is then applied inside a unique (uniform) ApplyResources method of ComponentResourceManager . This method looks for properties tagged with the Localizable attribute, then gets the localized text from the satellite assembly. If you want to find out more, you can read Improvements to Localization in Visual Studio 2005 and Localization in Whidbey. The latter describes also the original localization model for Visual Studio, which was based on design-time reflection, not on run-time reflection.
That's not the full story, however: note how the Items collection is being directly localized into the generated code, not through reflection. This seems more like a quirk, so let's skip that for now. Also, notice how ApplyResources is needlessy called on the listview as well, although there is nothing to be localized - only the headers really need to be localized. Again, this seems more like a small quirk, something that could be easily fixed.
At the end of the day, the declarative approach to metaprogramming extensibility is quite handy: you only have to tag your localizable properties with the right attribute, and the metaprogram will do the rest (supported by the run-time framework, especially under the run-time reflection model).
As usual, you can only get that far with a declarative model. Here, the need to localize a finite set of types (like string) is a huge simplification. Again, a good designer will look for the right balance of extensibility and rigidity, possibly providing alternative ways to deal with the most complex situations.
Just a final comment: there is a common problem with the adoption of a metaprogramming (or AOP with static weavers) approach to -ilities. Since the code dealing with the -ility (like "localizability") is either generated or woven as part of the mainstream code, the application will always be "bloated" by the ility, even if you don't need it.
Consider an application that will be used 90% of the cases in English-speaking countries, 10% in others. You want to provide localization, but ideally, you shouldn't pay for it when not needed (90% of the cases). In fact, a popular paper from David Parnas was "Designing software for ease of extension and contraction". Somehow, most people got the "extension" part, but just a few got the "contraction" part. Moore's law did the rest :-).
Next time, I'll take a look at how we could approach the same problem under a strictly object-oriented approach - no metaprogramming allowed. Then we'll move to AOP and the Virtual Machine metaphor.
In this post, and in the following 2, I'll take a look at the same problem under the metaprogramming, OOP, and AOP perspective. While doing so, my aim won't be to evaluate the respective merits of each approach. Right now, I'm more concerned about how any given approach will influence the reasoning process. Also, for AOP, I'd like to show how the Virtual Machine metaphor could be useful to think at the appropriate abstraction level.
I've chosen a problem that I hope will be familiar to many programmers: localization. To keep the discussion focused, I've further restricted the problem to GUI localization, and within that, only to text localization (leaving aside, for instance, bitmaps that may need to be localized as well).
Metaprogramming
As Gregor Kiczales said, The key property that all metaprograms share is that they explicitly manipulate other programs or program representations. Examples of metaprogramming include a code-generating IDE, so let's take a look at how Visual Studio 2005 is dealing with localization in .NET (Windows Form).
When you add controls to a Windows Form application, Visual Studio generates code (that's, of course, metaprogramming). If you add (for instance), a button, a listbox with a few preloaded values, and a listview with a few column headers, you'll get code like:
this.button1 = new System.Windows.Forms.Button();
this.listBox1 = new System.Windows.Forms.ListBox();
this.listView1 = new System.Windows.Forms.ListView();
this.columnHeader1 = new System.Windows.Forms.ColumnHeader();
this.columnHeader2 = new System.Windows.Forms.ColumnHeader();
this.columnHeader3 = new System.Windows.Forms.ColumnHeader();
//[...]
// button1
this.button1.Text = "button1";
// listBox1
this.listBox1.Items.AddRange( new object[] {
"value1",
"value2",
"value3"} );
// column headers
this.columnHeader1.Text = "Column1";
this.columnHeader2.Text = "Column2";
this.columnHeader3.Text = "Column3";
String literals are placed directly into the code. That makes the code itself fast and lean, but won't help much with localization. In a while, we'll take a look at how the code changes if you set the Localizable property to true in the container (usually the form). The code changes, because in Visual Studio / Windows Forms, localization is being dealt with at the metaprogramming level. Is that a sound approach? As I said, that's out of scope, but to keep a long story short, metaprogramming at the IDE level has some nice properties: most notably, very high integration with the development tool itself. Assuming, of course, that you consider localization as an integral part of software development (I do not necessarily agree with that).
Leaving the soundness question aside, if you're a designer thiking about providing metaprogramming-based support for localization, you'll have to look into a few common metaprogramming issues:
- the programmer may inadvertendly tamper with the generated code. Microsoft answer was to introduce partial classes in .NET, which are obviously just a partial solution, but not that bad anyway.
- the programmer may have problems debugging some (possibly) obscure, machine-generated code. This was certainly an issue for ugly macro-based, forward-only code generators like the old style ATL COM wizards. For the relatively simple job of static GUI creation, the issue is rather moot.
- extensibility: what if the programmer rolls his/her own GUI controls? How will they participate in code generation, or ultimately, in string localization?
The latest issue is probably the single most important problem affecting many metaprogramming tools. When we design a metaprogramming environment, we're basically building a code generator. The input to the code generator is a language in itself, possibily a visual language. In this case, the input is the (visual) selection of a control from a palette of tools, plus a set of properties.
In the most naive approach, the code generator needs to "know" the GUI, that is, each control and its localizable properties. It needs to know that a button has a Text property that must be localized. It needs to know that a listbox may have design-time values that must be localized. It needs to know that a listview may have headers, and that they in turn have a text, which must be localized.
Under the naive approach, the designer of the metaprogramming environment is thinking about a closed world: the set of supported controls is fixed, because knowledge about the localization of each control is built inside the IDE itself. Needless to say, that's quite easy to do, but very limiting.
Once you open up to user-defined controls, you can basically follow one of the following approaches:
- The programmatic approach. Here the component takes part in code generation. Each component with localizable properties will have to implement some "code generation" interface, defined in what can be considered a metaprogramming framework. The IDE will call the appropriate functions, but the component will ultimately be responsible for code generation. This approach is very flexible, but uncomfortable. There is also the nontrivial issue of having code generated in a specific, high-level language (like C#; but what if I'm using the component in a VB.NET form?) or directly into a neutral, platform defined, low-level language (like MSIL).
- the declarative approach. Here the component will "tell" the metaprogram (through some mechanism yet to be defined) which properties need to be localized. The metaprogram will then work uniformously over all those properties, perhaps guided by type (e.g. a string Vs. a collection of strings). If you're familiar with .NET, attributes may come to mind as a way to "tell" the metaprogram about localizable properties.
Let's take a look at how Visual Studio 2005 deals with localized strings. Here is the former example, with localization turned on:
this.button1 = new System.Windows.Forms.Button();
this.listBox1 = new System.Windows.Forms.ListBox();
this.listView1 = new System.Windows.Forms.ListView();
this.columnHeader1 = new System.Windows.Forms.ColumnHeader();
this.columnHeader2 = new System.Windows.Forms.ColumnHeader();
this.columnHeader3 = new System.Windows.Forms.ColumnHeader();
// button1
resources.ApplyResources( this.button1, "button1" );
// listBox1
this.listBox1.Items.AddRange( new object[] {
resources.GetString("listBox1.Items"),
resources.GetString("listBox1.Items1"),
resources.GetString("listBox1.Items2")} );
resources.ApplyResources( this.listBox1, "listBox1" );
// listView1
resources.ApplyResources( this.listView1, "listView1" );
// column headers
resources.ApplyResources( this.columnHeader1, "columnHeader1" );
resources.ApplyResources( this.columnHeader2, "columnHeader2" );
resources.ApplyResources( this.columnHeader3, "columnHeader3" );
The underlying idea is a mixture of reflection and metaprogramming. Metaprogramming is applied as, for each control, a call to ApplyResources is generated. Also, through metaprogramming, strings are not stored inside code anymore. Without going into much details, strings will be placed in resource-only DLLs (known as satellite assemblies), one for each language. Note that the choice of storing the strings into DLLs has nothing to do with the metaprogramming approach - they could have chosen to store strings into an XML file (I would have appreciated that).
Reflection is then applied inside a unique (uniform) ApplyResources method of ComponentResourceManager . This method looks for properties tagged with the Localizable attribute, then gets the localized text from the satellite assembly. If you want to find out more, you can read Improvements to Localization in Visual Studio 2005 and Localization in Whidbey. The latter describes also the original localization model for Visual Studio, which was based on design-time reflection, not on run-time reflection.
That's not the full story, however: note how the Items collection is being directly localized into the generated code, not through reflection. This seems more like a quirk, so let's skip that for now. Also, notice how ApplyResources is needlessy called on the listview as well, although there is nothing to be localized - only the headers really need to be localized. Again, this seems more like a small quirk, something that could be easily fixed.
At the end of the day, the declarative approach to metaprogramming extensibility is quite handy: you only have to tag your localizable properties with the right attribute, and the metaprogram will do the rest (supported by the run-time framework, especially under the run-time reflection model).
As usual, you can only get that far with a declarative model. Here, the need to localize a finite set of types (like string) is a huge simplification. Again, a good designer will look for the right balance of extensibility and rigidity, possibly providing alternative ways to deal with the most complex situations.
Just a final comment: there is a common problem with the adoption of a metaprogramming (or AOP with static weavers) approach to -ilities. Since the code dealing with the -ility (like "localizability") is either generated or woven as part of the mainstream code, the application will always be "bloated" by the ility, even if you don't need it.
Consider an application that will be used 90% of the cases in English-speaking countries, 10% in others. You want to provide localization, but ideally, you shouldn't pay for it when not needed (90% of the cases). In fact, a popular paper from David Parnas was "Designing software for ease of extension and contraction". Somehow, most people got the "extension" part, but just a few got the "contraction" part. Moore's law did the rest :-).
Next time, I'll take a look at how we could approach the same problem under a strictly object-oriented approach - no metaprogramming allowed. Then we'll move to AOP and the Virtual Machine metaphor.
