Carlo Pescio - blog

"Out of clutter find simplicity;
From discord find harmony;
In the middle of difficulty lies opportunity."
Albert Einstein

Note: I was teaching requirements engineering today, and I spent more time than usual talking about viewpoints, task descriptions, personas, the need for the analyst to invent requirements and so on.
For some reason, while walking to the hotel those words of wisdom came back to my mind. Indeed, those principles apply equally well to analysis, design, or coding, and in a sense, they are at the core of quality software development.
The pursuit of simplicity and harmony is a recurring theme in Einstein's writings, some of which I've quoted before .

Labels: profession, quote

I've been talking about code clones before. It's a simple metric that I've used in several projects with encouraging results.

Till no long ago, however, I thought code clones detection was useful mostly to:

1) Assess and monitor an interesting quality aspect of a product
This requires that we constantly monitor code clones. If some code already exists, we can create a baseline and enforce a rule that things can only get better, not worse. I usually monitor several internal quality attributes at build time, because that's a fairly flexible moment, where most tools allow to insert some custom steps.

2) Identify candidates for refactoring, mostly in large, pre-existing projects.
This requires, of course, a certain willingness to act on your knowledge, that is, to actually go ahead and refactor duplicated code.

Sometimes, when the codebase is large, resources are scarce, or the company interest in software quality is mostly a marketing statement disconnected from reality, a commitment to refactor the code is never taken, or never taken seriously, which is about the same.

Here comes the third use of code clones. It is quite obvious, and I should have considered it earlier, but for some reason I didn't. I guess I was somehow blinded by the idea that if you care about quality, you must get in there and refactor the damn code. Strong beliefs are always detrimental to creativity :-).

Now: clones are bad because (in most cases) you have to keep them in synch during maintenance. If you don't, something bad is gonna happen (and yes, if you do, you waste a lot of time anyway, so you could as well refactor; but this is that strong belief rearing its head again :-).
So, if you don't want to use a code clones list to start a refactoring campaign, what else can you do? Use it to make sure you didn't forget to update a clone!

Unfortunately, with the tools I know, a large part of this process can't be easily automated. You would have to run a clone detection tool and keep the log somewhere. Then, whenever you change some portion of code, you'll have to check if that portion is cloned elsewhere (from the log). You then port your change in the other clones (and test everything). The clones list must be periodically updated, also to account for changes coming from different programmers.

Better tools can be easily conceived. Ideally, this could be integrated in your IDE: as I suggested in Listen to Your Tools and Materials, editors could provide unobtrusive backtalk, highlighting the fact that you're changing a portion of code that has been cloned elsewhere. From there, you could jump into the other files, or ask the editor to apply the same change automatically. In the end, that would make clones more tolerable; while this is arguably bad, it's still much better than leave them out of synch.

From that perspective, I would say that another interesting place in our toolchain where we would benefit from an open, customizable process is the version control system. Ideally, we may want to verify and enforce rules right at check-in time, without the need to delay checks until build time. Open source tools are an obvious opportunity to create a better breed of version control systems, which so far (leaving a few religious issues aside) have been more or less leveled in term of available features.

Note: I've been writing this post on a EEE PC (the Linux version), and I kinda like it. Although I'm not really into tech toys, and although the EEE looks and feels :-) like a toy, it's just great to carry around while traveling. The tiny keyboard is a little awkward to use, but I'll get used to it...

Labels: article reference, metrics, profession, tools

Some (most :-) of my clients are challenging. Sometimes the challenge comes from the difficult technical problems they face. That's the best kind of challenge.
Sometimes the challenge comes from people: that's the worst kind of challenge, and one that right now is better left alone.
Sometimes the challenge comes from the organization, which means it also comes from people, but with a different twist. Challenges coming from the organization are always tough, but overcoming those challenges can really make a difference.

One of my challenging clients is a rather large company in the financial domain. They are definitely old-school, and although upper management can perfectly see how software is permeating and enabling their business, middle management tend to see software as a liability. In their eternal search for lower costs, they moved most of the development offshore, keeping only an handful of designers and all the analysts in-house. Most often, design is done offshore as well, for lack of available designers on this side of the world.

Analysts have a tough job there. On one side, they have to face the rest of the company, which is not software-friendly. On the other side, they have to communicate clear requirements to the offshore team, especially to the designers, who tend to be very technology-oriented.
To make things more complicated, the analysts often find themselves working on unfamiliar sub-domains, with precise regulations but also with large gray areas that must be somehow understood and communicated.
Icing on the cake: some of those financial instruments do not even exist in the local culture of the offshore team, making communication as difficult as ever.

Given this overall picture, I've often recommended analysts to spend some time creating a good domain model (usually, a UML class diagram, occasionally complemented by some activity diagrams).
The model, with unambiguous associations, dependencies, multiplicities, and so on, will force them to ask the right questions, and will make it easier for the offshore designer to acquaint himself with the problem. Over time, this suggestion has been quite helpful.
However, as I said, the organization is challenging. Some of the analysts complained that their boss is not satisfied by a few diagrams. He wants a lengthy, wordy explanation, so he can read it over and see if they got it right (well, that's his theory anyway). The poor analyst can't possibly do everything in the allotted time.

Now, I always keep an eye on software engineering research. I've seen countless attempts to create UML diagrams from natural language specifications. The results are usually unimpressive.
In this case, however, I would need exactly the opposite: a tool to generate a precise, yet verbose domain description out of a formal domain model. The problem is much easier to solve, especially because analysts can help the tool, by using the appropriate wording.

Guess what, the problem must be considered unworthy, because there is a dearth of works in that area. In practice, the only relevant paper I've been able to find is Generating Natural Language specifications from UML class diagrams by Farid Meziane, Nikos Athanasakis and Sophia Ananiadou. There is also Nikos' thesis online, with a few more details.
The downside is that (as usual) the tool they describe does not seem to be generally available. I've yet to contact the authors: I just hope it doesn't turn out to be one of those Re$earch Tool$ that never get to be used.

From the paper above, I've also learnt about ModelExplainer , a similar tool from a commercial company. Again, the tool doesn't seem to be generally available, but I'll get in touch with people there and see.

Overall, the problem doesn't seem so hard, especially if we accept the idea that the analyst will help the tool, choosing appropriate wording. An XMI-to-NL (Natural Language) would make for a perfect open source project. Any takers? :-)

Labels: analysis, article reference, profession, requirements, tools

In many software projects lies some dreaded piece of code. Legacy code nobody wants to touch, dealing with a problem nobody wants to look at. In the past few days, I've been involved in a (mostly unpleasant :-) meeting, at the end of which I learnt that some "new" code was actually dealing with a dreaded problem by calling into legacy code. Dealing with that problem inside the "new" code was deemed risky, dirty, hard work.

Now, as I mentioned in my Work Smarter, Not Harder post, I don't believe in hard work. I believe in smart work.

After thinking for a while about the problem, I formulated an alternative solution. It didn't come out of nowhere: I could see the problem frame, and that was a language compiler/interpreter frame, although most of the participants didn't agree when I said that.
They had to agree, however, that my proposed solution was simple, effective, risk-free, and could be actually implemented in a few days of light :-) work. Which I take as an implicit confirmation that the problem frame was right.
I also had to mix some creativity, but not too much. So I guess the dreaded problem can now be solved by doing much less than expected.

This could explain the title of this post, but actually, it doesn't. More than 20 years ago, good ol' Gerald Weinberg wrote:
Question: How do you tell an old consultant from a new consultant?
Answer: The new consultant complains, "I need more business." The old consultant complains, "I need more time."

I guess I'm getting old :-), and I should probably take Seth Godin's advice. Seth is best known for "Purple Cow", a wonderful book that ought to be a mandatory reading for everyone involved in creating a new product. The advice I'm thinking of, though, comes from Do Less, a short free pamphlet he wrote a few years ago. There he writes:
What would happen if you fired half of your clients?

Well, the last few days really got me thinking :-)).

Labels: book reference, link, profession

posted by Carlo.Pescio @ 6:11 PM <permalink>  2 comments
bookmarking:    

I postponed writing on Form (vs. Function) for a while now, mostly because I was struggling to impose a rational structure on my thoughts. Form is a complex subject, barely discussed in the software design literature, with a few exception I'll quote when relevant, so I'm sort of charting new waters here.

The point is, doing so would take too much [free] time, delaying my writings for several more weeks or even months. This is exactly the opposite of what I want to do with this blog (see my post blogging as destructuring), so I'll commit the ultimate sin :-) and talk about form in a rather unstructured way. I'll start a random sequence of posting on form, writing down what I think is relevant, but without any particular order. I won't even define the concept of form in this first instalment.

Throughout these posts, I'll borrow extensively from a very interesting (and warmly suggested to any software designer) book from Christopher Alexander. Alexander is better known for his work on patterns, which ultimately inspired software designers and created the pattern movement. However, his most releavant work on form is Notes on the Synthesis of Form. When quoting, I'll try to remember page numbers, in which case, I'll refer to the paperback edition, which is easier to find.

Alexander defines two processes through which form can be obtained: the unselfconscious and the selfconscious process. I'll get back to these two concepts, but in a few words, the unselfconscious process is the way some ancient cultures proceeded, without an explicit set of principles, but relying instead on rigid tradition mediated by immediate, small scale adaptation upon failure. It's more complex than that, but let's keep it simple right now.

Tradition provides viscosity. Without tradition, and without explicit design principles, the byproducts of the unselfconscious process will quickly degenerate. However, merely repeating a form won't keep up with a changing environment. Yet change happens, maybe shortly after the form has been built. Here is where we need immediate action, correcting any failure using the materials at hand. Of course, small scale changes are sustainable only when the rate of change (or rate of failure) is slow.

Drawing parallels to software is easy, although subjective. Think of quick, continuous, small scale adaptation. The immediate software counterpart is, very naturally, refactoring. As soon as a bad smell emerge, you fix it. Refactoring is usually small scale, using the "materials at hand" (which I could roughly translate into "changing only a small fraction of code"). Refactoring, by definition, does not change the function of code. Therefore, it can only change its form.

Now, although some people in the XP/agile camp might disagree, refactoring is a viable solution only when the desired rate of change is slow, and only when the gap to fill is small. In other words, only when the overall architecture (or plain structure) is not challenged: maybe it's dictated by the J2EE way of doing things, or by the Company One True Way of doing things, or by the Model View Controller police, and so on. Truth is, without an overall architecture resisting change, a neverending sequence of small-scale refactoring may even have a negative large-scale impact.

I've recently said that we can't reasonably turn an old-style, client-server application into a modern web application by applying a sequence of small-scale changes. It would be, if not unfeasible, hardly economic, and the final architecture might be far from optimal. The gap is too big. We're expecting to complete a big change in a comparatively short time, hence the rate of change is too big. The viscosity of the previous solution will fight that change and prevent it from happening. We need to apply change at an higher granularity level, as the dynamics in the small are not the dynamics in the large.

Curiously enough (or maybe not :-), I'll be talking about refactoring the next two days. As usual, I'll try to strike a balance, and get often back to good design princples. After all, as we'll see, when the rate of change grows, and/or when the solution space grows, the unselfconscious process must be replaced by the selfconscious process.

Labels: book reference, design, form, profession

posted by Carlo.Pescio @ 7:25 PM <permalink>  8 comments
bookmarking:    

I've been conspicuously absent from my blog in the last two weeks. Hectic life takes its toll :-), and it's not always possible to talk here about what I'm actually doing.
I'm often involved in very different activities, from weird bugs at the hardware/software interface, to coding tricky parts, to designing [small] application-specific frameworks, to making sense of nonsensical requirements. Recently, however, I've been helping a few customers make some rather fundamental decision about how to evolve (or rewrite) their applications.

Indeed, a significant subset of existing applications have been developed with now-obsolete technologies ("old" languages, framework, components, architectures), or even obsolete business concepts (e.g. a client-side application sold as a product, instead of a web application sold as a service).

Now, when you're planning the next generation of a successful application, you often end up trapped into a very natural, logical, linear thinking: we start from the core of the application, build a new/better/modern one, then we keep adding features till we have a complete application.
Sometimes, it's not even about starting at the core: you start with a large framework, and expect that you'll be able to build your applications in a snap (when the humongous framework is completed).

Now, this seems very logical, and if you look at it from a technology perspective, it makes a lot of sense. By dealing with the core first, you have the best chance to make a huge impact into the foundations, making them so much better. All those years on the market taught you a lot, so you know how to make the core better.
It's also easier this way: the new application will be entirely based on the new technology, from the bottom up. No need to mix old and new stuff.

As usual, a successful development strategy is completely context dependent! If your application is small, the natural strategy above is also the best overall strategy. In a relatively short time (since the application is small) you'll have a clean, brand-new application.
Unfortunately, the strategy does not scale at all to large applications. Let's see a few (well known, and often ignored) problems:

- The value delivery curve is pretty flat. Looks more like fig.1 in an old post of mine, as you can't really sell the new application till is finished.
People usually argue that by choosing the "right" core, they can start selling the core before the whole application has been ported. Yeah, sure, maybe in some alternative reality, but truth is, in many cases your existing customers won't downgrade to a less powerful, usually incompatible application (unless they're having major headaches from the old app).
New prospects won't be so exhilarated by a stripped-down core, either. Over the years, for a number of reasons, it' very likely that most innovation happened at the fringes of the old application, not at the core. You're now taking this stuff away for a potentially long time. Your old product will be competing with your new product, and guess what, lack of features tends to be rather visible.

- Although management may seem initially inclined to support your decision to rebuild everything from scratch, they won't stay silent as the market erodes. Soon they will (very reasonably) ask you to add some stuff to the existing application as well.
Now, by doing so, you'll slow down the development of the new application (resource contention) and you'll also create a backlog of features to be ported to the new application, once it is finished.

- All this conjures for very long development times. If you're working with intrinsically unstable technologies (like web applications), there is even a significant chance that your new application will be technologically obsolete before it gets to the market!

Let's try to model these (and a few more) issues using a diagram of effects



You may want to spend a little time looking at the different arrows, and especially at self-sustaining feedback loops. It's not hard to see that this is a recipe for having hard times. Yet companies routinely embark into this, because redoing the core is the most logical, most technologically sound thing to do. Unfortunately, it doesn't always make business sense
As usual, there are always other choices. As usual, they have their share of problems, and often requires better project management practices and skilled software designers. They also happen to be very context-dependent, as you have to find a better balance between your business, your current application, your new application.
Ok, time and space are up :-), I'll add a few details later.

Labels: link, profession, project management

posted by Carlo.Pescio @ 3:20 PM <permalink>  4 comments
bookmarking:    

Multicore CPU are now widely available, yet many applications are not tapping into their true potential. Sure, web applications, and more generally container-based applications have an inherent degree of coarse parallelism (basically at the request level), and they will scale fairly well on new CPU. However, most client-side applications don't fall in the same pattern. Also, some server-side applications (like batch processing) are not intrinsically parallel as well. Or maybe they are?

A few months ago, I was consulting on the design of the next generation of a (server-side) banking application. One of the modules was a batch processor, basically importing huge files into a database. For several reasons (file format, business policies), the file had to be read sequentially, processed sequentially, and imported into the database. The processing time was usually dominated by a single huge file, so the obvious technique to exploit a multicore (use several instances to import different files in parallel) would have not been effective.
Note that when we think of parallelism in this way, we're looking for symmetric parallelism, where each thread performs basically the same job (process a request, or import a file, or whatever). There is only so much you can do with symmetrical parallelism, especially on a client (more on this later). Sometimes (of course, not all the times), it's better to think asymmetrically, that is, model the processing as a pipeline.

Even for the batch application, we can see at least three stages in the pipeline:
- reading from the file
- doing any relevant processing
- storing into the database
You can have up to three different threads performing these tasks in parallel: while thread 1 is reading record 3, thread 2 will process record 2, and thread 3 will store [the processed] record 1. Of course, you need some buffering in between (more on this in a short while).
Actually, in our case, it was pretty obvious that the processing wasn't taking enough CPU to justify a separate thread: it could be merged with the read file operation. What was actually funny (almost exhilarating :-) was to discover that despite the immensely powerful database server, storing into the database was much slower than reading from the file (truth to be said, the file was stored in an immensely powerful file server as well). A smart guy in the bank quickly realized that it was our fault: we could have issued several parallel store operations, basically turning stage two of the pipeline into a symmetrical parallel engine. That worked like a charm, and the total time dropped by a factor of about 6 (more than I expected: we were also using the multi-processor, multi-core DB server better, not just the batch server multicore CPU).

Just a few weeks later (meaningful coincidence?), I stumbled across a nice paper: Understand packet-processing performance when employing multicore processors by Edwin Verplanke (Embedded Systems Design Europe, April 2007). Guess what, their design is quite similar to ours, an asymmetric pipeline with a symmetric stage.

Indeed, the pipeline model is extremely useful also when dealing with legacy code which has never been designed to be thread-safe. I know that many projects aimed at squeezing some degree of parallelism out of that kind of code fails, because the programmers quickly find themselves adding locks and semaphores everywhere, thus slowing down the beast so much that there is either no gain or even a loss.
This is often due to an attempt to exploit symmetrical parallelism, which on legacy, client-side code is a recipe for resource contention.Instead, thinking of pipelined, asymmetrical parallelism often brings some good results.
For instance, I've recently overheard a discussion on how to make a graphical application faster on multicore. One of the guy contended that since the rendering stage is not thread-safe, there is basically nothing they can do (except doing some irrelevant background stuff just to keep a core busy). Of course, that's because he was thinking of symmetrical parallelism. There are actually several logical stages in the pipeline before rendering takes place: we "just" have to model the pipeline explicitly, and allocate stages to different threads.

As I've anticipated, pipelines need some kind of buffering between stages. Those buffers must be thread safe. The banking code was written in C#, and so we simply used a monitor-protected queue, and that was it. However, in high-performance C/C++ applications we may want to go a step further, and look into lock-free data structures.

A nice example comes from Bjarne Stroustrup himself: Lock-free Dynamically Resizable Arrays. The paper has also a great bibliography, and I must say that the concept of descriptor (by Harris) is so simple and effective that I would call it a stroke of genius. I just wish a better name than "descriptor" was adopted :-).

For more predictable environments, like packet processing above, we should also keep in mind a simple, interesting pattern that I always teach in my "design patterns" course (actually in a version tailored for embedded / real-time programming, which does not [yet] appear on my website [enquiries welcome :-)]. You can find it in Pattern Languages of Program Design Vol. 2, under the name Resource Exchanger, and it can be easily made lock-free. I don't know of an online version of that paper, but there is a reference in the online Pattern Almanac.
If you plan to adopt the Resource Exchanger, make sure to properly tweak the published design to suit your needs (most often, you can scale it down quite a bit). Indeed, over the years I've seen quite a few hard-core C programmers slowing themselves down in endless memcpy calls where a resource exchanger would have done the job oh so nicely.

A final note: I want to highlight the fact that symmetric parallelism can still be quite effective in many cases, including some kind of batch processing or client-side applications. For instance, back in the Pentium II times, I've implemented a parallel sort algorithm for a multiprocessor (not multicore) machine. Of course, there were significant challenges, as the threads had to work on the same data structure, without locks, and (that was kinda hard) without having one processor invalidating the cache line of the other (which happens quite naturally in discrete multiprocessing if you do nothing about it). The algorithm was then retrofitted into an existing application. So, yes, of course it's often possible to go symmetrical, we just have to know when to use what, at which cost :-).

Labels: algorithms, article reference, C++, concurrency, multicore, multithreading, pattern, profession

posted by Carlo.Pescio @ 6:39 PM <permalink>  7 comments
bookmarking:    

In the last few posts I've been writing a lot about AOP, and very little about what I'm doing every other day. It's plain impossible to catch up, but here is something that has kept me busy for quite a few days lately: Client Side Augmented Web Applications (I should file for a trademark here :-). What I mean is a regular web application, that can be regularly used as a stand-alone application or, when you install some additional modules on the client, can also interact closely with other applications on the client side (e.g. the usual office suite, and so on).

Naturally, that means web pages must have a way to send data to client side application, and to obtain data from the client side application. For several (good) reasons, we wanted this data exchange to be as transparent as possible to the web app developers. Also, we didn't want to write different web applications (regular and augmented). That would have had a negative impact on development and maintenance times, and it could also have proven to be an inconvenience for the users. This had some impact on page navigation design, which could be an interesting subject for a future post.

Now, I can't get into the specifics of the project, or disclose much about the design of the whole infrastructure I've designed and built (yes, I still enjoy writing code :-). However, I can show you the final result. If you want your ASP.NET page to obtain (e.g.) the filename and title of your Word document, and send back to Word a corporate-wide document number, all you have to do is add a few decorated properties in your .aspx.cs source file, like:

public partial class DocumentProperties : System.Web.UI.Page
  {
  [AskClient("filename")]
  public string Filename
    {
    get
      {
      return filename;
      }
    set
      {
      filename = value;
      }
    }

  [AskClient( "title" )]
  public string Title
    {
    get 
      { 
      return title; 
      }
    set 
      { 
      title = value; 
      }
    }

  [SendClient("description")]  
  public string Description
    {
    get
      {
      return description;
      }
    set
      {
      description = value;
      }
    }

  // here goes the usual stuff (methods, data members)
  }

that's it. The attributes define the property name as known on the client side; the invisible infrastructure will take care of everything else. In a sense, communication between the web application and the client side application has been modeled as a virtual machine concern, and the attributes are used to tell the virtual machine where interception is needed. Of course, this is only the tip of the iceberg. There is a lot going on under the hood, also on the client side, as your browser and your favorite application are usually not good friends, and to be honest not even acquaintances.

Is my abstraction leaky? Sure it is. To make all that stuff working, you also have to drop a non-visual component into your page at design time. That component, if you follow the virtual machine metaphor, is indeed the implementation of the (server-side) virtual machine layer that will deal with the communication concern.
If you don't drop the component into the page, the virtual machine layer just won't kick in, and your attributes will stay there silently, stone cold. This is a functional leak that I'm aware of, and that as a designer I have accepted. In fact, all the alternatives I've considered to avoid this leak had some undesirable consequences, and keeping that leak brought the best overall balance. Besides, there are ways to somehow hide the need to drop the control into the page (like using page inheritance or a master page), so it's really no big deal.

A final reflection. I do not believe that something like this could come out of refactoring code that was simply meant "to work". It's relatively trivial to make a web application and a client-side application talk. It's quite different to make this talk transparent. Having prototyped the first option (just to make sure a few ideas could actually work) I can honestly say that without the necessary design effort (and skill), it's extremely unlikely to come close to the final result I got.

Time to drop a few numbers: overall, I've spent roughly 20% of total development time experimenting with ideas (throwaway code), 35% designing, 30% coding (this includes some unit testing), 10% doing end-to-end testing, 5% debugging. Given the relative novelty of several techniques I adopted, I should actually consider the 20% prototyping an inherent portion of the design activity: you can't effectively design much beyond the extent of your knowledge. Sometimes, the best way to gather more knowledge is to talk; sometimes, to read; sometimes, to write code. Of course, I was looking for knowledge, not for code to keep and cherish, so I happily scrapped it away to build much better code. In the end, that's often the difference between intentional and accidental architecture.

Labels: AOP, ASP.NET, design, profession

posted by Carlo.Pescio @ 9:51 AM <permalink>  3 comments
bookmarking:    

I've spent the last two days (and tomorrow as well) over the design of a new product. The whole project has been an exercise in fighting complexity and "featuritism" or, put another way, in shrinking down requirements and technology to a bare minimum. The goal was clear (at least to me :-) : provide the core benefits to the user, avoid any useless and costly feature, squeeze development times by allocating features where they belong.
We obtained a dramatic improvement in development time (and in my opinion, in product marketability) by moving some features from the PC to the embedded side (!). Another big gain came from kicking a standard protocol out and using a tailored command (within the protocol framework, but still entirely custom); this improved reliability as well. The final, substantial gain came from kicking out a large, reusable infrastructure we built years ago for similar applications, and designing a (much smaller) custom solution.
Now, this is not the kind of suggestion people would expect from me. But reuse is not a value per se. It's a value when it provides an economic advantage. In this case, the initial investment needed to fit the problem inside the large, complex architecture just wouldn't pay off. Besides, the custom solution is not badly designed. We have a clear point where we could theoretically sweep out the custom part and plug a bridge to the more powerful infrastructure. This is, in my opinion, quite sensible design. We have the minimum overengineering needed to move to an higher level later, at low cost, if we ever need this. Meanwhile, the project will sell, and hopefully sustain itself.
Economy 101 :-), or a careful use of Real Options theory? More the first than the latter, but again today, when designing a reporting subsystem, there was a clear separation between what is useful to design now (everything down to an XML document), and what is better (economically better) to postpone until a more careful evaluation of alternatives has been carried out. This is easily explained by Real Options - the option to wait on this part has more economic value than the option of designing it straight away. This (as well as kicking code reuse) is somewhat entangled with the notion of Second Order Ignorance from Armour, but I'll leave this as the dreaded exercise for the reader :-))).

Labels: design, profession, real options

posted by Carlo.Pescio @ 9:52 PM <permalink>  3 comments
bookmarking: