Sunday, July 31, 2005 

The perils of abstraction

When we build large systems, abstraction is our best friend. We can't constantly worry about all the tiny details of each and every component, class, library, API, etc. Still, abstraction is not a substitute for understanding. We can safely abstract away what we know, not what we don't know.
Real-world example: I know how to do asynchronous I/O using completion ports in Windows. I know how it works (right down into the kernel, although this depth is not really necessary) and when to use it. I've also built a small framework of classes to abstract away some details and crystallize some sensible design decisions. It is easier to build systems over the abstract view of my miniframework; however, I can always look under the hood if needed, because I'm abstracting away something that I know.
We can also try to abstract away things we don't really know. We routinely do that with communication protocols: we don't need to know the gory details of TCP/IP to open a socket and send/receive a few data. Well, maybe we do need to know :-) if we want to squeeze good performances out of it. Abstracting the unknown is the primary purpose of many libraries, components and technologies.
Whenever we use abstraction to shield ourselves from missing knowledge, however, we are taking a chance. Two things can go wrong with abstracting the unknown:
- the abstraction is not working the way it should. In this case, people often resort to exploratory programming, trying out things (parameters, flags, execution order) and looking for a combination that works. This is just a way to avoid looking under the hood, but we end up with a system that seems to work, but you don't know why.
- much more dangerous: we build our system on fragile foundations, because we are misinterpreting the abstraction.
Real world example again (this is actually what inspired me to write this post). I was reading an article on the Caching Framework Application Block in .NET. The author says that you can host the CacheService in-process, out of process, or in another (shared) computer, which is right. He then goes on to say that you can choose your storage to be Singleton, Memory Mapped File, and Sql Server. He says that Singleton is a hash table, therefore not thread-safe, therefore to be used only in process. That Memory Mapped File is thread-safe. That Sql Server is persistent and can handle large date volumes. Everything in italics is basically wrong. Seems like he didn't knew the underlying abstractions, ending up with the wrong reasons to choose the storage. Singleton is a hash table, and a hash table can be easily turned into a thread-safe container simply by using a synchronization primitive. Indeed, even in an in-process scenario, you may have multiple threads, so it would be ridiculous if the Singleton storage wasn't thread safe. However, since the hash table will be stored into the address space of one single process, it won't be easily shared among processes. A Memory Mapped File is not thread safe by itself. We still need to build synchronization around it to avoid conflicts. Still, a MMF can be easily shared between processes, and is the ideal candidate for an out of process scenario. Again, MMF are easily shared between processes but not between computers. Opening your MMF on a shared server does not reliably work. If you want to share your cache between multiple computers, you need a standalone process communicating over the network and keeping your stuff cached. A database is a ready-made implementation of that, which accidentally is also persistent and can handle large storages (hardly useful for a cache anyway!).
In this specific case, the misunderstanding wasn't really that dangerous, but I see this happening all the time: people committing to (e.g.) COM+ or J2EE not because they need those services, they understand what is going on under the abstraction, and want the details abstracted away, but because they don't want to understand what is going on, and they hope the abstractions will take care of everything.
Conclusion: there is no free lunch - we need to understand the potential gain and potential risk of our decisions, including abstracting away the unknown, weight the two, and choose responsibly.
Comments:
Visto che hai citato gli "asynchronous I/O" mi vien voglia di farti una domanda piuttosto tecnica con al solito la libertà da parte tua di rispondere o meno. In particolare mi riferisco alla WriteFile per fare comunicazione su porta seriale sfruttando la modalità overlapped. Orbene esistono su MSDN chiari esempi di come sfruttare l'API in questione per essere "allertati" quando un'operazione di scrittura si è conclusa. Peccato che quest'evento corrisponda a quando i dati sono fuoriusciti dai buffers dei vari strati software, ma non a quando hanno lasciato l'UART. Quello che sotto DOS si poteva sapere con certezza con poche istruzioni assembler (anche inline) interrogando lo stato dello shift register, non può essere fatto agevolmente se non ricorrendo alla scrittura di apposito device driver. Concordi? Perché è così importante conoscere l'esatto momento in cui un carattere è uscito dall'UART? Perché in una comunicazione multi-drop (RS-485) questo è vitale per commutare da trasmissione a ricezione oppure per cambiare parità da mark a space (il che ti consente di sfruttare tecniche di wake-up nelle periferiche di campo - la tua pregressa esperienza di dialogo con apparecchiature medicali mi fa ritenere a ragione che tu sappia di cosa stiamo parlando). Senza per forza ricorrere alla scrittura di un device driver di seriale, che ora sarebbe più conveniene data la dismissione di Win9X, NT4 e 2K, conosci un metodo per risolvere la cosa ad alto livello? Non accenno poi più di tanto alle inefficienze rilevate con adattatori USB-Seriale. Basta monitorare con un oscilloscopio per rendersi conto come operazioni di commutazione della parità con una seriale standard abbiano tempistiche accettabili, mentre invece con tali adattatori salgono a valori inaccettabili se i tuoi timeout devono stare entro i 3/4 millisecondi. Lo so che è tutta questione di protocolli e cambiarli renderebbe al vita più facile...
 
Le 485 le ritrovo piu' spesso in impianti industriali che negli apparati medici, comunque ho presente... e anche per i poveri convertitori usb-seriali l'esperienza e' quella...
Come dici giustamente, e' spesso questione di protocolli ("vecchi"), ed in tanti casi la soluzione migliore e' stata proprio un cambio di protocolli, dove aveva senso gia' a partire da quello fisico :-).
Sulle soluzioni "ad alto livello" direi che a questo punto ne parlero' nel post di oggi (stasera, probabilmente :-) generalizzando un po' sull'accesso all'hardware. Rimane vero il fatto che talvolta sarebbe necessario / comodo cablare un po' del protocollo dentro il driver. Considerando che nei vari DDK ci sono di solito i sorgenti del serial port driver, non e' neppure questa impresa cosi' titanica...
 
This post has been removed by a blog administrator.
 
Post a Comment

<< Home