Friday 17 August 2007

Committee Standards and Defacto Standards in IT

Here is the question:
Is there one usable IT standard which has been brainstormed and created from scratch out of committees and other consortiums ?
Since I do software development (wow more than 10 years now), I've seen several standards being released out of committees and being discarded or beaten by a contender because the so called standard only exists on paper, it isn't partical, it is too complex or it is too ambitious. This happened many times and will happen again in the future. Let's take some concrete examples:

1. X.25 and TCP/IP

Who can remember X.25 which was strictly following the OSI layers Model (by implementing layer 2 and 3) and should have been used to create the backbone of what we now call Internet. However the winner was TCP/IP mostly because TCP/IP was easier to understand and managable even if it was not fully following the faboulous 7 layers OSI Model. TCP/IP became also the defacto standard because it was adopted by the research community in order to create a WAN (CERN, NASA, American Universities, ...).

2. OMG CORBA

mid-nineties, Internet was emerging and becoming important, the object oriented programming was the most dominant paradigm and naturally there was a need to create a standard in order to build object oriented applications distributed over several sites. So the Object Management Group (OMG) a consortium regrouping the influential players in IT such as Hewlett-Packard, IBM, Sun Microsystems, Apple Computer and others gave birth to CORBA (Common Object Request Broker Architecture). Everybody also knows that giving birth can be quite painful and here it was the case as the different partners started to fight and push different ideas in CORBA. More Microsoft was not part of the club and had decided to develop a concurrent called DCOM based on Windows technology. At the end, the created standard ended-up to be very cluttered and was covering too many topics. It became a nightmare for the CORBA implementors as it was impossible to build a complete CORBA implementation covering all the different services. It was also very easy to have two compliant implementations which were not interoperable. Does it ring a bell ? yes, the standard WS-I. One thing which was also totally missed by these Computer Science Experts was that HTTP would become the only way to communicate between two sites because of the apparition of the firewall. Indeed these little beasts were configured to only let go HTTP from the WAN to the DMZ.
At the end CORBA has progressively disappeared and has been replaced by simpler Object protocols relying on HTTP (REST for example as SOAP will also probably vanish).

3. The Rest of the herd

So in those days, the list could be very long. Obviously you've understood that I do not particularly fancy the WebService Infrastructure (WS-I, SOAP, WS-Security, ...) and most of the XML standards which seems to have been birthed within endless meetings as most of the time they are totally unpractical (XSD and XSLT are part my favorites here). On the other hand, you have the REST stack which is slowly maturing and being implemented to provide real services while taking on board the same good ideas as the ones used to create WS-I.

As you can see the same story seems to be repeated all the time.

4. OGC Standards: the current annoyance

You could now say where does this guy want to end-up ? In fact I am currently working on a very interesting project where the main goal is to create a framework and deploy an infrastructure for discovering, distributing and sharing weather and climate data. All this is closely linked with the Geoweb as all these data are spatio-temporal data. Within our world there are 3 different contenders: You have a pragmatic effort started by the Earth Science Community beginning of the 21st century and lead by the American universities and main research centers in climatology (NetCDF, OpeNDAP, etc). You have now the geoweb lead by Google to display geo-spatial data on the web (google maps, google earth, KML). You also have a standard pushed by a consortium called OGC (Open Geospatial Consortium) which defines everything from the protocols and Web Services used to exchange the geo-spatial data, how to orchestrate these services, how to discover these data and how to represent these data (GML).
The issues comes from the last one because it is creating lots of disruptions and interferences within our project. The OGC standard is a patchwork of ideas which creates a clumsy model at the end. The OGC WS is somehow broken as it does not offer ways to request asynchronously data, does not embed security mechanisms and wants to do too much. I am not even talking about the discovery part which is based on the idea to have a metadata standard for all these datasets when the same idea applied to the Web never worked.
GML is the jewel of the crown there but his "concurrent" KML is controlled by only one body (Google) and can evolved very quickly. KML also has got the Google Earth advantage as it is a shiny GIS client.
Lots of people would like to push us to be compliant with the OGC Standards but when you look at it, you can guess that in few years time, this will have vanished or mutated to use defacto standards such as KML and friends. So we should probably concentrate on creating a pragmatic solution efficient for our users and slowly migrate to matured and proven technology when necessary.
After our goal is not to be compliant to standard A or B but to improve the user's life.

To conclude here is my proposal : to avoid loosing time, money and produce tons of CO2 in travel we should ban and minimize the number of committees created to regulate the IT world as they will irremediably fail and provide very little satisfaction to their participants.