Why Public IT projects Fail – Continued

Yesterday I commented on the status for the Property Registration System and mentioned, that another major IT failure seemed to have been hiding from the public eye – namely the wreckage of the Danish Emergency Management System. Normally I don't claim to possess a crystal ball, but today the news hit the World: Under the heading: ' The Prime Minister knew that the Emergency Management System was high risc', a paper from the Ministry of Finance dated May 16 2007 documented that the Minister of Finance, the current Prime Minister knew that the Emergency Management Project, a Dkr. 2 B + project, was evaluated at high risk, 4 out of a scale 1 to 5.

This document was originally confidential, confidentiality raised in August 2007, when the contract with the consortium led by Motorola was signed. But the contract – although a major investment – did not cover the 'control room SW', which was out for tender and not signed before 2008.

This contract was won by Terma, a Danish company until then only known for it's supply of high tech military systems, radar controlled defense systems and firearms control systems for helicopters, fighters and ships. The contract value was at the time estimated to be a mere 200 – 250 mio Dkr. For the Control Room SW including 4 years of operation, as opposed to a total budget for the entire emergency management system at 2.718 Mio Dkr.

At the time of the evaluation, the Ministry of Finance could not include Terma and the contracted solution (that also included Intergraph and a couple of smaller, Danish companies) in the risk evaluation. So the reason a '4' was the result, was a combination of 4 for organizational complexity, 4 for technical solutions, 3 for vendor risk and 5 for complexity of stakeholder interest.

As a contingency plan to reduce/meet the risk, 193 Mio was set aside as a buffer, and the conclusion was, that if any of risks materialized, the consequences were thought only to impact the project plan and the budget.

At this point in time it is clear beyond any doubt, that although the control room SW contract only covered a mere 10% of the total project cost, this part of the project has failed completely, Terma's Staff for has been fired, and the department Public Safety, created for the occasion, has been dismantled. The Government Attorney is currently handling the case.

But as I mentioned in a blog, the Government Attorney is no programmer, and this will not create anything but a financial resolution of the conflict between Terma and the Government.

The secretariat set up to handle the emergency system – SINE – has been radio dead around this.

See their homepage at http://www.sikkerhedsnet.dk/om+sine.

So how can we determine what went wrong? If you look at the SINE homepage you will find a link to European Tetra-net projects that more or less all seem to have succeeded. What went wrong in Denmark? What is different and why couldn't we just adopt this seemingly uninteresting piece of technology and enable the fire brigade to talk to the police and the rescuing service to the civil defense units? There are several answers to this question:

1. Underestimation of the importance of the 'control room SW'.

2. A very complicated and partly conflicting group of stakeholders

3. A lack of a systemic overview of both tenders before call for tender

4. An steering committee consisting of gatekeepers instead of decision makers

5. Lack of understanding of the competence level of the winning company (Terma)

6. Lack of understanding of technological improvements in the emergency mgmt arena

As is clear from the international references, the Tetra net is not a very complicated technology – it has a rather small bandwidth, but in case of emergency you could deal with only limited amount of data and voice interchange. Also Motorola is known to have skills and competence in this area, but the exchange between different organizations was explicitly placed in the second call – 'control room SW' – that also covered integration with GIS (Intergraph) as well as links to all the different organizations variety of communication media – plus an integration layer to back end systems like the Central Personal Registration system, interface to local office systems, hospital exchange, and interfaces to privately operated ambulance services and in some areas also fire brigades.

The complicated mixture of stakeholders were supposed to be coordinated by the Police, that although in practice more than 80% of all cases of (day-to-day) emergency management calls do not involve the police. And the police was in the middle of a major reorganisation, where this system was just another brick in the wall. The tender was headed by the Agency of Economy, part of the ministry of Finance, and the Emergency Management organization was merged with the Ministry of Defense a few years earlier and was probably looked upon as a sort of step child here. The complication also included the above mentioned private ambulances and fire brigades, covering about 50% or more of the local Government emergency services.

The lack of systemic overview before the 2 tenders calls for a reevaluation of how and why Gartner Group, hired as a consultant company to prepare the tender, could overlook the key role of 'Control Room SW' and underplay the importance of technical interoperability between the different stakeholders and not least the level and complexity of change management to ensure the take-up of the solution by all major players.

It seemed that the steering committee probably covered the right institutions: Ministry of Justice (which includes the Police) acting as chairman, Ministry of Finance, Ministry of Defense, Ministry of Welfare, Ministry of Science and Technology, Association of Municipalities and Danish Regions Association. Until proven wrong it is my hypothesis that the steering committee meetings concentrated more on status of progress, spending of money rather than risk management and change management like acceptance and enacting of the various groups 'in the fire' literally speaking. If stakeholder committees act as gatekeepers, it is at best an advisory committee, not a steering committee. In stead the head of this group should have been a dedicated 'CZAR' with sufficient decision power to change events – if possible by changing the contract.

Lack of understanding on the proper level of competencies and vendor capabilities. Terma had never before worked in the field of Public Safety; nor had they ever participated in major SW integration tasks with administrative stakeholders – their use of standard systems was and is 'Java'. When the contract negotiations squeezed Terma, they skipped partners that could have provided the skills and muscle, that Terma obviously did not possess when the signals of crisis were raised. And because of the nature of public tenders, they were allowed by a steering committee to continue for too long.

The lack of understanding of concurrent technological development was another reason, why the course was not changed; over the last few years the terminology 'Unified Communications' and 'Collaboration Services' has been greatly improved, mainly led by the defense forces, but using standard SW components and basing communication of the Internet. To day at least 2 vendors offer SW capability for multi-platform same-time support of communication systems supporting both Tetra Net (Motorola), standard mobile communication channels as well as highly sophisticated and encrypted VPN tools for same-time video conferencing, exchange and overlay of Geographic information. ( See CISCO Solution, IBM solution )But nobody was allocated to watch this field while the hand coding of Terma progressed. This could actually have saved the project – if attention to the acceptance of the solution by all stakeholders also had been in focus.

Beredskabssymposium - når krisen kradser..

Torsdag den 8. maj afholdt Forsvarsministeriet og Militærteknisk Forening sammen med ingeniørforeningen i Danmark en konference med fokus på samordning af det danske katastrofe-beredskab. I sig selv en overraskende debat, da Terma sammen med Økonomistyrelsen i disse måneder er i færd med at rulle SINE-kontrolrumsløsninger ud. Terma's løsning, der spiller sam-men med Intergraphs GIS_løsning, vedrører styring og koor-dinering af TETRA-nettet, der iøvrigt leveres af Motorola, der vandt selve radiokommunikationsnettet for en 8+2 årig periode.

SINE er radiokommunikation mellem indsatsenheder fra de 'Blå Blink', men giver altså, som Forsvarsministeriet åbenbart er helt enige i, ikke svaret på nonnens bøn om hvordan man skaber et samlet overblik på tværs af de 135 krisestabe, der i en national katstrofesituation kommer i spil.

Dette forhold var vi i Teknologirådets bestyrelse meget opmærksom på, og der er udgivet to rapporter om emnet, den første hed: Katastrofeberedskabet mangler klar kurs: http://www.tekno.dk/subpage.php3?article=1376&language=dk&category=6&toppic=kategori6
og den anden efter en konference i Januar med titlen: Danmarks nye katastrofeberedskab under lup - en lang række tiltag er nødvendige, hvis SIkkerhedsNEttet skal leve op til forventningerne: http://www.tekno.dk/subpage.php3?article=1474&language=dk&category=6&toppic=kategori6

Det var heller ikke en del af Økonomistyrelsens kravspecifikation, der koncentrerede sig om selve Tetra-nettet og de faste kontrolrum. Til brug for tværgående kommunikation ved hjælp af internet-teknologi har IBM den såkaldte VoC-løsning, der bygger på IBM's collaboration Suite og Dynamic Team Management/Tivoli Directory Integration, der gør det muligt dynamisk at sammensætte diskussionsgrupper og etablere virtual teams rooms mellem medarbejdere, der sidder i adskilte organisationer, og som i en krisesituation skal samarbejde. Løsningen er implementeret i en række steder i USA (New Jersey Police, Missouri), har været anvendt i forskellige sammenhænge, bl.a. til Olympiaden i Athen, VM i fodbold i München og er nu i efteråret blevet installeret som basis for emergency operations i Sydafrikas hovedstadsprovins.

Mødet i IDA var særdeles interessant, det fremgik at forsvaret ikke ser SINE som noget som helst endegyldigt svar på behovet for fælles situationsoverblik og kommunikation, især Vicebrandchef Søren Brydholm fra København havde de store visioner om dynamisk kommunikation, men også Hjemmeværnets repræsentanter og Michael Flagstad fra forsvarskommandoen havde interessante synspunkter.

Af debatten blev det meget klart, at der er et behov for hurtigt at identificere løsninger, der kan skabe overblik, hvor man nemt kan få adgang til beredskabsplaner, og hvor man kan benytte det operative system også til at gennemføre omfattende øvelser.

Det var en overraskelse for flere, at de nationale sundhedsmyndigheder ikke har sæde i de to øverste krisestabe, og at man på lokalt niveau ikke har faste repræsentanter. Overlæge Annlize Troes fra Sundhedsstyrelsen redegjorde fint for hvilke opgaver, der ville være vigtige: fugleinfluenza, husdyrepidemier, andre epidemier m.v.

Der er ingen tvivl om at behovet for tværsektoriel kommunikation, herunder ikke mindst at kunne inddrage kommuner og sygehuse samt hele transportsektoren og forsyningssektoren i et fælles kommunikationsnetværk, kommer til at spille en meget stor rolle i debatten fremover.
Et af de tilbagevendende problemer er, at militæret og efterretningstjenesternes netværk skal være fysisk adskilt fra ekstern adgang, og indtil en reel teknisk løsning er fundet, er det altså ret vanskeligt at dele billeder, video, kort information.