This is for those
who have expressed concerns to me about Project Phoenix.
I joined Emirates Group on the day Project
Phoenix was announced, 1 July 2006.
Patrick Naef had been in post for five months and it was clear to him (indeed
everyone) that some serious work had to be done to improve the service being
delivered to the business. The number of
operational outages was mind blowing and the project delivery record appalling. I do not know when Patrick had engaged the
external consultants (who were heavily involved on the day of the announcement)
but he clearly had decided at an early stage to take drastic action, deciding
to dismantle the organisation and starting again.
It did not take long in my new post (VP Technology
Services) to appreciate the scale of the problems, nor to identify the main
areas to work on. Most of the
infrastructure was not fit for purpose (a lot of it was so old it no longer
enjoyed support from its supplier), effective standard IT support processes
(such as change, risk and incident management) did not exist, management
competence was scarce and the IT departmental culture was swamped with cronyism
and blame. Quite a list! But there were some positives, including an
incredibly dedicated and hard working team, all of whom agreed (and desired)
that things had to change.
Virtually all the IT infrastructure,
including the entire Data Centre, needed replacing and this would take time. But there were some serious issues which
required immediate attention. As an
example, I discovered that the Data Centre had no resilience in the air
conditioning system. The failure of just
one small unit would have put the entire Data Centre out of action. The reason given to me for this state of
affairs was “no budget”! Also, there was
an auto start generator in place to provide backup power in the event of a main
supply failure. The generator was
perfectly capable of supplying all the IT equipment, but it was not connected
to the full air conditioning system! So,
all it gave us was time to power everything down in a controlled manner,
leaving our passengers waiting until the power came back. It is a credit to the Data Centre staff and
also our colleagues in the Facilities department, that the necessary remedial
work was completed before the peak (in terms of both passenger numbers and
temperature) of the summer of 2007. We
did not have to spend a huge amount of money (if my memory serves me well, the
air conditioning units were around AED10K each), but carrying out the work (particularly
power) without impacting a 24/365 operation was a major challenge for the teams
involved. The work was completed without
any unplanned service disruptions. There
is absolutely no doubt that, without this work, the summer of 2007 would have seen
outages unheard of in the industry, with very serious consequences for Emirates
Airline.
The good news was that Patrick had already
gained approval for what was called the Resilience Project. Although the benefits had been oversold to
the business (“seamless changeovers during server outages” which was not
actually achievable on a lot of systems), it was good that procurement of new
hardware was underway. I can only assume
that such exaggerations of benefits were necessary in a political environment
which I had never encountered before.
The company clearly had money, but had chosen to spend virtually none of
it on its internal IT systems. I believe
that the Resilience Project had been scoped before Patrick had arrived but he
had given it the necessary impetus to get it started. But there was a long list of other work that was
required, such as on the email system which was out of support and about to
collapse.
High severity incidents, i.e. those that had
a direct operational impact on the business, were running at over 300 per
quarter, more than 3 per day. I found
this difficult to believe. My previous experience
had been in environments where the target was always zero, with just a single such
incident in a quarter generating huge complaints from our customers. But, at Emirates, we often had two, sometimes
three, such outages at the same time!
The problem was not solely due to overloaded and ‘not fit for purpose’
infrastructure, there was no effective change management process. The resolution for many outages was often stated
as “resolved by itself” which was IT speak for ‘someone carried out an
unauthorised change, it went wrong, so it was backed out quickly’. Activities were not logged so no-one, other
than the individual who had carried it out, would ever know about the
change. Not all changes were done in
secret, others were planned and notified, but there was rarely any effective
risk management applied. The answer to
my standard question of “what can go wrong” was normally “nothing, we’ve done
it before”.
A huge factor was the general level of
management competence. There were exceptions,
but some managers were struggling. They
were very dedicated people with sound technical skills but I am afraid that,
across the globe, IT people do not always make good managers. Understandably, a lack of management
responsibilities will always dilute a job evaluation in any organisation but
Emirates HR had apparently insisted that the ceiling for any ‘non-management’
IT operational job was G8. This was
absurd. A company the size of Emirates
which provided its IT support internally should be able to accommodate
technical subject matter experts as far as G10, as long as the individual’s depth
and width of knowledge can justify the grade.
Technical staff should certainly have been able to aspire to a G9
technical role without getting bogged down with management chores, something that
many did not want. But the HR rule meant
that the only way a good technical person could be adequately rewarded, was by
making them a manager. The result was
inevitable.
But the culture was probably going to be the
biggest challenge. The name of the game
was to please the boss and drop your colleagues into as much trouble as
possible. And this applied at all levels
in the organisation, it was a standard survival technique. Subservience dominated – most managers in my
team would not get close to disagreeing with me, so management debate was initially
almost non-existent. In a meeting with
my managers, at the end of my first week, I asked them what they thought of
me. They looked at the floor. Eventually, one of them said that he thought
that “I was a reasonable sort of chap”. That
was the first time anyone had told me that!
So there was a lot to do, but it was
achievable and I was more than happy to get on with it. One immediate focus had to be on the physical
infrastructure replacements, so funds had to be secured. No elaborate business cases were needed, everything
was about to go pear shaped so the company had no choice. For some reason Emirates had made the
decision to run the whole place on a shoestring and now the consequences were
staring us in the face. The change/risk/incident
elements were easy, they could be (and were) implemented immediately with a
total hands on approach initially, and then slowly delegating to those who
showed the ability and inclination to take it on. You cannot effectively (and therefore should
not) change management teams overnight.
Working with everyone allows time to understand strengths, career
aspirations, learning needs, etc. and, if it proves necessary to move an
individual into a non-management role in due course, they themselves will
appreciate it. By then, the G9 Technical
role issue would have been resolved – HR had to change their minds, they were
simply wrong. As for the culture, that was
going to take a bit longer! But everyone
contributes to the culture, so a push in the right direction with the right
level of resolve, often works wonders.
So, had Phoenix not been thought of, personally
I would not have engaged external consultants to effect the required changes. I would have considered using outside help
with accelerating some hardware replacements (indeed we did engage an external
organisation when the mail system finally collapsed) and I am sure at some
stage help with the culture may have been appropriate. But I believe in continued management
ownership of change; not only directing
it, but being part of it and then living with the failures as well as the successes
with everyone else. Only then can you genuinely
claim to be part of the team.
But Patrick Naef had a wider portfolio than I
did, so he probably would have seen more problems than I had, and possibly seen
fewer potential fixes. And, until I had
arrived, he had no-one to lead the assault on the appalling operational service
(my post had been vacant). Had I joined
earlier, I am sure I would not have been championing his cause of bringing in a
team of consultants, I would have encouraged him to take it on ourselves. But I would not have ‘died in a ditch’ on the
issue. I am not the world’s biggest
advocate of consultants and am aware that affects my approach to them at times. So, had Patrick felt that this was the only solution,
I would have given it my full support. As
I had arrived on ‘announcement day’, my opinions were not only irrelevant but
far from crystallised at that time, so I naturally gave the Phoenix Project my total
support without any debate. Patrick Naef
often accused me of “not supporting Phoenix because it wasn’t your idea”, but
he had no grounds to do that. I do not
know if this was a result of one of his misunderstandings, or if it was one of
his usual ‘pitch rolling’ antics so that, if his Phoenix Project failed to deliver,
he had an excuse of “Tom undermined it” lined up. I only ever made one serious point about
Phoenix and this was right at the start and I never mentioned it again. The entire IT department was effectively
dissolved and a new organisation would evolve.
All managers would have to apply for whatever jobs emerged. I pointed out that we ought to put the
managers ‘at risk’ because it was possible that some could be without jobs at
the end of the process. I was
patronisingly brushed off along the lines of ‘you don’t need to introduce UK HR
processes here’ but it had nothing to do with processes and legal definitions,
just common sense and reality. If
someone ends up without a job, they are redundant. It does not matter if you use a capital R or
a small r, the impact on the individual and his/her family is the same and it
is a major issue which will need managing.
(In fact, it did happen and it was indeed difficult to manage.) The final response on the subject (not by
Patrick, I must add) was, and it would not be the last time I would hear it, “Tom,
this is Dubai. We can do what we like.” The Phoenix Project was always given my full backing,
both during and after it. Views I had on
various matters were voiced with my usual passion within the project as it
progressed and then, whatever decision was made, was given my full and
continued support. For Patrick Naef to
say otherwise is disingenuous.
It was clear that there would be some serious
challenges thrown up by Phoenix. Not
only were managers being told that they would have to apply for jobs, they were
being excluded from all day to day aspects of the project. A team of individuals, around 20, had been
selected to assist the external consultants.
This was a major opportunity for the individuals and they threw
themselves into the task with great enthusiasm and commitment. But it left the senior managers feeling very
exposed and alienated. The name of the
project did not go down well either, the term Phoenix was supposed to represent
something ‘rising out of the ashes’, in other words the so called ‘ashes’ of a
department that many hundreds of loyal staff had dedicated themselves to over
many years. If you wanted to find
someone responsible for that fire, looking in the direction of senior
management would have been a good start.
It had been the loyal staff who had been perpetually fighting the
fire. Ironically, Patrick Naef (the newly
recruited ‘fireman’) had arrived with an unlimited supply of kerosene and
matches, which he would use liberally over the coming years. But no-one knew that, as Patrick was
innocently waving a fire extinguisher at the time.
Specific approval for the project had to be
secured, so a presentation was drafted.
The cost took us all (including Patrick Naef) by surprise. It would be inappropriate to reveal the
amount, but it was far bigger than I had expected. And no account was taken for internal labour
costs, we were just looking at the bill for the consultants. I was surprised that it was the only option
being proposed. There were no quotations
from other suppliers (there had not even been a tender) and no other options
(such as using in-house resources, smaller scope, etc.). I suggested that we ought to have other
options but Patrick insisted that there was no plan b. The proposal was going to Nigel Hopkins and
Gary Chapman. In my experience, you normally
need plans down to at least ‘f’ in such circumstances. But it sailed through and was then ratified
by the IT Steering Board (EVP’s and above).
Here I have to take my hat off to Patrick Naef and I do not know how
long it took Gary Chapman to spot the handy work. Patrick Naef had introduced a project
approval process where the senior business managers across the Group (Portfolio
Review Board – PRB) had to approve all IT expenditure so, theoretically,
Phoenix should have been approved by the PRB.
But version one of the PRB process not only had the obvious lower
threshold of expenditure requiring its approval, strangely it also had an upper
financial limit. This limit was a very
big number, but it was well short of the cost of Phoenix. So the business leaders in the PRB did not
have any sight of the amount or of the project.
Had they done so, I think it would have been a pretty short discussion. At each PRB meeting there were always a chunk
of ‘non-negotiable’ projects (for example those meeting regulatory needs) which
had to be rubber stamped and this then led into the more challenging part of
the meeting which was, in effect, a bidding war for about 80 projects, with the
remaining money enough to pay for just about a quarter of them. Phoenix would have consumed over a quarter of
that money in one go! Not many months
later, someone came up with the bright idea of all projects being presented to
the PRB first, with the very big ones requiring further ratification by the IT Steering
Board. Patrick Naef had no issues with
that proposal. The horse, Phoenix, had
already bolted!
Interestingly, the entire external sales
operation and organisation (Mercator) was left out of scope of Phoenix. Patrick Naef said that Mercator would be subject
to a separate review which would be carried out by Gary Chapman, Nigel Hopkins
and him. It’s a pity, because the
Phoenix team was far too competent to have come up with the selected solution,
i.e. setting totally unrealistic growth targets, continually blaming everyone
who failed to meet those targets, significantly diluting the ability to deliver
solutions to our own businesses, totally mismanaging the acquisition, ongoing operation
and disposal of a third party company, taking seven years to wake up to the
fact that the model was never going to work and then ignominiously abandoning
the entire doomed operation.
Then there was the Phoenix purchase order to
consider, so the Procurement department was asked to place an order with the
supplier. Prior to this, they had not
been involved. The process had bypassed
all formal company procedures so, understandably, Procurement refused to handle
the work. At this stage I not only began
to question Patrick Naef’s judgement but also wondered how long he would
survive in this environment. Patrick Naef
had engaged a company (with which he had previous contact) without a tender,
without any alternative bids, without a plan b and had placed a contract of
enormous value with them. He brushed off
any criticism with his usual aggression and seemed not to care. I still do not know how he managed to get
away with it.
Once the project was underway, there were a
few major milestones and decision points, some needing approval from Gary
Chapman. At a presentation to Gary, the
process for filling posts at the completion of Phoenix was included. A thorny issue was one of grades of the new
jobs. It was going to be a bit difficult
to get them graded before advertising them without putting an unacceptable
delay in the whole process. A simple
solution was to advertise the jobs with ‘grade to be advised’ and we discussed
this the evening before the meeting with Gary.
Of course, HR said this was not possible but Patrick’s argument, which I
agreed with, was that people should apply for jobs because they wanted the job,
not the grade. That was all very well
but meanwhile, back in the real world, especially the real world of the most
hierarchical and grade conscious company I had ever come across, this was
probably not the right time. HR did not
want to do it this way, I did but not yet and Nigel Hopkins did not want to do
it yet either. (This was the topic mentioned
in an earlier update when I had Patrick Naef telling me the following day never
to disagree with him in front of Nigel Hopkins again.) Those of us ‘against’ just felt it was too
much too soon for everyone, particularly as views of management were, at best,
highly suspicious. But Patrick won the
argument and from then on I fully supported him on the matter. When the proposal was put to Gary Chapman, he
seemed to like it. He turned to the HR
ensemble, which included Sophia Panayiotou, and said “And HR fully support
this” I deliberately leave out a
punctuation here as I genuinely do not know whether he was using a full stop or
a question mark. The HR team obediently
nodded. Forgive me for remembering that
exchange when Gary Chapman assured me four years later that “HR fully
supported” my sacking.
The idea of Project Phoenix was supposed to
be ‘by the people, for the people’.
Managers were excluded but I do not think anyone believed for one moment
that Patrick Naef was not pulling the important strings behind the scenes. As part of the approved structure of the
project, Patrick Naef had ‘full and final veto’ and had even gained assurances
right from the top of the company that no decisions made by Phoenix (in effect,
by him) would be overturned. I have to
say that I was impressed that, within six months or so, Patrick Naef seemed to
have wrapped the entire Emirates Group around his little finger.
Although not involved in day to day matters
of the project, there were many opportunities for me to join the team during
the regular progress reports and feedback sessions. I must say I was very impressed with all
aspects of the project. Out team members
were relishing the opportunity and had grasped it with both hands. The consultants were easily the best I had
ever worked with and had created a very relaxed and open working atmosphere which
added to the liberation of my colleagues.
For me, this aspect alone suggested that the project would be a success
– all we had to do was extend this working philosophy into day to day working
and we would have cracked one of the biggest cultural issues we faced. Maybe the consultants were expensive, but
they were good. They also did not do
what almost every company does, that is to use their A team for the sale only
to withdraw it once the project gets underway.
We had the full attention of their A team, but that should not undermine
the other members of their team, as they had good strength in depth. I thoroughly enjoyed, and learnt from, the
time I spent with the project team.
The project duly came up with the necessary
processes and an organisation to support it.
Not all of it suited everyone but it was sound enough for me. I never get too excited about organisation
charts, as long as everyone knows what the customer needs and how they, and
everyone else, can meet those needs, it is fine with me. Filling the vacant management roles was the
obvious challenge. On one hand we were
in a good position with a clean sheet of paper and therefore the ability to
select people with the right management capabilities, but we still had the
fundamental problem – a lack of management skills. There was a lot of emotion expended on the
process, particularly the limit of two applications per person (otherwise the
whole procedure would have taken months) and the need to apply for jobs before
they had been evaluated (a lot of people assumed this was a trap and the result
of the evaluation would depend on who got the job!). Cronyism was hardly wiped out in this single
exercise. After all, the initial project
team itself had been hand chosen so many people felt those individuals had a
head start in securing the new jobs. And
I was told much later that the selection process for some management roles was
polluted by secret pre-selection meetings to decide on so called ‘self’
nominations and therefore to generate contrived candidate lists. But, for the first time, there was at least
an open process of interviews, with several managers and HR involved in each
interview. Yes, the whole process was
more rushed than we would have liked.
Yes, we would have preferred to have given more people more choice. Yes, we made a few appointments that turned
out to be less successful than we would have hoped. But
the biggest problem was a lack of suitable candidates and inevitably we had to
relax our criteria to some degree. In my area, I was left with far too many
vacancies but I drew the line and refused to make the same mistakes that had
been made before. The brutal truth was
that we had to look outside of EG-IT to find the necessary talent. It was not a comfortable position for a
number of people but I stand by my decisions as strongly now as I did
then. I can look everyone in the eye
now, as I did then, and explain my reasoning which was based on my clear
understanding of what the organisation needed.
With so many managerial vacancies, I had to adjust my organisation a bit
to avoid too many empty roles in one management line – when you find that your
boss’s boss is, in fact, yourself, you know you have a problem!
Was the project a success? Yes, it met its objectives and, for every
critic of the outcome, there were many supporters.
Was Phoenix value for money? That is a difficult one as we had no
comparisons to make. It was very
expensive but, if it had fixed IT for the Emirates Group forever, then it may well
have been a good price to pay. And, in
the Patrick Naef land of wasting inordinate amounts of company money, it was by
no means a leader. The direct losses
associated with the Mercator Asia debacle were over double the cost of Phoenix,
possibly treble, maybe more.
Was Phoenix a waste of money? Eventually, most of it was. Much of the process work added and, I assume,
continues to add value but the ground work on the culture was progressively
undone by the fear subsequently generated by Patrick Naef. That fear grew and grew and has spread like a
cancer across EG-IT. Culture cannot be
measured and, even if it were, I am not there now to measure it. But I am consistently informed that the
culture in EG-IT is now significantly worse than it was in 2006. In 2006 I would have said that was not
possible and I cannot stress how upset I am about this situation. Say what you will about Phoenix, at its
completion we had the perfect springboard to turn EG-IT into something that everyone
could have been forever proud of.
And, of course, five years after Phoenix was
completed, the concept was dumped and Patrick started yet another change
project. Presumably, the company was
happy to write the Phoenix ‘investment’ off over the five year period. But that does beg the question why it had
previously expected to squeeze seven, even ten, or more, years’ life out of
infrastructure equipment which other companies would write off over three
years.
Was Patrick right to do it? If I had been in his shoes I would have
waited, focussing on some immediate tangibles first and then doing something
less revolutionary, mainly from in house resources. But then Patrick Naef always was impatient,
too drastic and a big spender. And he
would accuse me of being too slow and too cautious! The fact is that IT in Emirates was in an
absolutely perilous state and, to Patrick’s credit, he actually did
something. From what I could see, no-one
else had done anything for a long time.
But I certainly would not have dismantled the
organisation and made managers re-apply for what were effectively their jobs. I do not think this approach has a place in
any company that purports to care for its people. Yes, changes had to be made but it must be
remembered that it was the company that put managers into their jobs. And it’s pretty weak, in my opinion, to hide
behind the fact that “it was before my time”.
We had a problem of the company’s making and, as senior managers of that
company, now had to deal with it, in a professional manner. ‘Professional’
means taking care of the individuals, as well as the company, and that takes
time. Time to improve if possible, time
to find a suitable alternative if not.
But Patrick Naef’s only answer to any individual not performing to his required
level is to remove them from their jobs.
And Phoenix did this for him perfectly and he did not even have to get
his hands dirty.
And then Patrick Naef did it again, this time
it was under the Shaheen banner. In 2006
we could perhaps have been justified in distancing ourselves from the
organisation that we had inherited. But
once the Phoenix organisation was in place, we had no excuses as the
organisation was ours. From when I
joined, right up until I was kicked out in September 2010, I take
responsibility for all appointments made right across EG-IT. Clearly I was not directly involved in every
selection, but I endorsed the processes and I personally drove them forward and
monitored them as actively as I could. I
ensured that as much openness and fairness was applied at all levels. And
Patrick Naef fully supported and endorsed those processes and appointments
(some, as always in the real world, being more successful than others), as did
every member of the IT Executive. If
anyone has ever tried to paint a different picture, they have not been truthful.
And we must remember that there was a large
redundancy exercise in EG-IT in 2009. I
managed that exercise myself and made sure that it was as fair and as
comprehensive as I believed possible. We
had a target, but we did not have a limit.
Everyone in the EG-IT organisation when Project Shaheen was initiated
had either emerged from Phoenix or had been recruited/appointed using robust
processes which had been totally supported by every member of the IT
Executive. Those in post during 2009 had
also survived a rigorous redundancy selection process which, again, was signed
off by the full IT Executive. It is therefore clear that Project Shaheen (which
resulted in staff being removed by yet another round of assessments) was applied
to an organisation for which Patrick Naef (as leader of the IT Executive) had
total ownership and responsibility. Nobody had been inherited, everyone was the
responsibility of Patrick Naef. Do not
ever allow him to present the situation differently.
Like everyone, once Phoenix was completed I
was keen on getting on with matters.
Apart from making the organisation work, we had many other issues to
deal with. In my area there were massive
problems to address, but we had already made good inroads on the service
front. Outages had been reduced by a
massive two thirds in the first six months, but I was annoyed when this, along
with further subsequent improvements which were made, was claimed by the
consultants to be the result of the change project. Phoenix achieved a number of things, but it
had no hand whatsoever in the drastic improvement of operational service in
2006-7. All that was achieved by a lot
of hard work by the network and data centre teams replacing hardware (some of
which Patrick had already initiated) and embracing a much more disciplined
approach to change, risk and incident management. Some benefits were realised before Phoenix
even started, most before the project finished.
I complained to Patrick, but he just shrugged it off which was hardly
surprising as it was part of a European road show to enhance his, as well as
the consultants’, reputation.
After the project was completed, Patrick Naef
regularly came up with initiatives that he wanted to implement and these were a
constant drain and undermined the Phoenix work.
Most of them were theoretically good ideas, many impractical and some wholly
inappropriate. For instance, he wanted
to introduce a ‘high performance culture’.
Well, we all supported the idea of an ordered meritocracy, I even had a
tool that I had used elsewhere which we could (and did) implement to support the
idea. But, to give the whole thing a bit
more edge, Patrick Naef wanted to “fire the bottom 5%” every year and replace
them with external recruits! Here we
were, with a vacancy level of around 20%, struggling to find suitable
candidates, beginning to recover from the huge disruption of Phoenix, with
everyone’s trust in management still virtually non-existent and Patrick wanted
us to pick off the weakest 5% in an annual cull. One of Patrick Naef’s problems is that he
reads about initiatives that have worked elsewhere and then tries to implement them
immediately, regardless of whether they are practical, ethical or even
needed. Fortunately, it was not only me
on the IT Executive who had major problems with the 5% idea. This was just one example (of many) of how
Patrick Naef wanted to manage EG-IT, with an approach which could only (and
did) systematically undermine all the benefits that came out of Phoenix.
Of course, internally, Patrick Naef never
formally put his name to Project Phoenix.
Everyone knew it was his baby, but he actually delegated the task of its
Project Management. I was told that this
was recognised at the time in some business circles as “firing his gun from
someone else’s shoulder”. After the
project’s completion, there was no doubt it was Patrick Naef’s project during
presentations at external industry gatherings.
However, when a government audit report concluded that the project had
been poor value and had been commissioned outside of company procurement
procedures, Patrick Naef took a very swift backward step. I do not know exactly when that audit was
published but I do know that just a few days after Patrick Naef told me about
it, he suddenly gave the project a mention (not related to the audit) in his
weekly staff update, taking the opportunity to remind everyone who the project
manager was (i.e. not him!). Maybe it
was just a coincidence, but the timing was interesting as the project had not
been mentioned for a long time, nor was it mentioned again for a long time. I do not know how many of you have seen the
illusionist and trickster Derren Brown in action, but most audiences find his
ability to fool people quite extraordinary.
Personally, having worked with Patrick Naef, I cannot say that I am ever
particularly overwhelmed by Mr Brown.
I will finish this off with a short quiz to
test readers’ understanding of what a high performance culture really means in
EG-IT.
1. How
many people who expressed concern about, and/or refused to paint a false
picture of the benefits of, the ‘Mercator Asia’ acquisition kept their jobs?
2. How
many people who fully supported and commended that acquisition lost their jobs?
3. How
much money was wasted by the Emirates Group on the acquisition, operation and
disposal of Mercator Asia?
I will provide a clue – the answers contain a
heck of a lot of zeros!
But, for me, the most intriguing question is
– what has Gary Chapman been told the answers are?
