[First posted on Who Killed Howard Johnson by Jerry Gregoire.]
I've worked as a contract SysAdmin in a bunch of places in decline - and found I had a knack for pulling the I.T. systems out of crisis. Only as much as one admin can, and only for the short time of my contract.
After the third or fourth company, I realised they are easily recognised... Literally in under 5 minutes!
Just how, later on.
Why do I claim I.T., especially IT Admin/Operations, is an accurate 'mine budgie' or early warning system of a company's 'death spiral'?
I.T. reaches through every part of a business and it's processes. In 2007, after 55 years of commerical I.T., it is still the single biggest point of leverage for most, if not all, businesses. Returns are 3-4 times higher than anything other investment according to a recent NYTimes survery/article. Full ITIF Report - 69pp PDF.
Not only is I.T. everywhere and embodies most of the business processes, but it radically improves staff productivity over the whole business. And once large companies have been successful, with slick (enough) processes and highly productive/effective staff. As well, the management has been at least "good enough" and probably much better.
I.T. Ops and Admin is where the rubber hits the road - where all those glorious I.T. benefits and 'force multiplier' effects are delivered. It is also the poor cousin of everything. Every year savings can be made by not fixing stuff, not replacing old equipment, not upgrading networks, power and cooling. And by reducing 'unnecessary' staff - a few each year. Budget death by a thousand slashes. At TNT, the transport company, they had reduced I.T. staff by 75% after two massive sackings on top of incremental layoffs - and were then building the numbers back up. Of course, the staff that are most needed will be gone early. The best staff are never the last left.
First the staff fall behind a bit on system updates, then doco, then they are busy fixing faults, and then they are flat-out firefighting - and all the time the systems are degrading and efficiency and effectiveness across the whole business is slowly eroded. More large scale outages, each a 'one of a kind', occur. Capacity may fall behind demand, but if the business is shrinking maybe not :-) The 'meta-functions', like Problem/Change/Config management, that prevent or fix problems before they occur and increase I.T. staff productivity, improve system stability and improve Service Quality are long gone or completely subverted.
It takes a while, and it is never anyone's job to detect and correct these global/systemic effects. Nobody measures the 'business benefits' of IT systems - so how could the decline in them be noticed?
And then they hire me - one in a long line of contractor admins - because their Ops staff turnover rate has gone through the roof and they either have young/inexperienced staff or "retired-in-place" barely competents or malapprops. The extra cost of contractors forces budget costs elsewhere, like maintenace, to be shaved - and the spiral intensifies.
The irony here is that all those companies have big, expensive I.T. projects rolling out systems "that will solve all problems" when they deliver. Of course they rarely finish, under-deliver and make no difference... That is, if they could even run on the infrastructure.
Be clear here - the decline of I.T. Operations is a symptom, not a cause, of the Corporate Death Spiral.
What are the symptons, this 'stench of Corporate Death', in the I.T. Admin and Operations area?
- frantic, busy staff - often literally running
- phones ringing off the hook
- fragile, antiquated systems
- messy cabling, machine rooms, lunch rooms
- unlabelled equipment, cabling, media
- backlogs of urgent jobs
- no progress on non-urgent tasks
- one or two 'technical despots'
- disrepectful inter-staff communication
- distant bosses. Don't talk down the heirarchy, only up.
- Only reprimands and upbraiding about mistakes and poor performance.
- No budget for process improvement. Includes refusing any staff initiative, no matter how well justified.
- Poor, incomplete record keeping. Especially software licences.
- Absence of teams and team meetings.
- Trivial, irrelevant incidents escalted to highest levels. Major incidents ignored.
- no staff 'think time', poor designs, ...
- no doco, no systems guides/maps, ...
- no audit or review processes, even informal
- no revision control, config mgt.
- absent or poor backups
- no induction or new starter processes
- lax security and widely shared administrator passwords
- and lots 'perks' for the annointed few.
Why is this so?
What's my take on the specifics of 'poor management' that produce these results?
Obviously complacency and smugness/self-statisfaction can contribute.
What's essential is an upper management 'team' who don't work as a team and where the magic triplet, "ignorance, arrogance and self-delusion/incompetence" reigns.
Ordinary people are very pragmatic - they need to feed their families and pay the bills. If the Corporate Culture says "don't rock the boat", "Bring No Bad News" or "Do Nothing New" - then they won't. And there's nothing in their job description that says they must or should.
Robert E. Kelley identified "The 10 strategies of Star Performers" - the first of which is initiative. Kelley notes 'the stars' are 10-20 times more effective than the average - these are people you really don't won't to lose.
A culture that censures change & improvement is anathema for initiative - these people leave - forcibly or volantarily.
And all these organisations are "highly political", especially the management.
What's political?
When individuals take decisions where there is a conflict between the company interests and their personal interests, whose interests prevail? 'Political' is taking decisions that result in personal benefit at the expense of the Company.
This can be as simple an innocuous as maximising your Frequent Flyer benefits.
In extreme cases, personal benefits prevail even in highly visible/public decisions - like buying expensive, unnecessary luxuries or extravagent displays of wealth/conspicuous consumption.
All the worker drones know the place is on the skids, sigh and just plod along dispiritedly. A few may even emulate the 'leaders' and take whatever they can get...
Not only is worker morale low, this will be a Blame Culture.
Human Beings are brilliant at Game Playing - everyone works out very quickly that it's a far, far better thing to Do Nothing, than to have ever tried and 'failed', even minutely. This bring the wrath of the Blame Daemons raining down on your head, and a indefinite posting to Alaska.
This is expressed as:
- meetings, bloody meetings. "The practical alternative to work and decision making".
- Decision by Committee - no-one's to Blame. Everything is compromised.
- Excessive/needless Bureaucracy.
- Micro-Management and extensive management 'correction/editing' of all work.
- Focus on Irrelevant and Trivial issues, even pursuit of hobbies/play at work.
- Extensive use of 'blame hound' consultants , with no follow-up/follow-through on recommendations/reports.
- Rampant Fifedoms, bullying, harassment and abuse.
- Re-organisations, massive layoffs, cost-cutting drives. Especially irrational ones.
- Ceasation of research and marketing "to reduce costs"
And when did you ever see a managers primary output, decisions, ever reviewed and assessed?
It just "Not done"!
And if it was, there would have to be a system to create 'consequences' - and that requires strength of character and resolve. Which, if it was common in the culture, would not have allowed the situtation to develop in the first place.
Lou Gerstner, who turned IBM around after 1991, talks about 'Execution' as critical.
Death Spiral companies can't 'Execute'. Neither do they have the will to execute.
Poor Management is the unwillingness to take decisions, execute plans, hold others accountable, seek and listen to feedback, communicate clearly and fully, foster and encourage teams, focus time/energy on business, not personal, issues and to consistently place company interests above personal benefits.
All this shows up quickly in I.T. Operations, 'the dispensible cost centre'.
And the Death Spiral builds, feeding on itself until it's impossible to pull-out. Every pilot knows about this!
When disaster is inevitable, the brave rational thing to do is salvage whatever is possible, take a nice severance cheque and pass what's left to new people. What often happens is "Crash and Burn" - nothing left, awful losses and many injured innocents.
More...
Summary...
