Jerry Landsbaum's Metrics - Measuring and Motivating Maintenance Programmers

Jerry Landsbaum wrote the book of his life in 1991/2. It reported on about 6 years of work at Monsanto - where they took a 'backwater' - the CIS maintenance programming area where new work was 'frozen' - and not only kept it viable, but got their management to appreciate them and their efforts.

Quote (p58, Section 3.3: What we Accomplished)

If we had no improvement in productivity, and had simply experienced the increases in salaries over that period, the cost to support a module in the fifth year would have been $1,232.

If we multiply this by the number of modules supported in Year 5, the budget would have been $11,400,000. This amount less the actual budget of $3,200,000 is a saving to Monsanto of $8,200,000. This alone proved our worth to our management, but we went further.

As a result of giving the staff greatly increased accountability, they generated over $600,000 in savings due to various projects they initiated.

CIS evaporated when Monsanto was taken over.
Jerry retired then and pursued his love of Jazz Music before passing away in 1999.

Jerry's work life started as a Mechanical Engineer for a railroad company - he knew what it took to deliver safe, sound, reliable and cost-efficient systems. He understood that Quality is a mindset, that it pervades every activity and comes before anything else. There's no point in building something if it's not going to stand up! There's no point in writing programs if they don't produce correct results. There's no point in offering business systems if they're not available when they're needed...

Jerry defined a set of metrics, in business terms, they provided to their management in their annual report. Excerpts follow.

Section 3.3 "What we Accomplished"

PATHVU Software Complexity Scores

	Average Score
	Complex	Architecture	Maintain	No Pgms	Average Lines of Code
Year 5	104	93	157	3521	1372
Year 6	96	92	145	4482	1465
Percent change	-7.6%	-1%	-7.6%	27.00%	6.80%
MSA adds	76	109	145	456	1190
New programs	76	50	100	515	1500
Deletes				10
AOB Year 5	190	127	235	174	3,418
AOB Year 6	158	97	191	198	3.25
Percent change	-17.00%	-24.00%	-19.00%	14.00%	-5.00%

---

CIS Statistics

Basic Statistics

	Year 4	Year 5	Year 6
Programmer-Analysts (P-A)	35	31	31
Management	6	6	5
Secretarial	3	2	2
Total staff size	44	39	38
Expenses, w/o 0/H(2), $MM/year(3)	$3.80	$3.50	$3.20
Overhead, $MM/year	NA	$1.80	$0.80
Total expenses	$3.80	$5.30	$4.00
Number of modules supported	8,950(4)	9,250	9,775
Number of programs supported	4,300	4,950	4,950
MM lines of code	5.4	5.4	5.75
$MM of inventory @ $25	$135	$135	$144
Average lines per module(5)	604	584	590
Programs moved to production	4,400	5,525	6,760
Compiles/year, batch	40,000	30,163	25,342
Compiles/program/year, batch	9.3	6.1	5.1
Batch processings/year	45,000	43,763	59,864
Abends/year, batch(6)	2,400	1,715	1,971
Contractors/professionals(7)	0.4	0.4	0.2
Systems supported	106	100	90
Jobs supported	2,100	2,250	1,825
On-line programs	1,000	1,200	1,255
Transactions/year (MM)	39.6	45	46.5
Repairs (emergency)	1,400	1,000	977
Enhancements	1,065	1,000	732
Consultations	8,100	11,400	10,377
Staff utilization percent	64.90%	73.80%	69.70%

2 w/o 0/H = without Overhead (the cost of upper management, flowed to all appropriate departments below).
3 MM = Million.
4 1 stack of paper 67 feet high, stretches out to 25 miles.
5 Modules = programs + .JCL files + data files.
6 Abend = abnormal program termination (did not run right)
7 Professionals = Programmer-Analysts + Managers.
8 P/A = Programmer-Analyst.

---

Quality Indicators

	Year 4	Year 5	Year 6
Abends/batch production run	0.053	0.039	0.033
Abends/month/production module	0.023	0.015	0.017
Abends/month/programmer-analyst	5.7	4.6	5.3
PATHVU maintainability score	NA	157	145

---

Productivity Indicators

	Year 4	Year 5	Year 6
Modules supported/staff member	203	237	257
Batch processings/P-A(8)	1286	1412	1931
Transactions/year (MM)/P-A	1.13	1.45	1.5
Repairs (emergency)/P-A	40	32	32
Enhancements/P-A	30.4	32.3	23.6
Consultations/professional	198	308	288
Production moves/module	0.5	0.6	0.7
Programs to production/month/P-A	10.5	14.9	18.2
Compiles per P-A, batch	1,150	973	817
Compiles/program moved, batch	9.1	5.5	3.7
Maintenance cost/year/module	$425	$378	$327
$MM inventory/professional	$3.3	$3.6	$4.0

---

Basics - Percentage Changes

	Year4/Year5	Year5/Year6	Year4/Year6
Programmer-Analysts (P-A)	-11.40%	0.00%	11.4%
Management	0	-16.7	-16.7
Secretarial	-33.3	0	33.3
Total staff size	-11.40	-2.6	-13.6
Expenses, w/o 0/H, $MM/year	-7..9	-8.6	-15.8
Overhead, $MM/year	NA	~-44.4	NA
Total expenses	NA	-24.5	NA
Number of modules supported	3.4	5.7	9.2
Number of programs supported	15.1	0	15.1
MM lines of code	0	6.5	6.5
$MM of inventory @ 825	0	6.7	6.7
verage lines per module	-3.3	-1.0	-2.3
Programs moved to production	25.6	22.4	53.6
Compiles/year, batch	-24.6	-16.0	-36.6
Compiles/program/year, batch	-34.4	-16.4	-45.2
Batch processings/year	-2.7	36.8	33
Abends/year, batch	-28.5	14.9	-17.9
Contractors/professionals	0	-50.0	-50.0
Systems supported	-5.7	-9.0	-14.2
Jobs supported	7	-18.9	-13.1
On-line programs	20	4.6	25.5
Transactions/year (MM)	13.6	3.3	17.4
Repairs (emergency)	-28.6	-2.3	-30.2
Enhancements	-6.1	-26.8	-31.3
Consultations	40.7	-9.0	28.1
Staff utilization percent	13.7	-5.6	7.4

---

Quality Indicators - Percentage Changes

	Year4/Year5	Year5/Year6	Year4/Year6
Abends/batch production run	-26.4	-15.4	37.7
Abends/month/production module	-34.8	13.3	-26.1
Abends/month/programmer-analyst	-19.3	15.2	-7.0
PATHVU maintainability score	NA	-7.6	NA
Modules supported/staff member	16.7	8.4	26.6
Batch processings/P-A	9.8	36.8	50.2
Transactions/year (MM)/P-A	28.3	3.4	32.7
Repairs (emergency)/P-A	-20.0	0	-20.0
Enhancements/P-A	6.3	-26.9	-22.4
Consultations/professional	55.6	-6.5	45.5
Production moves/module	20	16.7	40
Programs to production/month/P-A	41.9	22.1	73.3
Compiles per P/A, batch	-15.4	-16.0	-29.0
Compiles/program moved, batch	-40.0	32.7	59.3
Maintenance cost/year/module	-11.1	13.5	23.1
$MM inventory/professional	9.1	11.1	21.2

CIS Annual Report

II. MANAGEMENT SUMMARY

A. Organization Profile and Metrics

Staff	Monthly average
Managers	6.25
Professionals	33.50
Secretaries	2.25
Co-ops	4.75
Contractors	12.00
Per Diem	1.00
Total	59.75

---

Staff movement	for the year
Promotions, In	4
Promotions, Out	1
New Hires	5
Lateral, Out	10
Lateral, In	4
Terminations	3
Co-op Terms	10
Charge-out Rate	96.60%

B. Business Profile and Metrics

	Quantity
Major Business Groups Supported	65
Systems	100
Jobs	2,250
Programs	4,950
All Modules	9,250
Lines of Code	5,400,000
Replacement Cost	$135,000,000
On-line Transactions	45,000,000
Asset Dollar Responsibility	$26,000,000,000

C. Business Results and Metrics

	Number of Occurrences	Man-days	Dollars
Emergency Calls	1,000	600	$270,000
Consultations	11,000	800	$370,000
Miscellaneous items	200	1,050	$560,000
Enhancements	1,000	4,200	$2,000,000
Totals	13,200	6,650	$3,200,000
CISS-initiated Cost Savings	33		$570,000

More...

Summary...

When companies fail - The Stench of Corporate Death.

[First posted on Who Killed Howard Johnson by Jerry Gregoire.]

I've worked as a contract SysAdmin in a bunch of places in decline - and found I had a knack for pulling the I.T. systems out of crisis. Only as much as one admin can, and only for the short time of my contract.

After the third or fourth company, I realised they are easily recognised... Literally in under 5 minutes!
Just how, later on.

Why do I claim I.T., especially IT Admin/Operations, is an accurate 'mine budgie' or early warning system of a company's 'death spiral'?


I.T. reaches through every part of a business and it's processes. In 2007, after 55 years of commerical I.T., it is still the single biggest point of leverage for most, if not all, businesses. Returns are 3-4 times higher than anything other investment according to a recent NYTimes survery/article. Full ITIF Report - 69pp PDF.

Not only is I.T. everywhere and embodies most of the business processes, but it radically improves staff productivity over the whole business. And once large companies have been successful, with slick (enough) processes and highly productive/effective staff. As well, the management has been at least "good enough" and probably much better.

I.T. Ops and Admin is where the rubber hits the road - where all those glorious I.T. benefits and 'force multiplier' effects are delivered. It is also the poor cousin of everything. Every year savings can be made by not fixing stuff, not replacing old equipment, not upgrading networks, power and cooling. And by reducing 'unnecessary' staff - a few each year. Budget death by a thousand slashes. At TNT, the transport company, they had reduced I.T. staff by 75% after two massive sackings on top of incremental layoffs - and were then building the numbers back up. Of course, the staff that are most needed will be gone early. The best staff are never the last left.

First the staff fall behind a bit on system updates, then doco, then they are busy fixing faults, and then they are flat-out firefighting - and all the time the systems are degrading and efficiency and effectiveness across the whole business is slowly eroded. More large scale outages, each a 'one of a kind', occur. Capacity may fall behind demand, but if the business is shrinking maybe not :-) The 'meta-functions', like Problem/Change/Config management, that prevent or fix problems before they occur and increase I.T. staff productivity, improve system stability and improve Service Quality are long gone or completely subverted.

It takes a while, and it is never anyone's job to detect and correct these global/systemic effects. Nobody measures the 'business benefits' of IT systems - so how could the decline in them be noticed?

And then they hire me - one in a long line of contractor admins - because their Ops staff turnover rate has gone through the roof and they either have young/inexperienced staff or "retired-in-place" barely competents or malapprops. The extra cost of contractors forces budget costs elsewhere, like maintenace, to be shaved - and the spiral intensifies.

The irony here is that all those companies have big, expensive I.T. projects rolling out systems "that will solve all problems" when they deliver. Of course they rarely finish, under-deliver and make no difference... That is, if they could even run on the infrastructure.

Be clear here - the decline of I.T. Operations is a symptom, not a cause, of the Corporate Death Spiral.

What are the symptons, this 'stench of Corporate Death', in the I.T. Admin and Operations area?

- frantic, busy staff - often literally running
- phones ringing off the hook
- fragile, antiquated systems
- messy cabling, machine rooms, lunch rooms
- unlabelled equipment, cabling, media
- backlogs of urgent jobs
- no progress on non-urgent tasks
- one or two 'technical despots'
- disrepectful inter-staff communication
- distant bosses. Don't talk down the heirarchy, only up.
- Only reprimands and upbraiding about mistakes and poor performance.
- No budget for process improvement. Includes refusing any staff initiative, no matter how well justified.
- Poor, incomplete record keeping. Especially software licences.
- Absence of teams and team meetings.
- Trivial, irrelevant incidents escalted to highest levels. Major incidents ignored.
- no staff 'think time', poor designs, ...
- no doco, no systems guides/maps, ...
- no audit or review processes, even informal
- no revision control, config mgt.
- absent or poor backups
- no induction or new starter processes
- lax security and widely shared administrator passwords
- and lots 'perks' for the annointed few.

Why is this so?
What's my take on the specifics of 'poor management' that produce these results?

Obviously complacency and smugness/self-statisfaction can contribute.

What's essential is an upper management 'team' who don't work as a team and where the magic triplet, "ignorance, arrogance and self-delusion/incompetence" reigns.

Ordinary people are very pragmatic - they need to feed their families and pay the bills. If the Corporate Culture says "don't rock the boat", "Bring No Bad News" or "Do Nothing New" - then they won't. And there's nothing in their job description that says they must or should.

Robert E. Kelley identified "The 10 strategies of Star Performers" - the first of which is initiative. Kelley notes 'the stars' are 10-20 times more effective than the average - these are people you really don't won't to lose.
A culture that censures change & improvement is anathema for initiative - these people leave - forcibly or volantarily.

And all these organisations are "highly political", especially the management.

What's political?
When individuals take decisions where there is a conflict between the company interests and their personal interests, whose interests prevail? 'Political' is taking decisions that result in personal benefit at the expense of the Company.
This can be as simple an innocuous as maximising your Frequent Flyer benefits.
In extreme cases, personal benefits prevail even in highly visible/public decisions - like buying expensive, unnecessary luxuries or extravagent displays of wealth/conspicuous consumption.

All the worker drones know the place is on the skids, sigh and just plod along dispiritedly. A few may even emulate the 'leaders' and take whatever they can get...

Not only is worker morale low, this will be a Blame Culture.
Human Beings are brilliant at Game Playing - everyone works out very quickly that it's a far, far better thing to Do Nothing, than to have ever tried and 'failed', even minutely. This bring the wrath of the Blame Daemons raining down on your head, and a indefinite posting to Alaska.

This is expressed as:
- meetings, bloody meetings. "The practical alternative to work and decision making".
- Decision by Committee - no-one's to Blame. Everything is compromised.
- Excessive/needless Bureaucracy.
- Micro-Management and extensive management 'correction/editing' of all work.
- Focus on Irrelevant and Trivial issues, even pursuit of hobbies/play at work.
- Extensive use of 'blame hound' consultants , with no follow-up/follow-through on recommendations/reports.
- Rampant Fifedoms, bullying, harassment and abuse.
- Re-organisations, massive layoffs, cost-cutting drives. Especially irrational ones.
- Ceasation of research and marketing "to reduce costs"

And when did you ever see a managers primary output, decisions, ever reviewed and assessed?
It just "Not done"!
And if it was, there would have to be a system to create 'consequences' - and that requires strength of character and resolve. Which, if it was common in the culture, would not have allowed the situtation to develop in the first place.

Lou Gerstner, who turned IBM around after 1991, talks about 'Execution' as critical.
Death Spiral companies can't 'Execute'. Neither do they have the will to execute.

Poor Management is the unwillingness to take decisions, execute plans, hold others accountable, seek and listen to feedback, communicate clearly and fully, foster and encourage teams, focus time/energy on business, not personal, issues and to consistently place company interests above personal benefits.

All this shows up quickly in I.T. Operations, 'the dispensible cost centre'.

And the Death Spiral builds, feeding on itself until it's impossible to pull-out. Every pilot knows about this!

When disaster is inevitable, the brave rational thing to do is salvage whatever is possible, take a nice severance cheque and pass what's left to new people. What often happens is "Crash and Burn" - nothing left, awful losses and many injured innocents.

More...

Summary...

CMDB - Not a DB, not 'definitive master'

Ok - this was a real insight to me. Hope it's not Old News :-)
[25-Jan/15-Feb-2007. This PDF from the "CMDB Federation [BMC, CA, Fujitsu, HP, IBM and Microsoft] supports a similar view - "Federated CMDB's".]

Was talking to a friend today who does config, change & release mgt for a large, secure Government Department.
He's struggling with Data Quality issues - surprise, surprise.

They import equipment details from a logistics system, they do a *lot* of shipping to/from 100+ overseas offices.
It doesn't enforce strict product id's - there could be 20 different versions of a single PC product name.

But talking to him further, it's axiomatic that CMDB's will not and cannot ever be the master or definitive source of all information.

CMDB's are, by definition, a join of existing, disparate databases mastered by other products/applications.
The only thing they can be is an 'intelligent collective repository [just made that up].

It reminds me of comments by Jerry Gregoire

When he was CIO of Dell he said :
"you don't want to use an ERP" - takes away your distinctive business process, costs to get in and out of, and ties you into one monster database. And slows you down - you can't change your business quickly if the ERP vendor doesn't support your new function/process.

Dell used an object broker and connected into existing apps & databases.
They implemented a brand-new Inventory system with *zero* new databases. The place has gone downhill since he's left - I sometimes wonder if that's linked.

Jerry also pens pretty straightforward advice which I heartily agree with.

More...

Summary...

Why is learning ITIL so hard?

Back from the first week of ITIL Service Managers' Training... Took me 4 days to recover - part of which could be the driving [only 7-8 hours each way].

Realised I needed email access whilst away - and my Palm PDA with 802.11 wireless doesn't cut it for email. Have acquired a laptop, and am creating 'dual boot' setup. Don't trust MS-Windows - especially those in Internet Cafes. Need 'ssh' to access mail.

So why was I exhausted?

Six of us doing the course - and all of us suffered the same. Lack of sleep, 'exam nerves' each day and extreme psychological reaction. At least a number of us seriously thought & discussed ditching the course - a seriously expensive move.

I don't have a good reason...

Everyone [all men] found the experience "intense". We are all used to change, acquiring new information, reading long tracts, writing, solving problems, creating/giving presentations and attending talks/lectures... And doing the odd test.

It's not like the ITIL material is 'deep' or 'difficult' like Queueing Theory [thanks Neil!]
It is broad - there is a lot to cover. Not that many Powerpoint slides [50 a day?]

Still don't know why I came back so wrung out. Not sure if that's a universal experience.

At this point, just have to take a note of the effect and look for other stories/experiences - and keep pondering over it.

More...

Summary...

The end of the "Silicon Revolution"

This seems to be one of the biggest IT stories not making news and not being actively addressed by Professionals and the Industry.

Neil Gunther writes about the change in the Moore's Law CPU speed constant. Which is why we have "multi-cores"... In 2000 and 2001, Intel released articles flagging thermal effects could be the next barrier - and predicted in 2010 a single CPU consuming 18 kilo-watts. More than ten times the average household power consumption!

Herb Sutter in The Free Lunch is over makes an aside, illustrated with a graph, on the inflection in CPU speed growth curve - in January 2003. Herb was talking about the insidious problems that true, ubiquitous concurrent programming incur.

Commercial IT has been going more than 55 years. It is becoming quite mature. The next big event horizon is end of the Silicon Revolution - when all the physical limits are met for CPU's (speed), memory size, disk size/speed/transfer rate and network bandwidth.

What will over IT Services look like then? IT groups will no longer be able to rely on the back of the rampant technology improvement. They will have to work, hard, to keep improving their figures.

Design will matter. Real talent, skill and understanding will become important. When large companies are spending 12-15% of their Operating Expenses on IT, the ones that can maintain service levels and business effectiveness and shave 1-3% off costs will have a substantial competitive advantage. The savings go straight to the bottom line, adding directly to Nett Profits.

Nett Profits usually lie between 1% - 10% of turnover. The IT savings above will boost whole company profits by 5%-30%.
Which will impress the market analysts.

This definition of Engineering gives the reason: "An Engineer does for a dollar what any fool can do for 10."

More...

Summary...

How does the "2% Rule" apply to IT and ITIL?

ITIL is about aligning IT with the business needs. IT acts as an internal business supplying customers - who may be captive.

Sometimes IT is the "2% Rule". Price can be (nearly) no object.

People may need a business result and price is not the main criteria. Recognise these situations and react accordingly.

The "utilisation" and "efficiency" of particular IT assets is not of prime importance - the Total Business Result is.

That's why we don't fret over CPU under-utilisation on desktop computers. And why we do fret over 'poor response' on those same machines.

That 'captive audience' of yours can fire you as a supplier: it's called "outsourcing".

It's all about 'perspective'.

More...

Summary...

The 2% Rule, or 98:2 Solution

You know the '80:20 Rule' - the Pareto Principle - that 80% of faults are caused by 20% of problems. Here's a similar rule: Items under 2% of budget get different rules.

Every organisation has it's "core business" and will/should actively monitor and control those inputs/supplies/services to remain profitable. They will be very sensitive to their major selection criteria on those.

But for the 'incidentals', different economics and criteria apply.

As a supplier or consumer, recognising which situation applies will help you greatly by being able to tailor your services to the consumers needs and maximise your profits.

If you're a builder and you can buy the same building products for 20% less with no other penalties, then there has to be a very strong reason to not change. Inertia isn't a strong business reason. "Family connections" probably are.

But what about those inputs/supplies/services that you don't use every day? The one and two percenters? The necessary 'noise'.

My plumbers charge $160/hour. If they need some printing done occasionally, how much time can be spent looking for 'the best deal'? The savings have to beat $160/hr spent. Taking a half-day to save 20% on a $1,000 printing job is a nett loss of $450!. (gain: $200, cost: 4*$160 = $640)

For the "1-2%" inputs, cost price is the least important determinant. Total Cost, including opportunity losses/forgone revenue etc, has to be used for a realistic economic comparison.

For the little things, the incidentals, most people put first one of:

• quick or available
  
• good or high-quality
  
• close
  
• reliable

For some people, it is always about the price. People on fixed-incomes are "time rich, money poor". They usually fall into this category. Others may be wealthy and always be "careful" or "tight" - there are no set rules for behaviour.

When the hotwater system has failed or the roof has flooded, you need someone Now! Someone who's going to do a Good Enough job, Real Soon. Even if they charge double you're probably happy to pay the money. The downside is not trading for one or more days - way more expensive.

If it's a service, like an accountant, that you are going to be using over an extended period and it's critical to your business, you may take some time over the decision and be very particular in your criteria and trade-offs. It's worth an hours' travel and 30% more to get the best advice and save a lot more!

The "2% Solution" has two business impacts:

• In your marketing analysis, decide how much of your business is not decided on price alone - and build package of service and price accordingly.
  
• Relationships are why do/don't return for repeat business. It's also why people ask for and give 'recomendations'.

The "2% Solution" can also inform what business segment you decide to be in.
If you are an Engineering and Construction firm and specialise in (steel) piplelines, you can choose to enter the "high volume/low margin" end of the business - supplying and laying the long straight bits, or enter the "low volume/high margin" end - building the complex valve/joiner units.

You can make good money at both ends.
Low Margins arise because of fierce competition - everyone can do the technical side.
High Margins are allowed when there is little competition - because the technicalities of the job are demanding/exacting.
Both ends need good management and tight fiscal control to remain profitable.

Technology has the horrible habit of quickly making the esoteric into the ordinary - being a technology leader as a point of differentiation means you can't stay still.

In 1991, it cost $10,000 for a CD-Writer and about as much for a disk that would store those 600Mb.
15 years later, writers were under $100 and disks over 20 times bigger for $150.

More...

Summary...

Mythical Man Month II

Brooke's Law doesn't include scaling effects. They're geometric. [There is another effect I can't yet name that kicks in for very large projects so they can never complete. This isn't just an asympote, but a maximum - total output reduces the bigger you make the project team without relaxing the time constraint.]

Here's the heuristic scaling law:
To double output, triple the resources.
[No evidence or research to support this.]

If you have 10 units of work to do and your staff of 6 can do it in 3 months, to produce 20 units of work in 3 months, you need 3 teams of 6 [modulo their effectiveness and project initiation time]. For 40 units, 9 teams, 80 units - 27 teams etc.
Alternatively, what the team can do in 24 months, might be possible in 3 months with 27 times more resource - or around 8 times the total cost, if it ever completes. [I don't have any ideas on the increase in unreliability of the estimates or the 'Project Risk' increase.]

Sometimes "Fast Tracking" may be necessary... But hugely expensive - and where are you going to find around 25 times as many people???

The Standish Group have been performing Primary Research (surverys) of I.T. in the USA since 1994. They've accumulated a large amount of data and 50,000 project case studies.

Their "Recipe for Success" (for I.T. projects) is simple - six people for six months.
It seems to confirm this hypothesis.

"Cheap, Fast, Good - pick any two".


From an article in "Software Magazine", Here's the list of Standish's Top Ten:

Recipe for Success: CHAOS Ten  
 
  Confidence Level Success Factors  
  Executive support  18  
  User involvement  16  
  Experienced project manager 14  
  Clear business objectives 12  
  Minimized scope   10  
  Standard software infrastructure 8  
  Firm basic requirements  6  
  Formal methodology  6  
  Reliable estimates  5  
  Other criteria   5

More...

Summary...

The Mythical Man Month Revisited

Fred Brookes wrote a definitive book, "The Mythical Man Month" on the management of large projects. He reported the development of the Operating System for the (then) revolutionary "IBM/360" around 1960.
His main conclusion is usually stated as: "Adding more programmers to a late project only makes it later."

Take together the last set of posts on programming/I.T. productivity factors:
Teams, Staff 'Morale/Attitude', Star Performers, Nett Negative Producers and Ineffective Management.

All the factors multiply together - the range of productivity, even for the same staff, is tremendous.
How can sensible "Man Month" estimates be made in these conditions?


Brooke's Law can be explained by a computer analogy:
It's a 'cache coherency' problem. The memory of the existing staff is being copied to the new staff - both groups cannot produce anything until the transfer is complete.

Projects and Admin/Operations tasks can't be forecast using a "mean average person" - individual forecasts have to be made for the productivity of the specific person, team, project, environment... And if you are "Going Boldly Forth where No Programmer has been before" - isn't that the definition of "research"?? It's a creative endeavour - you are done when you're done, not before. Applying pressure, especially in the form of deadlines, is counter productive. Only the bravest of staff will 'push-back' and refuse to release incomplete, incorrect and buggy code or services. And once a 'milestone', like any Victory in war, is declared complete, there is no going back. The body of work, no matter how poor, can only be tinkered with, never fixed.

Or restated: By applying arbitrary deadlines, you guarantee faulty code. Then you are stuck with the mess as the foundation for the rest of your work. You will slowly drown in the morass of your own mistakes - and all easily preventable.

The real Myth, is that there is an average person who's output can be projected.

The best person to forecast completion of a unit of work is: the one who's going to do it!
And the 'horizon' is only clear for about 2 weeks ahead. So detailed work plans can only be constructed for the immediate future. Yes, there will be a design, but it will evolve and it will be a set of road signs.

The old rubric applies: Double your best estimate, then double it again.
And don't take on tasks you don't know how to do... You'll only create a huge mess, and everyone, including you, will be very sorry.

More...

Summary...

Teams - the Killer App?

Jerry Weinberg, the doyen of the Quality Software movement, in his landmark four-volume set "Quality Software Management" comments on models of Software Metrics - that five factors, each of which could vary productivity two-fold, were left out because they were too hard to measure.

Team Performance is one of these five [from memory].
In "Pop Management" books, there's a lot of "soft" assertions and anecdotal evidence on the value and creation of "High Performing Teams" - but where's the solid research that shows Teams Really Do Work? And what their multiplier factor is. [Jerry is much better than that. He's been an academic and reports some very nicely constructed experiments.]

"Dynamics of Software Development" by Jim McCarthy, the real-life lessons from Microsoft's C++ team, focused on team factors. McCarthys mantra is: "The Software is the Team, the Team is the Software". If there is a problem in the team, it will show up in the software. Faults in the Software are caused by failures in the team process. Team dynamics impact performance immediately and greatly.

Bob Lewis succinctly defines the difference between 'teams' and 'groups' - a 'team' needs everyone to complete the task. There are no "Sales Teams" - individuals make sales.

What I'd like to say is: "Here's the definitive reference that shows Teams improve software & I.T. productivity by a factor of 2-5".
When I find the research, you'll be the first to know.

More...

Summary...

Getting More for Less out of your staff

What to make more profit, lower costs and increase financial viability of your company?

David H Maister in "Practice What You Preach: What Managers Must Do to Create a High Achievement Culture" gives you the formula. And his model really is a mathematical formula!

And it's simple - staff will perform better, stay longer and even work for less if they like working for you.
Message: Treat your people well, and you will benefit many-fold.


For some people, this will be a "non sequitur" - a "yeah, so what?". For those who believe all workers have to be whipped to produce anything, and will skive off given half a chance, nothing will convince them - not even going broke.

For those wanting to be convinced, Maister does a lovely bit a research using a multi-national marketing company with a full range of business types - small to large, premium products to low-cost mass-market.

Masiter comes up with a good mathematical model relating a number of factors to a composite figure he calls "Profitability" - not just one years' profit, but longer term financial viability. He also has a nice way of presenting the factors in hierarchical form.

Think this through with "brain-workers" (or Knowledge workers) - you can't order them to "produce some brilliance". You can't see inside their heads nor really know just how good the 'stuff' they produce is. Software is an intangible - it's hard to measure and harder to quality assess. You can't beat good ideas out of them...

This is the definitive environment for "passive aggressive" behaviour and for subtle undermining and sabotage.
Screw your staff over, and you'll reap the consequences for a very long time. And you won't even know who did what, when.
At the very least, the more "meek and mild" staff will just withdraw and do an absolute minimum.

You did remember to hire bright people, didn't you? They will expend their creative efforts in looking very busy and producing as little as they can get away with.

Unless you are an expert in the field, and current at that, you won't be able to pick it.

Message to managers:
Treat your I.T. staff well. Get them off-side and you will suffer.

The great thing is that they are very easy to keep happy - give them the stuff they ask for or be clear about what you can afford and why, listen to their requests for changes and especially for reasonable deadlines and occasionally show them you appreciate their efforts. In return you will get high output and when you need it, they will go to extraordinary lengths for you. You have to earn their trust and loyalty - not bully, intimidate or demand it.

There's a secret here you've heard a thousand times from the mouths of Pop Stars: "I'd do this for free".
Yep - good and great programmers/I.T. practitioners love what they do.

And there is a proof the best will do great work for free: Open Source Software.

Technically there is a reason - there are very high intrinsic rewards in programming/I.T.
We can guess it comes from the feelings of "Flow" [Mihaly Csikszentmihalyi ], the human reaction to 'achievement' (a nice high) and the rewards of Problem Solving.

[There has to be some good psychology names for these processes]

More...

Summary...

The 100-fold advantage

There is a Silver Bullet for programming productivity - pick good people. It probably works for all other areas of I.T. as well.

There is a definitive study, rigorous research, by a Professor of Behavioural Psychology.
It's published in the book "How to be a Star Performer" by Robert E. Kelley.


BTW, Kelley talks of only 10- and 20-fold performance differences amongst "brain-powered" staff at the Bell Labs unit that programmed the "5ESS" telephone switch.

"In the wild", this varies more. Best I've heard of is Rob Kolstad who in a half-day rewrote a key program that had taken a year or more effort - and it ran 100,000 times faster. I've sped-up a critical process by 2- 5,000 times, again with a small tool, quickly implemented. Sadly, no studies, no references.

"100-fold" is a guesstimate, an accumulation of experience and observation, not solid, proven research.
But then again, how do you measure 'productivity' for programmers and other I.T. professionals. Lines-of-code a day doesn't work - you just drown in bugs...

Kelleys' take-away is that 'Stars' are made, not born.
"Initiative" is the central key element - everything else builds on that.

With more highly productive staff, you should be able to get more done - if your organisational processes allow and encourage it, and your managers can get out of the way of the technical staff.

From the on-line page, the Contents of the book:

I. The Productivity Secrets of the Star Performers
 1. What Leads to Star Performers
 2. Stars are Born, Not Made
 3. Creating the Star Performer Model

II. The Nine Work Strategies of the Star Performers
 1. Initiative: Blazing Trails in the Organization's White Spaces
 2. Knowing Who Knows: Plugging In to the Knowledge Network
 3. Managing Your Whole Life at Work: Self-Management
 4. Getting the Big Picture: Learning How to Build Perspective
 5. Followership: Checking Your Ego at the Door to Lead in Assists
 6. Small-L Leadership in a Big-L World
 7. Teamwork: Getting Real About Teams
 8. Organizational Savvy: Street Smarts in the Corporate Power Zone
 9. Show-and-Tell: Persuading the Right Audience with the Right Message
10. Become a Star Performer: Making the Program Work for You

More...

Summary...

I.T. Management 101

What do they teach in "Management 101" in the School of Hard Knocks?

"Tech-heads are knuckle-heads, Management Knows Best".

Simple, stupid and ineffective... What are the things you want in your technicians? Breadth and Depth.

You want Bright, Capable people that are motivated, knowledgeable and productive. You also want innovative, cost-efficient solutions...

Adopting a "Command and Control" mentality in a cognitive based task is not just wrong, but pessimal - you can't do it worse. Managers who've been 'Technical' are often worst - they believe they know all the answers and are still technically relevant.

In 2007 I know of a major Australian outsourcer using 300Kb Word documents as its Change Control mechanism.
That could've been a cute idea in 1995. It's a mornings' job for a young gun to produce a PHP-MySQL solution running on the Intranet. The 'Managers' can't see a problem...

No references - don't know where to start or how to define the question...

More...

Summary...

The secret that shall never speak it's name

Nett Negative Producers

That's it. This is one of the biggest dirty secrets of the I.T. business.

It's not just that there is a huge difference in both the capability and productivity of I.T. practitioners, there are a large number that are negative producers - if they were gone, more would get done.

I'd give you references and real research - but it doesn't exist, because this is problem that doesn't exist...

Want to give your I.T. productivity a big boost - discover and remove these people.
They could be managers, could be technicians - but they will always seem and look busy.

More...

Summary...

I.T. Failure =/= Practitioner failure

In this piece I make a case that problems with I.T. are not failures on the part of the Practitioners.

Computing/I.T. failures have been noticed as serious since the mid-1960's. Within the first 15 years of commercial computing.
There has been ample time to fix the problems we see around us today.
There are very good models of what can be done, and how to do it: e.g. Aviation

So what's going on??
The people that can force change and improvement are not.

Some other game is afoot. What it is we can only guess at...

More...

Summary...

To change the Culture

From Bob Lewis in his Keep the Joint Running column, Office biogeography of 26-Feb-2007.

"Culture is the learned behavior people exhibit in response to their environment. Want to change the culture? Change their environment. It's the only choice you have, unless you think preaching will have an impact."



ITIL is a Culture Change. I.e. People will have to do their tasks differently, will have to perform to different expectations and may even suffer consequences for doing just what they do now. i.e for not changing.

I've heard it asserted that "(the people in) IT are the most change resistant group" in any company. Purports to come from a mid-1980's survey .

And elsewhere, it's part of the received wisdom that "IT is about Change" (i.e. not us, but 'them' the users), so one of the most important aspects of successful I.T. implementations is "Managing Change" - the (other) people dimension. (And you have the internal people (us) dimension of the team as well.)

The synthesis:
Adopting ITIL is probably the single biggest change, hence most disruptive event, in the life of most I.T. Operations/Services groups. Modulo all getting fired and the work 'Outsourced'.

Bob Lewis's observation is that you have to change the environment to change what people do, the culture.
And a really big part of that would be: The Management.

So I'm wondering:
Do all successful ITIL implementations 'start from the top'??

More...

Summary...

Notes from the Field. Post "Service Level Management"

A 'post course' report - some of thoughts from reflecting on the "Service Level Management" Practitioners course.

First:
"Hats off" to course presenter, Ed Broome, and "Itilics" in Melbourne.
Their home site.
Give them 5 out of 5 for content, delivery and physical stuff.

Follows a bunch of observations on ITIL in general.




"There are 40,000 orgs around the world doing ITIL".
Hmmm, there's 3.3M ABN's in Oz & 2,000 companies on the ASX. It's commonly accepted there are around 1M 'companies' in Oz. And Oz is ~2% of Global Market.
You might expect 50M companies globally - and around 100,000 in the medium-to-large area. And at least the same again in 'Government'.

Use: Market is small. Selling through existing 'franchises' might work.
Question: If it's so *good*, why hasn't it taken off after 20 years??


'Practitioner' courses are not for novices or 'juniors' or maybe even
'intermediate' skill level people.
Needs more than 5 years of experience in I.T. to understand & pass...
Use: There's a niche teaching long, practice/tool based courses for the
those below this level.


"ITIL is Public Domain" - the same way that buying a Lexus is PD! Anyone can do it, if they can pay!
Use: Cheap resources & courses are a niche market that's crying out!


ITIL seems to have missed, "The Internet Changed *everything*"
Use: Super-model. 100% Availability paradigm, ...


The ITIL model is quite thin... Think 80's, think 'mainframe', think 'large government', think 'common sense' (choke, splutter, gag) vs.
SysAdms are not mentioned. Technicians just do magic.
Apparently 'technicians' are all of equal talent. Falls into the "Mythical Man Month" Fallacy. Not all 'Subject Matter Experts' are of equal talent - by at least 100:1 (that's one HUNDRED to one). Professional vs. Professional. (And I wish I had more to back this up than 30+ years observation!)

Performance Analysis, Tuning & related stuff only come up in context of 'Capacity Management'.
Use: Hmmmmm

Corollary: PRINCE2 is assumed for Project Management - but the interfaces don't seem well drawn.
There must be a technical/SysAdm taxonomy possible as well.
Are there other parallel disciplines that should be noted (Data Analysis, Security, ??) - and their interfaces defined.


ITIL explicitly doesn't support making use of quantum leaps forward in computing - applying the latest technology to achieve big gains is specifically 'out of scope'.

The "Service Improvement Program is about *gradual* (steady?) improvements in Quality" and "Improving the TCO of IT Services" (an implied 'Slowly').

We've just come through a number of really significant 'event horizons' that need to be digested/put into practice.

• Everything runs on Intel (most everything)
  
• Q1 2003, Moore's Law constant for Processor Clock speed went from 52%pa. to ~20%pa. [Can't find original source]
  
• Win XP was the last 'Big Breakthrough' in O/S
  
• Virtualisation is now free & ubiquitous - XEN and Vmware.
  
• Disk is really cheap... $1,500/Tb.
  

Next two/three:

• Multi-core that does real SMP! And Apps that use it.
  
• DataCentre in a Box. Do the Google & treat CPU & Disk like fluorescent tubes. Let them break & replace in one big sweep
  
• Snapshots. When the NetApp patents run out or they license. Reliable Data, Everywhere...
  

Use: Add to the super-model. Show people how to do it. 100% Availability
models.


There are 3 kinds of datastreams gathered by instrumentation, used for:

• service level metrics (Nr Tx/hour, end-end user response, ...)
  
• system metrics used for internal systems processes - tuning, forecasting, analysis, ...
  
• diagnosis/troubleshooting [catch *every* packet on an interface for
  later analysis]
  

Use: Performance Analysis, Service Level Reporting, Tool development.


IT Services are about SERVICE (the capital-S kind that great restaurants do).
Not defined in ITIL. Not even noticed from what I've read...
And this is about selecting people for there (psych) attributes, attitudes and aptitude - of course ignored.
Use: Add to the super-model


ITIL is probably too disciplined & formal for most *management* groups (they're not teams).
They like to not be held accountable, play 'games' etc etc.
And they have staff who are used to these games and will
Use: Will never sell to these people... Flee immediately! when spotted.


The *whole* group (6 ppl) were all high achievers. Was this the norm for "Service Level" Management? (Apparently not - Ed confirmed we were an unusual group.)

We were given a 'sample test' - and everyone was disappointed they didn't score 100%!
Lowest score was 17/25 (12 or 13 needed).
Use: For some of the ITIL disciplines, those who've done the training are pre-qualified as 'movers & shakers'.


The ISEB is very British... Good thing they are having ITIL testing taken away - hopefully the next group is a little more amenable to small changes & giving feedback.
Use: none. Barrier to entry & successful learning.


There was also some mention of "Organisation IT Services Maturity Level" assessments similar to CMM.
[AXA Insurance is at the top level, '5']. Don't know where that standard comes from...
Use: A great service to sell :-)


Now the BIG NEWS:
ITIL is undergoing a 'refresh' - version 3 is being released this year.
BUT - there is an ISO standard for "Quality Management of IT Services" - that can be audited... ISO/IEC 20,000.
It is ITIL *based*, but has a slightly different taxonomy (14 disciplines vs. 11).

This would be a perfect standard for the Sarbanes-Oxley people to pick up on & adopt for big public corporations in the USA... And what the US does, we tend to follow... Or not :-)

This apparently is not a new idea. Awww Shucks.


And I passed the previous 'Problem Management' exam :-) Received the paperwork from the UK today. Now I have to write to the "iTSMF" in the UK for a badge :-(

Does this mean I now have a post-nominal?? (But what? 'IPPM' - ITIL Practitioner - Problem Management)


One of the really interesting questions: Just *what* in ITIL is copyright and what can be freely reused (like the names of the 11 'disciplines' or 'processes' - are they copyright?)

To write a commentary on ITIL, big chunks of it have to be quoted to be critiqued...
To write practice-based courses - ditto...

Now a pie-in-the-sky question:
What are the *limits* to "IT (Ops) inventing themselves out of work?

Where does all the automation & embedded appliance/service stuff end?
How do we make a buck in that world - or is it after I retire :-)


A precise of the actual content will follow in a later post.

More...

Summary...

What's this thing called ITILOPIA?

Is this "ITIL Utopia" or "ITIL Myopia"??


I've started on my "ITIL Journey". I first worked in "Information Technology" (then called ADP - Automatic Data Processing, 'Computing' or 'Programming') in 1972 and have specialised in it since end of 1974. I was in the first generation to get a major in 'Computer Science' (The term 'Software Engineering' was only invented 3 years prior to me starting Uni) and was in the fist generation to use Unix (in fact, the second ever course in the Unix kernel, by John Lions).


I've worked in most areas of I.T., most organisation types (Public, Private, Not-for-Profit), as contractor, consultant and permanent, on Projects, Operations and Maintenance, in almost all sizes of companies (1 to 25,000), Design, Architecture, Analysis, Ops/Admin and programming/implmentation, Real-Time, Batch, GUI, High-performance computing, Database, Data Analysis and even Accounting/Billing/Book-keeping.

And I'm usually the local expert in Troubleshooting and Fault Diagnosis.


I'm yet to be a CIO, Project Manager of >$1M, or Team Lead more than 6 people.

But my projects always deliver. Even in the face of extreme obstacles and people running 'interference' (You know who you are!).


This not only gives me a bunch of experience, but a unique perspective.


'ITIL' may be a good start for moving into the world of "Mature Computing" (the first commercial computer, the LEO (Lyons Electronic Office) was produced circa 1951).

Or it may have some gaps and oversights...


Which is it? Join me on the Journey :-)

More...

Summary...