March 2, 2010

Why Do Public Sector Software Development Projects Fail – Part 3?

Part 3 – Real Causes

In this third part of our series on why public sector IT projects fail we’re going to discuss the real reasons we believe public sector IT projects fail.

Information technology projects encounter serious problems not as a result of deficiencies in technology, support, methodology or technology staff (note: these problems do occur but they are typically quickly identified and remedied by competent project teams). Looking at IT projects that ran into serious problems (failure, long delays, significant cost overruns), they almost always include most or all of the following:

Users drive development

Ask a user what he or she wants in a system and the response will invariably be some version of: “Everything.”  In public sector IT project, users don’t pay for the system.  They don’t see that every feature and function they request adds cost, time (reflected in the schedule), and complexity to the system. Because they have no way to associate scope (features and functionality) with cost, they ask for everything.  Users when asked will tend to include everything they’ve ever wanted.

Initial and continuing estimates of cost and size are inaccurate

Systems often start with inaccurate estimates of size, which translate into inaccurate budget, return on investment (ROI), and schedule estimates. Once reality strikes, in the form of increased estimates of size and work, budgets and schedules have to be revised, and the project is “late and over budget, again!”  Return on investment goes out the window (“we’re too far along to stop now!”).  However, even revisions to initial estimates  are often skewed by expediency (“senior management won’t accept a 6 month delay, we’ll do it in 3!”) and place the project right back in the same boat it started in: with no real idea of cost, ROI, or timelines.

Measurements used provide the illusion of control

An IT project generates an enormous amounts of data from staff time cards, hours per task, hours of work completed, percent of work completed and so on.  This data is transformed to project metrics that provide a tremendous “feel good” factor (“we are 80% complete on the task”).  However it is of little value if you can’t use it to predict what’s coming for the project (“the remaining 20% will take as much time to complete as the first 80% did”).  Most measurements are worse than no measurement because they tend to placate individuals who don’t know better (“Wow – we’re 80% done – that’s great!”).

Reporting focuses on completed work, not remaining work

Related to the above, projects often focus on what’s been done, and find amazingly accurate ways to measure completed work. What is done is in the past doesn’t give you a very good idea of what the future looks like. Data about how far you’ve come is a bit like driving a car by looking in the rear-view mirror; “We haven’t hit anything so far, so we’re okay.”

Scope is not really managed

The scope of an IT project will change. The business of the organization can change, parameters of the project might change (e.g. funding), and on nearly every systems project the further we get into it the more we learn about the problem we are trying to solve. Early estimates may have over or under-stated (typically under-stated) the real effort required to design, develop, test and implement the system.  However, typically only the largest scope changes are acknowledged and accounted for on the schedule.  The little things add up.

Historic information is used to place blame and avoid responsibility

Historic information unfortunately serves one predominant purpose on too many projects: providing the facts needed to place blame when things go wrong. Failing projects abound with sponsors, staff, vendors, managers and users pointing the finger at each other and producing reams of reports showing that it’s not their fault. The project took twice as long and cost twice as much as planned, doesn’t fully do what we need, and the users hate it. But it wasn’t our fault!

The decision making culture is inappropriate

In anything but the smallest project (one or two staff members) decision-making by micro-management or consensus won’t work. Project managers can’t make all the decisions for all the staff; he or she hasn’t the time, skills, or the needed information. At the same time, consensus management won’t work, either; soon everyone spends all of his or her time in meetings trying to reach consensus. Decisions have to be made by the people with the necessary information, and the responsibility to make them.

Staff work hard, but not necessarily on the right things

IT projects don’t have staff members sitting around with their feet on their desks, working crossword puzzles. Generally, staff members work hard on their projects. However, we regularly find staff are working hard on things that don’t matter much.  Staff have been focused on things that don’t keep the project moving forward.  This is not a problem with the staff but rather with project management.

Next we’ll examine some of the “symptoms” that show up on projects that have these problems.

This is the third in a multi-part series of posts analyzing the causes of failures in public sector IT projects and proposing some pragmatic solutions

Part 1 – Overview

Part 2 – Apparent Causes

No Comments

February 22, 2010

New State Health Information Exchange (HIE) Toolkit Released

The State Health Information Exchange Leadership Forum, lead by the AHIMA Foundation with sponsorship from the Office of the National Coordinator for Health IT, has developed a comprehensive State Health Information Exchange (SHIE) Toolkit to support State HIE grantees during the planning and implementation of their statewide interoperability projects.  The Toolkit is regularly updated to include guidance, state examples, and lessons learned in the areas of Planning, Governance, Technical Infrastructure, Finance, Nationwide Health Information Network, and Grants Management.  The Toolkit will also be updated to provide sample strategic and operational plans as plans become available.

To access the SHIE Toolkit, visit: http://statehieresources.org/

If you would like some help learning how to use the toolkit please contact us at info@pubknow.com.

No Comments

February 21, 2010

Can Independent Verification and Validation (IV&V) be too rigorous?

Each proposal you receive for Independent Verification and Validation (IV&V) of a software project will state the vendor’s approach is “rigorous”.  Vendors fight to explain how their methodology is more “rigorous” than a competitor.  This is what you want right?  Isn’t rigor a good thing?  Maybe, but not when rigor:

Takes over for experience. Vendors will try and bury you in details about how rigorous their process is so you’ll hopefully overlook the fact they are staffing your project with individuals that have barely used software let alone written it, performed quality assurance on it, or managed it’s development.   The truth is effective and efficient IV&V is art as much as science.  You need qualified consultants as much as rigorous process.  Make sure you examine who will staff your project very closely, asked to see the results of each individual’s previous IV&V efforts.

Is used as marketing hype.  Vendors will try and convince you they are “scientists” or “engineers” performing science or engineering on your projects/software. After all, V&V was pioneered by NASA and, for software IV&V, the base standard is defined by the IEEE (Rocket Scientists and Engineers) so who better to perform it?  IV&V is, conceptually, pretty simple:

  • Identify the products and processes to undergo verification and validation (preferably before or in the early stages of development),
  • Determine the criteria with which each of those products and processes can be evaluated (starting with standards where available e.g. the IEEE-1012-2004 standard for software verification and validation),
  • Assess the products/processes while in production and upon completion to verify they meet the predefined criteria and note where they don’t,
  • Re-assess products/processes when deficiencies have been addressed.

IV&V is hard work that takes expertise (there are many nuances in developing criteria and evaluating products etc), independence, and yes a degree of rigor.  But it’s not “Rocket Science”.  We’ve seen a significant amount of rigor added to an IV&V process that adds little value to the client and seems to be there to convince the client that IV&V is engineering and they are hiring the best “engineer”.  There is no professional certification of IV&V “engineers” like there is for electrical or civil engineers.  Don’t buy into the hype that this line of reasoning purports.  Understand you need competent, experienced professionals but this is not a mystical art.

Is used as an excuse for lack of social and political savvy of vendor staff. Beyond competence in performing IV&V your consultants need communications skills and an understanding of the political landscape public sector projects face.  We’ve actually witnessed an IV&V vendor reporting to a state legislature stating strictly facts (“the project is 2.3 months behind schedule” and “523 out of 1232 test cases failed”) without explaining context (“critical features were added” and “we’re only a quarter of the way through testing”).  The poor agency receiving the “benefit” of IV&V spent the next month trying to keep funding for what was a thoroughly successful project.  When called on the carpet about this the IV&V vendor explained that their rigorous methodology required they describe only the facts not the context of those facts.  Just what you need, Joe Friday (“Just the facts ma’am”).

Creates unnecessary bureaucracy that just wastes your money. Sign-offs of intermediate IV&V products, large review meetings, and repeated reviews of materials provide you with little value.  Often this “rigor” is applied not for your benefit but for the consultants, to ensure their rear is covered (“but you approved the interim deliverable”) or to enhance consultant cash flow (“you approved the first interim IV&V checklist so we can bill you”).  This unnecessary bureaucracy simply costs you time and money.

IV&V helps your software projects succeed and should be rigorous – meaning thorough, focused, and disciplined.  However, be aware “rigor” can be used for the wrong reasons.  When working with your IV&V consultant or the next time you’re reviewing an IV&V proposal keep in mind excess “rigor” provides little if any value; is used to cover a vendor’s weaknesses or benefits the vendor not you; and unnecessarily raises the costs of IV&V services.

No Comments

February 12, 2010

Why Do Public Sector Software Development Projects Fail – Part 2?

Part 2 – Apparent Causes

In this second part of our series on why public sector IT projects fail we’re going to discuss who or what commonly gets blamed for failure.  In our unfortunately all too many postmortems of failed IT projects here’s what we typically find people saying:

Technology

Technology failed us. ”We didn’t have the right tools, We needed different software/hardware/network/programming language and we would have been ok.”  The assumption here is there is some magic technology bullet that would have pushed us over the threshold of success.  It’s possible but highly unlikely in our experience.

Methodology

Our methodology failed us ”We used ABC (insert CASE, SCRUM, Agile, Waterfall, Information Engineering – we’re old enough to have seen them all) system development and XYZ method of project management. If only we had used 123 project management and 456 system development methodology we would have been okay.”  Similar to technology the idea is that there was some magic technique that if followed would have guaranteed success.  This is usually a specious claim as most projects follow little or no real methodological discipline.  Even if followed to the letter a methodology won’t save a project that’s destined to fail.

Staff

The staff failed us. “ We used contract staff instead of state staff. If only we had used our own staff we would have been okay” or “our staff just don’t have what it takes to do a big project like this.”  Most developers are, in our experience and with good hiring practices, very competent.  Same holds for program staff.  Though there are occasions when a slick salesman provides bad resources these are usually weeded out quickly by competent management.

Vendor/Consultant

The vendor/consultant failed us.  “They said they could build a rocket ship but it turns out they couldn’t”.  OK Vendor’s do over commit.  They are in business to make money and will often promise more than they can deliver.  Was this the cause of a projects failure?  Possible but unlikely, the root of the problem was in place well before the consultant was hired.

Senior Leadership/Politics

Our own leadership sabotaged us.  “They never supported this project from the beginning!” or “The political winds were blowing against the project so they case us adrift.”  In our post mortems we’ve simply never found this the case.

The Project Manager

The project manager didn’t understand technology.  ”The manager really didn’t get the consequence of the decisions s/he was making, if only someone technical was in charge!”  or “S/he micro-managed us the whole time!”  Our reviews of failed projects have revealed many a PM who wasn’t a good project manager – but we’ve yet to see a project fail because the manager wasn’t technical enough or spent too much time with the project.

Oddly enough we rarely hear “our Quality Assurance vendor (QA) failed us.”  QA has failed on these projects.  However, their contribution to failure is rarely recognized, after all it was QA that pointed out the failure in the first place.  However, as a buyer of Quality Assurance services you need to ask where was the QA vendor BEFORE failure began, why didn’t they recognize the seed of the problem before it sprouted?…but that’s a topic for a different time.

You’ve probably guessed by now we don’t believe any of these are the root cause of project failure.  At best they might be contributory.  No matter what they are certainly a diversion from finding the real cause and instituting practices that will save the next project.

Next we’ll give you a few ideas about what the real cause of failure might be.

This is the second in a multi-part series of posts analyzing the causes of failures in public sector IT projects and proposing some pragmatic solutions.

Part 1 – Overview

No Comments

February 3, 2010

Why Do Public Sector Software Development Projects Fail?

Part 1 – Overview

All major public sector enterprises have an increasing investment in software (and IT in general) and a growing need to earn a better return on their investment.  Recent studies have shown that software projects are often over budget and behind schedule.  Although studies come up with differing numbers, generally statistics show that:

  • Over half of all medium and large software projects do not deliver their expected benefit, and exceed their schedule and budget.
  • Over half of the medium and large software projects either fail completely (management pulls the plug) or require big recovery efforts to get them to completion.

Software projects, due to their scale and scope present special problems.  Management often over estimates the business improvements with optimistic delivery dates before there is a good understanding of the cost and time it will take to complete the project. User needs change, staff members move on, and the scope, schedule and budget begin to grow. The software development project soon becomes a black hole, into which the organization pours dollars and people. Agency management hears nothing but good news from project management and contractors until it is too late.  The problems start to surface and “blame game” begins; management blames the project staff, the project staff blames the contractor and the contractor blames the project staff.  The end result is a software project that is over budget, delivered late, and the business needs and quality expectations are not met.

So what to do?

This is the first in a multi-part series of posts analyzing the causes of  failures in public sector software projects and proposing some pragmatic solutions.

No Comments

Read More →