In a previous post we asked the question “Does being late on a project matter?”. In that post we note a trend toward accepting that projects will be late as the norm. Recently we’ve run into several projects that were going to be late and the consequences were grave. Some of those consequences (aside from the obvious increse in costs we mentioned in the previous post) include:
- You (as project manager, sponsor, project team member) lose credibility with stakeholders not just on matters of schedule but in general. Simply put people either believe you don’t know what you are doing or don’t believe you tell the truth about project issues. This effects not only the project but your career;
- Externally imposed deadlines (for example, Federal deadlines for Health Insurance Exchanges) exist in which penalties will accrue if you are late or payments from external sources (e.g. Federal matching funds for MMIS systems) are delayed causing financial hardship; and
- Loss of funding as projects spread across budget cycles. We’ve recently seen several cash strapped jurisdictions defund existing projects simply because they don’t have the financial resources to continue them.
So, on many projects, being late matters. It could have serious consequences. The solution to lateness is not to accept it but to:
- Accurately estimate project schedules in the first place;
- Keep those estimates up to date and keep stakeholders informed of any changes; and
- Identify and develop mitigation strategies for the issues causing schedule slip.
Public Knowledge, LLC’s Predictive Management Framework® provides specific techniques, based on 25 years of hands on project management experience, to accomplish all of these. It is designed to supplement project management methodologies with some practical techniques often overlooked by those methodologies. While the techniques don’t eliminate late projects they do, if followed, provide better initial estimates of effort and duration, keep those estimates up to date, keep those interested in the project aware of schedule issues, and provide a framework for mitigating the risk of schedule slip.
According to research conducted by Capers Jones the average amount that a software project’s scope expands from the time when the specifications are approved to the time when the system is delivered is:
- 10% to 15% for small systems,
- 30% to 35 % for large systems.
This “scope creep” or expansion of the functionality usually occurs because either:
- The systems users are able to persuade the systems developers to add various “bells and whistles” enhancements, or
- Parts of the specifications were vague in the first place, and full exploration of their implications causes significant additional work to be added project.
The solution to scope creep has three parts:
- Develop sufficiently complete and detailed specifications in the first place, to remove vagueness, ambiguities and room for misinterpretation,
- Ensure that the customers (users of the future system) participate meaningfully in development of the specifications, and fully understand them and their implications, and
- Carefully manage the change control process.
We will discuss what constitutes “sufficiently complete” specifications, meaningful participation, and management of the change control process in future posts.
All of us at Public Knowledge, LLC want to wish you a happy and safe holiday season. We’ve had an exciting year and thank all of you that participated in that year. We’re looking forward to another great year (things look busy!) and we hope you will be part of our new year.
Earlier we discussed the difference between risks and issues. In this post we’re going to talk about four aspects of risk you want to make sure and consider when dealing with risks. We find these often missed in discussions about risk. All of these help you determine the materiality of the risk and will help you prioritize the effort you put into responding to and managing the risk (more about managing risks in a later post). The aspects to think about and capture are:
- Impact: How big is the potential impact on the project? We’ve seen this measured in a vague terms as “High, Medium, or Low” or as precise as a dollar amount or specific time the project will be delayed. In our minds the more precise the estimate the better. This is not necessarily because the more precise measure is more accurate but because the thought process required to come up with a precise estimate will help better define the risk.
- Likelihood: What chance is there that this risk will turn into an issue for the project? Are the “odds” high? As with Impact, the more precise measurement the better, to a degree. While high, medium, or low may be a bit light, 27.56% is an estimate that gives an illusion of precision. Likelihood is predicting the future and false precision will give the impression you have a better handle on risk than you ever could. In general we find risk expressed as a fraction with 4 or 5 in the denominator to be sufficient. So for example theres a 1 in 4 (1/4) chance this risk will occur or a 4 in 5 chance. This also provides sufficient detail should you choose to build a mathematical model for risk prioritization (more on mathematical models of risk in another post).
- Immediacy: How soon will the risk occur? Projects are often about prioritizing what’s important today. Having a handle on when you think a risk will occur will help you prioritize when to start worrying about them. This is the most difficult of the aspects to estimate and is the least precise. Generally we think in terms of immediate (defined as within the next 3 months), mid-term (defined as between 3 and 6 months), and long term (greater than 6 months).
- Triggering/Preceding Events: What visible events will precede the risk becoming an issue or indicate the risk is becoming an issue? Be concrete, for example, if we see Milestone X is missed or have an indication it will slip it will indicate that the risk of late product delivery has turned into an issue. Often risks will have several triggering or preceding events. The biggest issue our clients sometimes have here is being too vague about defining the events.
There are, of course, other aspects of risk you want to capture when performing risk analysis and management on a project (for example: risk management strategy and tactics). We’ll be more specific in a later post, but the aspects above are the most often missed on projects and some of the most beneficial in dealing with risks. Remember overall it’s less the list of risks you come up with in the end but the process of periodically identifying and analyzing risks that is of most value to the project.
We were recently brought in to help a public sector client get out of a jam with a major systems procurement effort (many millions of dollars). The problem was it was too late. The approach to procurement they had used resulted in the selection of a vendor and solution that did not best meet their needs.
They had constructed their procurement around a typical strategy: Identify the 3 or 4 categories of things that mattered to them (for example: solution provider experience and viability, solution fit with requirements, and cost), weight those things, and score the vendors on them. In this case here was the problem: in these tight economic times cost was viewed as critically important and given a high weighting. Vendor’s know this and game the system by offering a minimally acceptable technical solution and low-ball the costs. They do this fully expecting to get further revenue through the change orders they know a customer will request (because they understand their proposed solution doesn’t meet the clients real needs). In the case of our client so much of the score was based on cost that the process resulted in a clearly inferior product “winning” the procurement. So what are some strategies for avoiding having an inferior product come out on top?
- Having well defined requirements will help. This will allow you to better compare the solutions vendors propose. It is however, difficult and time-consuming to develop a detailed set of requirements. Because you don’t typically do requirements you will probably not have the skills in-house to develop these well. A third party specializing in developing requirements can pay dividends in the end.
- Unless you are purchasing a commodity item, like pencils, consider a “value” based procurement instead of strictly a “cost” based procurement. There are multiple ways of evaluating value, for example, score the “value” of a proposed solution by considering the cost per technical point. This will mediate “cost” based gaming of the procurement.
- We’re not suggesting total cost doesn’t matter. In your procurement strategy leave yourself room to negotiate total cost. Pick the vendor with the highest “value” but leave yourself room to cut the total cost of the proposal by cutting scope. Have a pre-contract negotiation phase where you can negotiate scope (and therefore total price) with the vendor.
- If your local procurement regulations prohibit pre-contract negotiation consider a best and final offer (BAFO) solution. In a BAFO, you give the vendors a chance to sharpen their pencil’s (and scope) and come back with their best offer. This doesn’t always steer procurement to best value but it can help lower total costs.
- Lastly, test your procurement strategy. Consider all possible scenarios (e.g. A vendor has a low total cost and high technical scores), document them, and run an evaluation of the results. Are the results what you thought? Perhaps you should revise your scoring strategy.
The most import lesson here is HAVE a strategy before you release a systems procurement. If you don’t procure major systems regularly (and who procures multi-million dollar systems regularly?) get some assistance with your procurement.
What’s the difference between an issue and a risk? A lot of times people use these terms synonymously when, from a quality assurance perspective, they are quite different:
- A risk is an event that may occur on a project that will have a negative impact (for example increase the time or cost to complete a project). Note that a risk may occur. It is not a certainty (it has a less than 100% probability of occurring). From a theoretical standpoint it is possible to have a risk that will have a positive impact on the project (shorten the time or cost of a project) but in quality assurance we typically don’t assess risks that could have a positive impact. Optimizing project outcomes are left to project management.
- An issue is an event that the project will or currently faces that will/does have a negative impact. That is, an issue has a 100% probability of occurring.
The key differentiator here is risks are something that have some likelihood (probability) of occurring in the future. An issue is something that will occur or is occurring. Issues “will happen or are happening” and risks “may happen”.
A concrete example:
- There is a risk a project may lose staff. We may even be able to assign a probability this risk will occur based on expert opinion or past experience. We can build a plan to address this risk should it occur. The plan might dictate by staff position what we will do if a staff member leaves the project. For example if we lose the project manager we will temporarily promote the deputy project manager to manager and begin the search for a replacement. We can also do some things to lessen the likelihood a risk will occur. In the “loss of staff” risk example above we might offer project completion bonuses to increase staff motivation to stay with a project through completion.
- There is an issue if a staff member leaves the project and we have to act now to replace her.
Obviously identifying risks before they become issues and having plans in place to address them is preferable to having an issue arise for which we are not prepared. In some cases it is even possible to lower the probability a risk will occur or even completely avoid the risk.
We are often asked “what are the benefits of performing Independent Validation and Verification (IV&V) on a software project”. Though there are many, we routinely site four to our clients:
- Early detection. IV&V identifies both problems and opportunities for improvement to processes and work products early on, so that they corrections and improvements can be made before the consequences become major. This has the result of reducing the total amount of work—and rework, which is especially expensive—that must be performed for the project to achieve its objectives. Note that IV&V does not just focus on problems and risks, but on solutions to problems and mitigation strategies and actions for risks. Every problem, risk or opportunity finding includes practical recommendations for solving the problem, mitigating the risk, or exploiting the opportunity.
- Improved quality. IV&V improves the quality of project work products and processes. These improvements come not only through early detection of errors, but through pro-active prevention of errors: IV&V provides checklists, templates, and other instruments to improve the quality of project work form its inception.
- Lower cost. By detecting errors, problems, and opportunities for improvement at the earliest possible points, IV&V can significantly reduce the over-all cost of an IT project. In an often sited study by NASA (A Case Study of IV&V Return on Investment -ROI; R.A. Rogers, D. McCaugherty, F. Martin; NASA; October 2000) estimates the return on investment for IV&V services is between 1.25-1.82. Despite the age of the study the figures appear valid (or even somewhat conservative) when applied to todays IV&V results.
- Reduced management burden. By providing proven, professional techniques to evaluate project products and processes, IV&V can reveal risks, mitigation strategies, and opportunities for improving the efficiency and effectiveness of processes, greatly reducing the stress and “fire-fighting” typically associated with project management and freeing up managers’ time to focus on more important strategic issues.
Though there are other benefits these four alone typically justify the expense of performing IV&V on a software project.
