More States are recognizing the value of having an unbiased, outside, opinion expressed about their systems development projects. Independent Verification and Validation (IV&V – for more information reference the IEEE-1012-2004 specification) is a standardized approach to providing these opinions that States are beginning to request contractor’s use. A key tenet of IV&V is “Independence”. That is the ability for your IV&V contractor to give you an opinion about your systems project without being biased or unduly influenced.
We have seen many “IV&V” contractors play a game with Independence. They avoid procurement conflict of interest rules by partnering with a systems integration (SI) vendor in one State on the construction or implementation of a project and bid in another State to provide IV&V over the same SI vendor. Technically they aren’t working for the systems integration contractor in your State but in the larger picture they do receive money from the contractor they are charged with overseeing on your behalf. This, obviously, negates the independence they profess to bring to your project.
IV&V vendors do need to remain technically smart about system integrators, system developers, software solution vendors, and fiscal agents offerings so we’re not suggesting there be no communication. Just that no financial relationships exists between your IV&V vendor and these entities.
So what do you need? You need a consultant who does not contract with or have an obligation to system integrators, system developers, software solution vendors, and fiscal agents that will likely bid on your projects. You need language in your IV&V procurements that guarantees this independence.
Tired of getting reports from your quality assurance (QA) consultant that are just a long laundry list of all the things that might go wrong with your project? We’ve seen 20 page reports with over 200 “risks” listed. The likelihood of many of those risks occurring is about the same as the likelihood of getting hit by a meteor while standing in your front yard. Some are so immaterial that even if they happen they won’t affect the successful outcome of your project. The long list of mostly irrelevant items overwhelms clients and erodes confidence in the QA process. Usually the laundry list just ends up getting ignored.
QA consultants often don’t understand their role or feel their role is simply to point out all possible risks. Sometimes they are on autopilot just working to a checklist. We’ve even seen QA consultants invested in a project failing because it validates all the things they’ve said in their reports. The real job of a QA contractor is to make your project successful. They should do this, in part, by advising you of the things that matter.
The first page of a QA report should give you the top three to five risks you need to attend to immediately. There should be a strong rationale for choosing the “front page” risks (usually magnitude of impact, likelihood of occurrence, and immediacy of risk are the factors used). The report should also provide pragmatic suggestions for managing or mitigating the top risks. Demand the top issues are brought to your attention in a concise and useful manner. Focus on resolving the most important things before you even look at the other 195 risks.
Is your consultant having a difficult time meeting your expectations? Sometimes from a clients’ perspective it appears consultants are unresponsive to their needs and miss their expectations. Typically, consultants focus on “deliverables”. These are usually written reports that when delivered to a client trigger a payment. At times consultants do this to the exclusion of resolving the problem the client sees as critical. This is done because consultants:
- Get overly focused on the product they produce rather than the problem they are trying to solve; and
- Want to make sure they meet their contractual obligations, typically outlined in a formal “statement of work”.
You focus on solving “the problem”. However, sometimes that problem isn’t well defined or expectations about its resolution are inadequately communicated. Your up front expectations for the consultants work are not typically discussed “up-front”. Further, as your project moves forward and more is learned, the original problem defined in the contractual “statement of work” may not really be the problem you need to solve. This leads to a situation where:
- Consultants are not meeting the your expectations;
- The problem being tackled isn’t clearly understood or worse is the wrong problem to solve;
- Activities starts to vary from the contractual statement of work; and
- Tensions between you and your consultant rise degrading the cooperative atmosphere required for a successful consulting engagement.
On projects where we find ourselves varying significantly from the contractual statement of work, don’t fully comprehend your expectations, or feel we aren’t on the same page regarding the problem we are trying to solve we will develop a “Deliverable Expectation Document” (DED). The intent of this document is to clearly define key expectations around the “deliverable” that results from our work. The DED is a short statement that outlines the problem(s) we are trying to solve, what we think the end product of the task will look like (both the form and outline of the content), who the audience is for the end product, and where we are varying from the original statement of work.
The DED should be short (think a page or two) and concise. It must be developed as a collaborative effort between client and consultant. The DEDs are developed iteratively throughout a project just before work begins on a major task. Over the course of a large project with many tasks you might develop many DEDs. The process begins by imagining that a specific deliverable is complete and asking “what is different now than before we started.” Most of the real value comes not from the document itself but from the focused collaboration between consultant and client required to develop it.
There are pitfalls to avoid and some downsides. Some consultants and clients want to specify the outline and format of the end product more rigidly or in greater depth up front than can practically be done. It is a mistake to turn the DEDs into the deliverables themselves. Creating a DED is time consuming requiring key staff to find the time to get together in person. At other times DEDs are simply not necessary, for example, when you and your consultant are clearly in agreement on what needs to be done or when your consultant is really a “body for hire” (you direct all their activities).
We don’t suggest the use of DEDs on every project or for every deliverable. However, we have found them a valuable tool to avoid miscommunication about content, get the project focused on the problem that needs to be solved, and addresses concerns associated with variance from a contractual statement of work by providing a clear audit trail of decisions about scope . When appropriately applied they can help communicate expectations, save time, result in higher value products being delivered to you, and help maintain a collaborative productive relationship between you and your consultant.