Why no less?  This one is kind of obvious.  If your consultant is doing less than you need you won’t get what you require.  Consultants will do this consciously if they are behind schedule and/or over budget.  We see it when we are providing QA for large system projects.  For example, the development vendor is behind schedule/over budget and they try and convince the client that they don’t need as much system testing or that the bugs can be fixed after the system is implemented.

Why no more?  There are really a few reasons here.  First if your consultant is being paid by the hour – it’s costing you money!  The more subtle but important reason is even if your consultant is on a fixed price contract doing more costs time.  Your projects won’t finish on time.  Allowing your consultant to do more may also distract them from doing their best on more critical tasks.

How do you make sure your consultants are within scope?  Here are a few simple suggestions that will go a long way to making sure what you need is delivered on time and within budget:

• First every project no matter how small should have a work plan.  This can be something as simple as a to-do list or as complex as a full blown task plan identifying task, task dependencies, timing, work effort, staff assignments etc.  Don’t worry the technicalities; your consultant should make their task plan available to you.
• You should review the plan before work begins and clearly understand how each to-do or task contributes to the end product you need or want.  If there are tasks that don’t appear to contribute ask your consultant why they are in there.  If something appears missing ask why it’s not there.  Remember every work step should contribute to the end result you hired the consultant to produce.  Be persistent until you get an answer you can understand.
• You should review the progress against plan on a regular basis with your consultant.  Regular status meetings are a good place for this.  Get them to tell you what they’ve done from the plan.  Be careful they’re not doing work that isn’t listed on the plan.  If they do things not on the plan ask them “why?”  If they are skipping tasks on the plan ask them “why?”  There may be very legitimate reasons.  Perhaps they forgot something that should be in the plan (have them add it to the plan) or there are things in the plan that really don’t need to be done (have them explain why and then remove it from the plan).  If there is work you think should be on the plan but isn’t raise the issue with your consultant.
• Last, as your project closes an easy way to verify your consultant has done what they promised is to review the plan to verify that each task is complete.  This doesn’t guarantee what they did meets your needs – just that they did what they said they’d do.

Shouldn’t managing the work plan be the consultant’s responsibility?  Absolutely.  We’re not suggesting you manage your consultants plan – what we are saying is you need to continually verify the work being performed will get you what you need.  Believe it or not consultants don’t always think about whether the work they are doing is leading to the result you want.  Many consultants get bogged down in the day-to-day work and loose sight of the big picture.

It is prudent for you as the buyer of consulting services to verify the project scope will lead to the result you want.

This is part of a series of articles designed to help clients and their consultants have more effective and efficient engagements.

At the start of every engagement you need to develop a clear set of expectations describing what you think will be different when your consultant is finished. An expectation can be something as simple as you’ll no longer have to deal with a specific personnel issue or something complex like you’ll have a fully functioning Medicaid system. You’ll likely have more than one expectation.

How do you go about figuring out what your expectations are? Use your imagination and pretend it’s 6 months after the consultants have completed their work. Everything is, of course, perfect from your perspective. Now ask yourself “What’s different now than when we started? What changed?”. Write down your answers. These are your initial expectations.

You need to share these expectations with the consultant at the start of the project (note: good consultant’s won’t wait for you to share it – they’ll try and drag your expectations out of you the first time you meet).

Does this mean the consultant will meet all your expectations? Of course not. When you present your expectations good consultants will inject their experience and knowledge into the conversation (that’s the reason you hired them right?). You’ll jointly develop a set of mutual expectations for your project. This is a good thing – it gets both you and the consultant bought into a common set of outcomes for the project. The jointly developed set of expectations should be written down used to regularly assess project progress and ultimately the success of the project.

Projects without a set of shared expectations wander ending up “somewhere else”. The question for you is “do you and your consultant know what will be different when the engagement is done?” If not perhaps it’s time to develop some mutual expectations.

This is part of a series of articles designed to help clients and their consultants have more effective and efficient engagements.

It significantly benefits both client and consultant if a single individual, from the client, is responsible for overseeing the consultant.  Typically this “client manager”:

• Gets regular status updates from the consultant;
• Coordinates the consultant activities within the client organization;
• Ensure any products the consultant delivers meets agreed upon timeframes, standards, and need;
• Communicates relevant information to the consultant and within the client organization;
• Handles any issues and problems that occur during the consulting engagement;
• Reviews and approves consultant invoices and manages financial issues;
• Handles contractual issues (or takes issues to the contract manager); and
• Fires the consultant if they are not performing.

Who this individual is depends on the nature of the engagement.  Sensitive organizational assessments or reorganization projects might report to the head of HR or the agency director.  On our QA projects we typically report to the project sponsor.  For coding projects it makes sense to have your consultants report to the development or project manager.  One way to think about this is to think about who will pay the price if the consultant (or project) fails – that is the person that should be in charge of the consultant.

Some common mistakes we see in setting up a client manager include:

• The client manager is him/herself a consultant rather than an agency employee;
• Consultants report to a “committee” of some kind (we’ve found this has the same result as having no one in charge); and
• The client manager is a low level staff person with little or nothing riding on the engagement outcome.

So the question for you is who is in charge of the consultants around your organization?

This is part of a series of articles designed to help clients and their consultants have more effective and efficient engagements.

• Are tired of consultant bait and switch strategies where superior resources are bid (and may even start) on the project but less experienced staff show up and do the bulk of the work;
• Want rapid access to known consulting resources for projects; and
• Want to make sure there’s continuity on project issues and tasks provided by the consultant.

There are however downsides to having consultants devoted exclusively to your project. Consultants:

• Become myopic over time and miss critical issues and solutions a fresh pair of eyes might catch;
• Are costly resources that should only be on board when they are needed, when they’re not fully utilized (and there are always those down times on projects) you don’t want to pay for them;
• Stop bringing in new ideas and experience gleaned from other projects that might be just what you need to thrive; and
• Become viewed as staff and their unique experience and voice is no long heard.

Of course we’re not suggesting the elimination of consistent consulting staff on your projects. You do need continuity and responsiveness. And we are also not suggesting consultants should be swapped out regularly. Just be aware there are hidden costs to demanding consultants be devoted full time to your project (particularly on longer projects) and benefits to having “fresh legs” in the marathon that projects can be.

Takes over for experience. Vendors will try and bury you in details about how rigorous their process is so you’ll hopefully overlook the fact they are staffing your project with individuals that have barely used software let alone written it, performed quality assurance on it, or managed it’s development.   The truth is effective and efficient IV&V is art as much as science.  You need qualified consultants as much as rigorous process.  Make sure you examine who will staff your project very closely, asked to see the results of each individual’s previous IV&V efforts.

Is used as marketing hype.  Vendors will try and convince you they are “scientists” or “engineers” performing science or engineering on your projects/software. After all, V&V was pioneered by NASA and, for software IV&V, the base standard is defined by the IEEE (Rocket Scientists and Engineers) so who better to perform it?  IV&V is, conceptually, pretty simple:

• Identify the products and processes to undergo verification and validation (preferably before or in the early stages of development),
• Determine the criteria with which each of those products and processes can be evaluated (starting with standards where available e.g. the IEEE-1012-2004 standard for software verification and validation),
• Assess the products/processes while in production and upon completion to verify they meet the predefined criteria and note where they don’t,
• Re-assess products/processes when deficiencies have been addressed.

IV&V is hard work that takes expertise (there are many nuances in developing criteria and evaluating products etc), independence, and yes a degree of rigor.  But it’s not “Rocket Science”.  We’ve seen a significant amount of rigor added to an IV&V process that adds little value to the client and seems to be there to convince the client that IV&V is engineering and they are hiring the best “engineer”.  There is no professional certification of IV&V “engineers” like there is for electrical or civil engineers.  Don’t buy into the hype that this line of reasoning purports.  Understand you need competent, experienced professionals but this is not a mystical art.

Is used as an excuse for lack of social and political savvy of vendor staff. Beyond competence in performing IV&V your consultants need communications skills and an understanding of the political landscape public sector projects face.  We’ve actually witnessed an IV&V vendor reporting to a state legislature stating strictly facts (“the project is 2.3 months behind schedule” and “523 out of 1232 test cases failed”) without explaining context (“critical features were added” and “we’re only a quarter of the way through testing”).  The poor agency receiving the “benefit” of IV&V spent the next month trying to keep funding for what was a thoroughly successful project.  When called on the carpet about this the IV&V vendor explained that their rigorous methodology required they describe only the facts not the context of those facts.  Just what you need, Joe Friday (“Just the facts ma’am”).

Creates unnecessary bureaucracy that just wastes your money. Sign-offs of intermediate IV&V products, large review meetings, and repeated reviews of materials provide you with little value.  Often this “rigor” is applied not for your benefit but for the consultants, to ensure their rear is covered (“but you approved the interim deliverable”) or to enhance consultant cash flow (“you approved the first interim IV&V checklist so we can bill you”).  This unnecessary bureaucracy simply costs you time and money.

IV&V helps your software projects succeed and should be rigorous – meaning thorough, focused, and disciplined.  However, be aware “rigor” can be used for the wrong reasons.  When working with your IV&V consultant or the next time you’re reviewing an IV&V proposal keep in mind excess “rigor” provides little if any value; is used to cover a vendor’s weaknesses or benefits the vendor not you; and unnecessarily raises the costs of IV&V services.

We have seen many “IV&V” contractors play a game with Independence.  They avoid procurement conflict of interest rules by partnering with a systems integration (SI) vendor in one State on the construction or implementation of a project and bid in another State to provide IV&V over the same SI vendor.  Technically they aren’t working for the systems integration contractor in your State but in the larger picture they do receive money from the contractor they are charged with overseeing on your behalf.  This, obviously, negates the independence they profess to bring to your project.

IV&V vendors do need to remain technically smart about system integrators, system developers, software solution vendors, and fiscal agents offerings so we’re not suggesting there be no communication.  Just that no financial relationships exists between your IV&V vendor and these entities.

So what do you need? You need a consultant who does not contract with or have an obligation to system integrators, system developers, software solution vendors, and fiscal agents that will likely bid on your projects.   You need language in your IV&V procurements that guarantees this independence.

QA consultants often don’t understand their role or feel their role is simply to point out all possible risks.  Sometimes they are on autopilot just working to a checklist.  We’ve even seen QA consultants invested in a project failing because it validates all the things they’ve said in their reports. The real job of a QA contractor is to make your project successful. They should do this, in part, by advising you of the things that matter.

The first page of a QA report should give you the top three to five risks you need to attend to immediately.  There should be a strong rationale for choosing the “front page” risks (usually magnitude of impact, likelihood of occurrence, and immediacy of risk are the factors used).  The report should also provide pragmatic suggestions for managing or mitigating the top risks.  Demand the top issues are brought to your attention in a concise and useful manner.  Focus on resolving the most important things before you even look at the other 195 risks.

• Get overly focused on the product they produce rather than the problem they are trying to solve; and
• Want to make sure they meet their contractual obligations, typically outlined in a formal “statement of work”.

You focus on solving “the problem”.   However, sometimes that problem isn’t well defined or expectations about its resolution are inadequately communicated.  Your up front expectations for the consultants work are not typically discussed “up-front”. Further, as your project moves forward and more is learned, the original problem defined in the contractual “statement of work” may not really be the problem you need to solve. This leads to a situation where:

• Consultants are not meeting the your expectations;
• The problem being tackled isn’t clearly understood or worse is the wrong problem to solve;
• Activities starts to vary from the contractual statement of work; and
• Tensions between you and your consultant rise degrading the cooperative atmosphere required for a successful consulting engagement.

On projects where we find ourselves varying significantly from the contractual statement of work, don’t fully comprehend your expectations, or feel we aren’t on the same page regarding the problem we are trying to solve we will develop a “Deliverable Expectation Document” (DED). The intent of this document is to clearly define key expectations around the “deliverable” that results from our work.  The DED is a short statement that outlines the problem(s) we are trying to solve, what we think the end product of the task will look like (both the form and outline of the content), who the audience is for the end product, and where we are varying from the original statement of work.

The DED should be short (think a page or two) and concise. It must be developed as a collaborative effort between client and consultant. The DEDs are developed iteratively throughout a project just before work begins on a major task.  Over the course of a large project with many tasks you might develop many DEDs.  The process begins by imagining that a specific deliverable is complete and asking “what is different now than before we started.”  Most of the real value comes not from the document itself but from the focused collaboration between consultant and client required to develop it.

There are pitfalls to avoid and some downsides. Some consultants and clients want to specify the outline and format of the end product more rigidly or in greater depth up front than can practically be done. It is a mistake to turn the DEDs into the deliverables themselves. Creating a DED is time consuming requiring key staff to find the time to get together in person.  At other times DEDs are simply not necessary, for example, when you and your consultant are clearly in agreement on what needs to be done or when your consultant is really a “body for hire” (you direct all their activities).

We don’t suggest the use of DEDs on every project or for every deliverable. However, we have found them a valuable tool to avoid miscommunication about content, get the project focused on the problem that needs to be solved, and addresses concerns associated with variance from a contractual statement of work by providing a clear audit trail of decisions about scope . When appropriately applied they can help communicate expectations, save time, result in higher value products being delivered to you, and help maintain a collaborative productive relationship between you and your consultant.