Abstract
This article outlines a 10-step process for business leaders in the pharmaceutical, biotech, health sciences, and clinical fields, who hire information technology (IT) applications consultants to design, develop, implement, and integrate custom laboratory automation systems. The goal of this model is to identify steps to dramatically improve the effectiveness of the consultant and to reduce implementation risk factors. The probability of project success can be increased significantly when the basics of IT systems development is understood by those internal to the organization and management guides the consultant to the finish line using a defined project control process.
Keywords
Introduction
Health science companies spend staggering amounts of much sought after R&D dollars annually on laboratory automation technology. Solidly designed custom applications and information management systems are critical to the success of most businesses. If the information technology development experience is not available in-house, the typical approach is to look outside the company for experience. After all, if experts who make automation projects their sole business are readily available, why try to recreate the wheel when time is of the essence? Forming a partnership with an experienced IT development consultant and augmenting existing development capabilities can be just the right strategy for companies that need to reduce time to market and accelerate development schedules. This is especially true for corporations whose core business is focused outside the IT arena. Non-IT based businesses may not have readily available the software design or systems development expertise that is necessary to complete the project. Unfortunately many business leaders who outsource IT projects are not knowledgeable enough regarding the fundamentals that support information technology development, which can lead to mismanagement of urgent and expensive projects.
Corporations do not have to look far to find consultants who are IT “experts” that are available to find “solutions” to complex automation needs. There may be a sizable implementation expense involved since (1) the consultant is the expert, (2) the client has an intricate business and is in immediate need of the automation solution, and (3) custom software/systems development by its very nature is costly. All of this is easily understood. What is not easy to understand is why then, do so many outsourced IT projects fail although the consultants are seasoned experts who should know better? Horror stories abound of failed software or other IT systems projects; the larger the project the more costly and harder they fall.
Although IT failure rates are difficult to estimate (many companies do not want to publicize failure), surveys have estimated that only 26% of corporate information-technology projects are successfully completed. 1 Billions of dollars annually are wasted on these failed projects, not including lost revenue or missed business opportunities. The average loss in an abandoned project is $4.2 million and blowups in Computerworld's Top 10 list cost much more than that projection. 2 Placing blame for this financial disaster is not simple but ultimately it is the responsibility of the company that is seeking the development to manage and control resources. However, many companies choose to manage outsourced IT projects outside of their standard project management processes, without the demand for reporting and performance expected of internal projects. Most outsourced IT project failures are also plagued by the standard project demons such as lack of communication, vague objectives, scope creep, and lack of involvement from senior management. Rigorous project management principles are frequently left out of these projects since corporations assume that their “expert IT consultants” are responsible for project management and have the goals of the corporation deeply integrated although the client always has more at stake. Competent consultants DO apply thorough development processes and project management principles but only the client can assure that the company's business objectives are addressed.
In part, failure of outsourced IT projects is a result of the mystery associated with IT development and the lack of knowledge regarding key processes parameters. If the companies who outsource IT development fully understood the development process, most likely, they would not have hired the consultants in the first place. Core fundamentals associated with IT development, in combination with cross-functional project management techniques, are needed to break this cycle. Although the companies who outsource IT development do not need to become the experts, they can maximize the probability of a successful project by applying a robust management process.
PROCESS MODEL
Diagram A summarizes a 10-step model and structure for the management of outsourced IT development projects. The following sections discuss the rationale and suggested actions involved in each of the 10 steps.
Process model diagram for outsourced IT projects.
1. IDENTIFY SENIOR LEVEL PROJECT SPONSORS
Critical projects must have adequate support from executive management in order to be established. Once the project is approved, it is common to assign a single executive sponsor. Typically after executives have agreed to fund the project and budgets are allocated, the pressure for that particular project diminishes while other priorities move in. The executives, who agreed that the project was critical and should be funded, quickly disappear to focus on other corporate initiatives and await news of project completion from the Lone Ranger executive sponsor. Since key executives agreed that the corporation should fund the project, why are they off the hook once the spending begins? In order to maximize the success of the project, it is recommended that a senior level team with cross-functional representation (i.e., sponsor team) be created who will be accountable for the project. Note this team is not the project team but a team of high level project sponsors.
Ideally this executive sponsor team will have representation from R&D, Finance, RA, QA/QC, and Operations, but this will depend on the specific project. A company should create a multifunctional team that best represents organizational interest and be careful not to overlook functions that may be assumed to be inactive areas. Frequently the most valuable senior level contributors to the team will come from functional areas that may not fully understand the technical details of the project. Again, the selection of client sponsors will be based on the size and complexity of the project, as well as the impact of the business objectives that result from the project. Clients should carefully consider who should participate in sponsor teams and assure adequate representation.
2. ESTABLISH THE PROJECT REVIEW PROCESS, LINK TO SPONSORS, AND THE PROJECT TEAM
Identifying internal project team members typically happens quickly once the project has been approved for funding. The project team is officially off and running, scrambling to develop schedules and identify milestones. Project review processes and links to senior management are an afterthought, especially for outsourced projects that fall under IT consultants. Prior to initiating the project, executive sponsors should outline the review process and define the boundaries between the sponsor team and the project team. Clearly articulate the decision boundaries, roles and responsibilities, and the expectations for each team, as well as expectations regarding interaction between the teams. Establish routine review meetings that are formal and well defined. A summary of recommended decision boundaries and review meeting objectives are summarized in Diagram B and Table 1 respectively.
Boundaries for sponsor team and project team.
Review meeting objectives and team responsibilities summary.
Identifying team members and establishing the project team is the responsibility of the sponsor team. Project team members from the client organization should represent the major functional areas involved within the scope of the project. The project team should include project leadership from both the client and the consultant but ultimate authority is assigned to the client's project leader. Be mindful that once the project is over, only the project leader from the client is accountable for post-implementation performance (unless the contract with the consultant specifically includes a warranty or post-implementation support). For projects that are large in scope and/or budget, it is imperative that the client assigns or hires an internal “expert” who can manage and challenge the consultants. One of the biggest mistakes non-IT technical clients frequently make is when client representatives on the project team do not truly understand the IT side of the project. It is common for clients to assign someone from R&D to manage the project team and consultant although this project leader may not have IT experience. The decision to outsource an IT project is frequently the best option for corporations not focused in IT development. However, not including at least one experienced technical resource independent from the consultant's team can be a very expensive mistake. Again, the key is including someone (in the project team) from the client's organization who is experienced in the technology being outsourced. If the project is critical, it is not enough to settle for an existing resource that has close skills; it is well worth the investment to hire someone new who is an optimal choice.
The sponsor team must set the standards and expectations for the project team. They must also understand that they completely own and are accountable for the project. If the project fails, then the sponsor team failed along with the project team. If the project is a roaring success, both teams succeeded. The sponsor team mentors the project team and guides them throughout the project. The goal of the project review process is to make the interaction between the two teams as efficient and painless as possible. Keeping the review focused on a simple agenda and list of deliverables will streamline and expedite the meetings. Once the sponsor team establishes decision boundaries, it is their responsibility to make decisions quickly. Giving immediate feedback to the project team at the end of the project review meetings will prevent unnecessary delays. As an example to illustrate this point, two areas typically of interest to the sponsor team are schedule and scope. If the project team finds that they are slipping behind schedule, they should present a list of options to the sponsor team that would include an analysis of the following:
The current scope of the project is unaffected, and the new timeline for delivering the project is estimated.
The current scope of the project is amended; less critical features are delayed to the next phase of the project and the timeline for delivering the project is unaffected.
Resources need to be added to the project in order to maintain current scope and timelines.
In addition to summarizing the options, the project team should also recommend an action. The key goal here is to make decisions - immediately, or as quickly as possible, in order to keep the project moving forward.
The project team must manage and implement the project. They are responsible for the daily grind of development and implementation. It is up to the project team to assure that the sponsor team is informed of all items that may cross into their decision boundaries (Diagram B). But it is also the responsibility of the project team to make all decisions that are within their decision boundaries without consultation from sponsors. Decisions that are technical or tactical in scope are the responsibility of the project team, as well as risk management. Identifying and analyzing the risks associated with the project is as critical as managing the resources, technologies, schedules, and budgets. Risk management is the most elusive project activity when working with consultants. Just like internal project teams, projects outsourced to consultants (no matter how much of an expert they may be) have critical risk components and additional risk associated with the consultant's ability to integrate into the processes and business of the client.
3. DEFINE HIGH-LEVEL REQUIREMENTS
Defining the high-level requirements for the automation project and creating a summary document is the first task for the client project team representatives and the first deliverable to the sponsor team. During this early phase of the project, the team should fully explore the goals of the organization and “what,” at a high level, is to be developed (from the end-user perspective). In order to define features to include in the new system, the project team conducts interviews with users and maps processes. The summary includes identifying what is important today and what the vision is for the future. This initial overview should concentrate on functionality, known constraints and limitations (i.e., schedule, hardware, operating system, budget, etc.), operational/performance parameters, and overall objectives for the system. Rating each requirement will help define what needs to be developed first and what can wait for later releases.
4. PREPARE A REQUEST FOR PROPOSAL (RFP) AND SELECT THE IT CONSULTANT
The RFP will be the beginning of the relationship with the outside contract organization and this initial impression will set the tone of how business is conducted. If the request is thorough and precise, the consultant will expect this level of scrutiny throughout the project and they will know that they are not dealing with IT amateurs. Organizations routinely solicit IT consultants armed only with a draft of system requirements. The high-level requirements will be the “meat” of the RFP; however, being specific in the information requested will provide insight in the consultant's work product and provide invaluable information to assist in selecting qualified professionals. Make consultants work upfront to provide the best possible overview of the project. Cost should not be the only selection criteria; specifics related to the consultant's organization and their abilities are necessary to complete a thorough evaluation.
Summarized below are recommended components for an RFP; the components may be modified based on the size of the project:
Client/Hiring Organization Background Summary
Organization overview/business summary
Project background and timeline for selecting a consultant
Key contacts and reporting relationship
Description of Requirements
Project scope and summary
“Mandatory” requirements and “desired” requirements
List of project deliverables, services requested, and project constraints
C. List of Specific Items to be Included in the Proposal
Consultant's corporate summary
Description of consultant's experience
Description of project approach
Description of how mandatory and desired features will be addressed
Proposed organization chart for project team (include resumes, description of responsibilities for team members, and location of work)
Proposed development methodology and activity schedule (include project phases and milestones)
Proposed methodology for tracking performance
Proposed project cost structure - fixed price vs. time and materials, development resources, etc.
Once all the proposals have been collected, the project team has the responsibility to analyze the responses and grade each based on evaluation criteria. Establishing a quantitative points rating system in advance of any review will streamline and standardize the selection process. Keeping the team focused on a quantitative rating process will keep emotion out of the selection process and apply a level of unbiased analytical evaluation to the selection process. The criteria and points applied to the rating will depend on the objectives of the hiring organization and project specifics.
5. OPTIMIZE THE CONTRACT
No one likes contract review, although a thorough contract will protect the hiring organization and keep them safe from unnecessary spending. Since the client has the most to lose if the project goes awry, and the client can best negotiate up front in advance of starting the project, one would assume that clients would never start projects without a contract. Very often this is not the case; clients procrastinate potentially the most important project component and initiate the project prior to finalizing a contract. The longer it takes to complete the contract, the more the IT consultant knows about the client's project and the more important they become which means that the client loses negotiation power. Make the completed contract a required artifact to start the project.
The standard contract specifics are best left to the attorneys but listed below are a few items to consider that are of interest when creating a contract for IT consulting/development for large-scale projects (it is recommended that clients hire an attorney who specializes in IT-related contracts to assist in drafting and negotiating the agreement):
Roles and responsibilities. Clearly define the roles and responsibilities, for the client and the consultant. It is not enough to simply state the scope of work.
Resources and cost summary. Define the dedicated skills and resources that will be provided by the consultant over the life of the project. Summarize expense associated with resources and other estimated project related expenses.
Key performance measures, quality standards, and schedule milestones. Complete this item as much as is feasible for the specific project. This can be difficult for complex projects that may include participation from several contributing sources. Assign quantifiable goals and keep the goals confined to short intervals. Expanded intervals between goals leave too much time for the project to potentially coast without any warning flags. Review areas where an incentive plan can be implemented to award on-time achievement of goals.
System warranty. At a minimum, the consultant should agree to fix discrepancies from the requirements (bugs) for a fixed time period post-release of the system. This is not the same as system support.
Billing rate for post-launch system support. Even if the new system is being supported in-house, negotiate the rate in advance so that in the event support is required, the client will not be in the position of negotiating for services that the consultant will know can't realistically/quickly be attained anywhere else.
Post-Phase I development. If a Phase II release is planned, negotiate services as much as possible and include the fundamental agreements in the Phase I contract (with an out option in case the client is not satisfied at the end of Phase I).
Disaster recovery. Request that a disaster recovery plan is included as an appendix. Clients may also want to request copies of code/system backups for storage at their facility.
Technology guarantees for third-party applications, development platforms, or tools selected by the consultant. Protection is necessary just in case the consultant selects an unproven technology to integrate into the system and suddenly the client is left with an application with limited support options or one that doesn't work like the consultant planned.
Operational performance criteria. Specify the critical performance criteria such as the volume/capacity (or number of users) expected out of the system. This measure should cover an extended time period, not just initial release. If the system is required to be scalable, this scenario should be covered in the contract.
Ownership of custom system/application and data associated with the body of work, and the intellectual property rights. This is different than standard confidentiality or exclusivity clauses and should be specifically stated.
Systems and/or data conversion expectations. If an existing system is being replaced, specifically detail the role the consultant will assume regarding data conversion or migration.
Audit rights. Define project audit rights based on the requirements of the client. Specify the expected project review processes or audit criteria/frequency.
Development location. Be specific in where the consultant is allowed to perform the work. The client may require a portion (or all) of the development to be completed on-site while the consultant may want to complete all work from their office.
Change procedures. Even internal projects can be plagued with “scope creep” if not carefully monitored. The consultant should be an expert at managing change procedures and all changes to specifications should be documented and approved. Cover the expected change management process.
Communication plans/schedules. Be specific in how, what, and when communications take place. This may relate to project team meetings, project review meetings, etc.
6. DEVELOP SOFTWARE/SYSTEM REQUIREMENTS SPECIFICATION (SRS)
Clients and consultants both will agree that the single largest cause of IT project delays and budget overrun is incomplete/incorrect understanding of project requirements. Steve McConnell states it best…“For each requirement that is incorrectly specified, you will pay 50 to 200 times as much to correct the mistake downstream - during the coding - as you would pay to correct the mistake at requirements time.” 3 Ask the consultant to train the project team on the process they use to generate detailed requirements. This should be a thorough and well organized process designed to uncover features and functions of the system. The client will generate the high level list of desired functionality and facilitate access to the users who have the most innate knowledge of what the system should do, but the consultant should be the expert on requirements engineering. Requirements must be completely understood and described for the software developers who ultimately complete the project; therefore requirements should be in their language as well as the client's language. This document is often termed an SRS, software/systems requirements specification.
There are three broad categories of software requirements - functional requirements, non-functional requirements, and design constraints, described below:
Functional requirements express what the system does, describes the inputs and outputs, general features of the system, keeping in mind the user of the system.
Non-functional requirements specify system attributes; for example: throughput, performance, usability, look and feel, reliability, and supportability.
Design constraints impose limitations on the system or process used to complete the project. These constraints may not affect the behavior of the system but are necessary for technical, business, or contractual obligations; for example: budget range, schedule constraints, technology selections, and platform limitations.
As much as possible, requirements should be verifiable and testable. All requirements should be individually numbered in order to eliminate confusion and to facilitate the completion of a traceability matrix. This matrix will trace all requirements to test procedures that verify the requirement was properly implemented. A matrix should be a project deliverable from the consultant. Both the client and the consultant should formally approve the final software requirements specification document. This document becomes the official bible for the project and anything not explicitly included will be outside of project scope and under change management procedures.
Regarding software requirements, for projects large in scope it may be worthwhile to consider requirements engineering as a completely separate phase of the project with a limited contract. Continuation of the project with the same consultant will be based on successful completion of the SRS and working relationship with the client. Scope and price this phase separately since the final SRS document can always be used to generate a more specific RFP. If the client is less than impressed with the consultant during this period, they are under no obligation to continue and all is not lost. Ideally, the same consultant will continue in the project but it is advantageous to the client to have options.
7. INSPECT PROCESSES AND REVIEW PROJECT DOCUMENTATION
The process that the consultant follows during the course of the project is the roadmap for development. When the FDA performs an inspection, they ask for the process and then they review how the process is implemented. Yes, they look for core components of the process but equally important is how well the process that is defined is being followed. The same applies to systems development. The consultant should have a “roadmap”; there are many ways to arrive at the same destination but some routes are more direct and efficient than others. The consultant should train the project team on their recommended development process for the project. Part of the training should include a summary of all deliverable documentation and standard operation procedures that the consultant follows. Once the entire team understands the process, the team should include periodic audits and process review milestones in the project schedule.
There are many valid and effective approaches to IT systems development and selection of the approach will depend on the size, complexity, and nature of the project. A defined software process contains a coherent, integrated set of software engineering and management procedures. A well-defined process can be characterized as including readiness criteria, inputs, standards, and procedures for performing the work; verification mechanisms, outputs, and completion criteria. Because the software process is well defined, management has good insight into technical progress on the project. This process can be summarized as standard and consistent because both software engineering and management activities are stable and repeatable. 4
The following standard documents are recommended as part of a robust development process:
Software Requirements Specification
Management Plan
Software Development Plan
Project Plan/Schedule/Milestone
Design and Architecture Documentation
Risk List and Risk Management Plan
Iteration Plan(s)/Integration Strategy
Test Plans/Quality Plans
Traceability Matrix
Technical Incident Reporting Procedure (Bug Tracking)
Configuration Management Procedure/Version Control Plan
Change Management Procedure
The client's representation on the project team may not fully understand how to develop software but they do understand the importance of establishing processes, planning, and documenting activities. These concepts are not new and this experience can be applied to systems projects.
8. IMPLEMENT REQUIREMENTS AND ANALYZE SYSTEM OPERATION COST
Implementation of the system requirements is the core of the development process and includes architecture/design, coding, and testing the software. Historically systems development has been described as building software when in fact most systems are far too complicated to be accurately described in advance; they require too much flexibility to completely specify every detail and are too complex to be built faultlessly. This metaphor is changing into one of grow, not build, software. 5 Incremental development and testing as the system is being created is a recommended approach to minimize project risk. Both the client and consultant should be involved in the testing activities as the system is growing. Applying an approach of controlled iterative development will allow the project team to understand and explore risk early, continuously integrate added functionality, and manage the project by prompting tactical changes.
Essential to tracking the performance of the project team during implementation is the establishment of milestone deliverables throughout the project schedule and Key project measures that chart performance to goal. For example, milestone deliverables may include architecture and design documents, data models, documentation of all test cases to be used during validation, or coding of modules that include specific functionality critical to systems development. Key project measures may include adherence to schedule/tracking of milestone delivery dates, defect tracking parameters (number of open defect reports, rate of closure of defect reports, etc.), and change management parameters (number of open change request, rate of closure of change requests, etc.).
Corporations who are considering hiring an IT consultant may have a need to replace an outdated legacy system/process with a new and improved, more efficient automated system/process. Many companies assume that since their new system is more efficient, the system must also be cheaper to operate. This may be true but assuming it is fact without first estimating the associated expense of the new system may leave the corporation open to budget overruns. The consultant will be at a stage to estimate project operational expenses as soon as the first pass design is complete. Include the following in the estimate of operational expenses:
Estimate the IT support staff required to manage the new system. Skill sets should be described with enough detail that the client can assess exactly who should fill the role. Without sufficient information from the consultant, the client may be under the impression that existing IT support staff can manage the new system to later find out the skills are very different.
If the system includes operators, processing techs, etc., summarize the skills required. Existing staff may seamlessly move from the old system to the new system but depending on the requirements, supplemental staff and/or an upgrade in the base skill level may be required.
If the project requires the purchase of new hardware, estimate hardware expense and analyze the current facility in respect to the new system. Estimate the expense associated with any necessary facility modifications and/or enhancements.
Summarize the third-party support contracts needed, and the associated expense and billing frequency.
Finally, if the client's objective is to keep operating expenses fixed or reduce expenses, define this as a constraint and communicate expectations up-front to the consultant.
9. USER ACCEPTANCE TESTING (UAT)
As the system is being developed, the project team is testing … unit testing, integration testing, system testing, etc. The final test for the system should be one by actual users: user acceptance testing or beta testing. No matter how much testing the project team conducts, novice users always find new and innovative ways to break the system. If users have been involved in the project up to this point, they should not uncover any major “design” issues in this test. This test cycle will allow the user to explore all areas of the system and bang on the features. The project team should prepare a basic protocol that guides the test group through the basic functionality of the system. Giving the users an established protocol will assure that they complete the minimal “banging” but also allow them time to test the system randomly.
During UAT is not the time to gloss over issues or rationalize user acceptance. It is too late to turn back and start over but it's not too late to fix genuine concerns. If users are complaining now, they will complain later (and louder) when the system is released. Hold the consultant accountable to fix issues uncovered during the 11th hour. Depending on the nature of the issues, some modifications may be planned for Phase II and the initial release/project approval can continue as planned. This may be the best approach if users identify issues that are change requests (i.e., functions or issues not defined in the software requirements specification).
10. SYSTEM RELEASE STRATEGY
If not managed properly, projects can progress well up to system release; then suddenly fall apart. The risk associated with releasing the system will vary depending on the scope of the project, but risk can be minimized. Risk management plans can save the project, but most project teams are so anxious to release the system that they omit this valuable planning step. Issues can come not only from the system, but also from insufficient user training or lack of skilled technical support for users.
Examples of system release strategies designed to minimize risk are summarized below:
Limited launch. Select a limited cross section of users for initial release. Either continuously expand the user base at a rate defined by the success of the system, or move to the new system when all functionality is proven to perform per specification. This type of approach will be useful if the system has not been tested at full capacity.
Parallel systems. Run the old system and new system in parallel for a limited period, splitting the user base over both systems. This will allow direct comparison of the functionality of the systems. In addition, this may allow the release of the system to be phased in such way that there is no need for migration of existing data. It is always less risky to start the new system by phasing in utilization without moving data from the old system to the new system.
Delayed functionality. Delay the release of secondary features and complete testing in parallel with launch of core functionality. Focus first on what is absolutely critical in the new system. If the project is under a time crunch to release, taking the most conservative approach to the release will satisfy the sponsor team and allow the project team additional time to test secondary features.
During systems release it is important to fully involve the consultant. Fixing all problems with functionality detailed in the requirements (SRS) should be the responsibility of the consultant but typically consultants push for project sign-off/approval once the system is released. It will be difficult to get the consultant's original project team back once the system is approved. Thorough negotiations during the contract phase will assure that the consultant remains on board until all issues are ironed out.
Summary
In summary, hiring an IT consultant is frequently the best approach to accomplish critical laboratory automation projects or other laboratory data management objectives but this can present challenges to management not familiar with information technology development activities. If adequate expertise is not available in-house, the most cost effective and efficient way to accelerate the development of new systems may be through use of an outsourced IT consultant. There are many experienced consultants to select from and picking the right consultant for the project should be an analytical decision based on both cost and ability to implement. If a consultant with adequate expertise and skill is selected, then how well the project will be executed will depend largely on how well the consultant is managed. Outsourced projects should be subject to stringent project management practices, maybe more than internal projects, as outlined in this 10-step project control process. A client must fully engage senior level business leaders, include applicable internal representation in the project team, and apply corporate knowledge of both technical development and process requirements. The probability of project success will increase if the client is familiar with the basics of systems development and can help guide the consultant to the finish line using a defined project control process.