Abstract
There are some issues contained in this paper that I would like to comment on and would welcome feedback from the authors of this tutorial paper.
References cited
The authors refer to ISO 9000 standards but these are very general and not typically applicable to laboratory equipment and systems without substantial interpretation. My suggestion would be to refer to ISO 17025 where the ISO 9000 Quality Management System is specifically adapted for testing and calibration laboratories and accreditation. This standard also assesses the competence of the staff as well. Section 5.5.2 1 requires specifications of laboratory equipment to be defined and tested against these requirements, and stating that “Before being placed into service, equipment shall be calibrated or checked to establish that it meets the laboratory's‡ specification requirements and complies with relevant standard specifications”.
This section has a specific footnote “†This is usually not the same as the manufacturer's specification”. Therefore, there is a need for a user requirements specification that must be written by the laboratory that reflects the intended use of the system, which is not explicitly mentioned or emphasized in the paper. Other specifications in the equipment sections of ISO 17025 require software should be validated.
The Good Laboratory Practice 2 and Good Manufacturing Practice (GMP) 3 regulations in sections 58.61 and 211.63, respectively, imply a specification to demonstrate that the system meets it's intended use. The authors appear vague about the need for a user requirements specification in this tutorial suggesting that a vendor would be better placed to write the document. In fact the laboratory, for both business (meeting laboratory objectives and investment protection) and regulatory reasons, should write the specification. 4
There is no reference to the Furman paper, 5 which is the basis of the holistic vs modular approach to qualification of computerized laboratory equipment in a GMP environment. The authors were FDA employees, and they advocated both that the individual modules of a system should be qualified and that a system (holistic) test should also be conducted to demonstrate that the overall system works.
Use of third partie
Page 49, Column 1: the authors note that … the entire qualification may be performed by an external company. As a consultant who is involved extensively in both laboratory equipment qualification and computer validation, this, in my view, is wrong. A third party may help with the qualification/validation of a system, but without the laboratory input, often significant, on defining the ways of working, this approach will be a waste of time and effort. The authors are correct with their statement that the overall responsibility for the work rests with the Qualification Committee within an organization, which will include the business process owner.
Equipment qualification and computer system validation
The title of the article refers to qualification of robotic equipment with IQ, OQ, and PQ terminology used. However, there is typically a large software component within a robot that has also to be validated as part of the same process of bringing the system into operational use. Here is a problem in that equipment qualification and computer system validation use the same terms but they mean different things, this is best shown graphically in Figure 1.
Differences between equipment qualification and computer system validation terminology. 4
The problem with the difference in terminology has arisen because a robot has both equipment and software, so both equipment qualification and software validation need to occur before the system is released for operational use. This is confused by the use of “qualification” and “validation” without definition of either term in the tutorial article.
I am confused with the meaning of the statement on Page 49 at the end of Qualification Program that states “The performance of the equipment during actual use in the laboratory fits with the specifications of the manufacturer. This is called PQ”. ISO 17025 notes that the specifications used by the laboratory differ from that of the vendor. Perhaps the authors should have stated “The laboratory must test the equipment or computerized system as they use it and not how the vendor has tested it (i.e. in the laboratory's operating environment, using the laboratory's analytical methods, specifications, sample capacities, and using the laboratory's networks)?”
Performing the OQ, here the authors suggest that the purpose of the OQ is to measure performance but there is no mention of any associated devices, samples or reagents. In many robotic systems there will be a need to check with reagents to demonstrate that a pipette operation within specified and scientific limits is sound. There also needs to be a holistic or system test to show that the whole system works as required as suggested by Furman. 5
The need for a requirements specification
My biggest issue with the paper is the apparent omission of specifications for the equipment and the associated software because this is the foundation of all equipment qualification and computer validation activities. 6 There needs to be more than a manufacturer's specification because robotic systems usually come with a language for the laboratory to program a robot to perform the laboratory method. This laboratory configuration or customization needs to be specified and documented. The authors mention user requirements in the OQ definition on Page 51 (“Testing the equipment during normal use reveals if the various tasks described in the users specifications are being performed by the robot properly”) but specification documents do not appear in the flow charts or the main text of the tutorial.
I welcome comments from the authors.
Yours sincerely,
Author's Response
We agree with the majority of comments offered by Dr. McDowall of McDowall Consulting and thank him for his candor. We would like to point out that the ISO-17025 standard referenced by Dr. McDowall is limited to testing and calibration laboratories. Our tutorial is dedicated to the qualification of robotic laboratory equipment, which does not always produce analytical results. As such, the approach that we outline is meant to apply more generally. While software is almost always associated with laboratory robotic equipment, the validation of software applications was not our primary focus. Figure 1, extracted from Dr. McDowall's book, presents an interesting perspective, but it is not part of the official reference. We do not agree with the distinction made between equipment and computer qualification because, in our case, the software is embedded in the robotic equipment and therefore is tested during operational qualification.
It was not our intention to suggest that laboratory robotic equipment qualification can be turned over completely to a third party. Any laboratory must have significant input into the qualification process. However, we do believe that third parties can carry out qualification procedures given detailed specifications that have been drawn up by the laboratory. This is similar to the way third parties can be contracted to develop laboratory equipment given detailed equipment specifications.
Sincerely,
Tanguy Le Neel, Ph.D.
Alain Truchaud, Ph.D.
Universite de Nantes