Sage Journals: Discover world-class research

Abstract

Laboratory activities in pharmaceutical R&D, including planning, experimental design, execution, data collection, processing, and reporting, rely on computer systems significantly. New computer systems are routinely implemented to increase the efficiency and quality of work in every aspect of R&D. One notable exception is the laboratory notebooks, which have been slow to “go electronic.” Only in the last few years, technology began to improve and more companies started to implement electronic laboratory notebooks (ELNs). Because the laboratory notebook is primary evidence for the ownership of intellectual property, concerns on whether electronic records can be used as evidence for invention is wide spread. Many early adopters of ELN technology took a hybrid approach: entering data electronically but printing and signing the printouts. However, there is very little basis for such concerns as regulatory agencies accept electronic records. What is important is that any records, electronic or paper, should be managed properly to ensure authenticity and trustworthiness. There are many benefits associated with ELNs, the most significant of which is the ability to search records electronically. The use of ELNs significantly increases productivity for scientists, peer reviewers, and supervisors in R&D. Time spent in searching for patent support data in discovery areas and on compliance and procedure-related tasks in regulated areas would be reduced, freeing time for other productive tasks. If integrated with other laboratory information systems, ELNs will also provide ability to directly drill down to hard data such as compound registration, biological assay, drug safety measurements, and pharmaceutical tests. Developing an information architecture that supports the complete spectrum of data will ensure proper interface between ELNs and other systems. The increased acceptance of ELN technology, combined with the maturing of R&D computing environment in the pharmaceutical industry, suggests that ELNs will have a wider acceptance in the near future. (JALA 2007;12:157–65)

Keywords

electronic laboratory notebooks ELN electronic records electronic signature data archiving pharmaceutical R&D

Paper Laboratory Notebook Management

A typical paper laboratory notebook management process includes the following:

•

Two types of laboratory notebooks may be used, bound notebooks or loose leaves. The preferred method is bound notebooks, because it has a fixed page sequence. Loose leaves are useful for capturing a large amount of printouts from instruments.

•

A notebook number is assigned to each new notebook before it is issued to a scientist.

•

The notebook owner and issue date are tracked.

•

Each page of the notebook is signed and witnessed shortly after the records are entered.

•

In the drug discovery area, the witness is knowledgeable of the data but is not a potential inventor.

•

For scientists who work in regulated areas, notebook pages are reviewed by a reviewer and sometimes approved by a supervisor. The reviewer and approver must be trained on the subject's matter and, at a minimum, capable of determining the suitability, accuracy, and effectiveness of the study.

•

After the notebooks are complete, they are archived. Most companies also create microfilms for archived notebooks.

Why Electronic Laboratory Notebooks?

Bound notebooks have been used as the primary records for scientific discovery in the history of science. This practice was successful, because most of the experimental observations were recorded manually. However, with the advent of automation and sophisticated instrumentation, in almost all branches of pharmaceutical R&D, paper laboratory notebooks became anachronism. Scientific data fall into two general categories, raw data and processed data. In an automated laboratory environment, raw data are generated by an instrument, recorded by data collection software, and processed by data analysis software. The processed data typically represent the conclusions of an experiment.

A common practice today is to handwrite the objectives and procedures in the paper laboratory notebook, perform the experiment, analyze the raw data using computer software, print the processed data, and paste it onto the notebook next to the conclusions. This process has several limitations that affect scientists’ productivity:

Data in the paper notebooks cannot be readily accessed, searched, and reanalyzed.

Cut and paste is tedious.

Paper record-keeping standards are hard to enforce.

Electronic laboratory notebooks (ELNs) are computer systems used for creating, storing, retrieving, and sharing fully electronic records in ways that meet all legal, regulatory, technical, and scientific requirements.¹ Specifically, ELNs serve the purpose of daily record keeping and as a result, provide “Analyst Diary” functionality for the laboratory scientist. Managing notebook data electronically would remove the limitations of bound notebooks.

Basic User Requirements

Because ELNs significantly affect the record-keeping workflow for an enterprise, it must meet the business requirements and have enough potential benefits to justify the cost. One must weigh the cost/benefit ratio carefully in different user areas, because the transition from paper to electronic will require significant effort. Below are some important aspects to be considered in order for the ELNs to be practical substitutes of paper-bound notebooks.

•

Tracking key information such as compound registration number, project name, author, experimental method, date, and other keywords.

•

Ability to map key metadata, such as project names, from discovery to development for data retrieval.

•

Linking all pages related to a compound for easy retrieval of chemical synthesis, stoichiometry, analysis, and spectra information.

•

Subject specific templates so that the users do not have to type in everything from scratch.

•

Flexibility in supporting unstructured, journal style information recording and allowing the scientists to revise the notes easily.

•

Robust and portable data entry device to enable data entry at the laboratory bench top.

•

Compact data entry device, ideally smaller than a laptop with some protection against solvent spills.

•

Handwriting recognition and content storage as both image and text.

•

Bidirectional data exchange integration with other data systems.

Will One Eln Suit All?

“Everybody wants their own smoothie” Jeffrey S. Wiseman, Ph. D., Locus Pharmaceuticals.

There is no general agreement what business processes ELNs provide functionality for. It must provide a secure, easy-to-use process to track and record the research observations, notes, comments, explanations, and other activities, while retaining the practical aspects of a paper lab notebook. The debate often focuses on two groups of users, discovery scientists and researchers, who work in the regulated environment during drug development phases.

Discovery and regulated laboratory environments are very similar and very different. A discovery laboratory typically conducts experiments that are in early research stages in drug development, such as target identification, lead optimization, and preclinical studies. Regulated environments include clinical studies, safety studies, and manufacturing. These areas are regulated by good clinical practice (GCP), good laboratory practice (GLP), and good manufacturing practice (GMP) to ensure patient safety. The regulated areas are collectively called GxP.

There are many common requirements for ELNs in non-GxP and GxP environments. To begin with, both areas require high degree of control to the electronic data managed in ELNs because data might need to be presented to the regulatory authorities or the patent bureaus. Title 21 CFR Part 11² of the U.S. Food and Drug Administration (FDA) is a section of the Code of Federal Regulations on electronic records and electronic signature. It explicitly states that electronic data and signature must be properly controlled to be considered equivalent to paper records. Although technically only GxP systems are subject to this regulation, non-GxP systems should at least meet the same standards if it is expected to stand up in a patent dispute.

Long-term data retention and accessibility is also a common requirement. The difference between non-GxP and GxP user requirements primarily lies in the particular functions and requires appropriate flexibility for each discipline. Intellectual property (IP) protection is the primary concern in a discovery lab. ELN data are required to be signed by the author and a knowledgeable witness within certain period of time after the data are entered. GxP environment calls for specific, well-defined experimental design and workflow with very few variations. Adhering to a standard operating procedure is of paramount importance in a GxP environment. Such procedures require that ELN data be signed by the author, reviewed for accuracy by a reviewer, and in some cases approved by a laboratory manager before the data can be released. Such procedures are put in place in a GxP environment to ensure that data are accurate and reliable so that a patient's safety is not compromised. In order for an ELN project to be successful, specific functional needs in GxP and non-GxP areas must be met.

The ELN business process roles can de divided into three typical categories (Table 1), each of which perform a set of tasks to ensure that the records entered in the ELNs are accurate, authentic, and trustworthy.

Table 1

Electronic laboratory notebooks discovery and GxP business process roles

	Discovery/research	GxP
Author	Author has rights to create, modify, and save experiments with all sources of authoritative data but may not witness his or her own experiments.	Author has rights to create, modify, and save experiments with all sources of authoritative data but may not review or approve his or her own experiments. Author is able to endorse his or her experiments for release to the approval process.
Witness/reviewer	Witness is the person who understands the Author's experiment and provides corroboration for the ownership of a study performed. Witness may not be potential inventor.	Reviewer is the person who will be responsible to review the Author's experiment. Same person cannot be Author and the reviewer of his or her own experiment. Reviewer is not able to edit or change the experiment of other Authors. Reviewer often also approves the record if a separate Approver is not designated.
Approver	Not applicable	Approver is the person who reviews and approves the experiment. In some cases, Approver is optional. If Approver is designated, experiment will not be approved until both Reviewer and Approver have signed off. Approver has the final authority in the approval process. Approver cannot endorse his or her own experiments.

Discovery/research

GxP

Author

Author has rights to create, modify, and save experiments with all sources of authoritative data but may not witness his or her own experiments.

Author has rights to create, modify, and save experiments with all sources of authoritative data but may not review or approve his or her own experiments. Author is able to endorse his or her experiments for release to the approval process.

Witness/reviewer

Witness is the person who understands the Author's experiment and provides corroboration for the ownership of a study performed. Witness may not be potential inventor.

Reviewer is the person who will be responsible to review the Author's experiment. Same person cannot be Author and the reviewer of his or her own experiment.

Reviewer is not able to edit or change the experiment of other Authors. Reviewer often also approves the record if a separate Approver is not designated.

Approver

Not applicable

Approver is the person who reviews and approves the experiment. In some cases, Approver is optional. If Approver is designated, experiment will not be approved until both Reviewer and Approver have signed off. Approver has the final authority in the approval process. Approver cannot endorse his or her own experiments.

The starting point of any ELN implementation is a good understanding of the existing laboratory processes. Generic ELN workflows are presented in Figures 1 and 2. While still providing the functionality of the paper version, ELN enhances the organizational efficiency by enabling seamless transfer of scientific information and preserving IP across research, development, and manufacturing. ELNs should ease the technology transfer process between user groups who work in different stages of the product development continuum. Significant gains in record-keeping efficiency and data quality may also be achieved by integrating ELNs with data acquisition systems. Without integration, data must be transcribed manually or imported with human intervention, a step that is likely to introduce errors and slow down data processing steps.

Figure 1.

Electronic laboratory notebook process of recording and reviewing an experiment. Tasks for the Author are highlighted in blue. Tasks for the Reviewer are highlighted in red.

Figure 2.

Electronic laboratory notebook approval process for GxP users. Tasks for the Author are highlighted in blue. Tasks for the Reviewer are highlighted in red. Tasks for the Approver are highlighted in green.

Eln as an Integral part of the Electronic Laboratory

One potential advantage of ELNs is the integration with other computerized systems. To maintain the established processes, a typical laboratory uses a number of specialized applications. The majority of those systems, for example, Chemical Registration Systems, labratory information management systems (LIMSs), ELNs, Chromatography Data Systems, Data and Document Repositories, that provide support for laboratory operations, are fragmented across the enterprise (Figure 3). Being most frequently used by the laboratory scientists, the ELN can be naturally transformed into a tool that integrates data from multiple systems. Therefore, the ELN may act as a portal, which may potentially bring in a variety of data mining applications.

Figure 3.

Common workflow and data exchange points of the Integrated Electronic Laboratory.

An important aspect of integration is data format standardization. In the chemical registration area, although extensible markup language (XML) based standards are being developed recently, the SDFile format from Elsevier, MDL remains the de facto industry standard.³ SDFile is an ideal format for transferring a large number of chemical structures between databases. Most vendors with chemistry-enabled ELN products, such as CambridgeSoft, support the SDFile format. Because SDFile format is in American Standard Code for Information Interchange (ASCII) text, custom integration can be developed with relative ease.

One of the complicated aspects of instrument-ELN integration is multiple data standards that negatively affect information exchanges between systems. In addition, pharmaceutical industry needs a common format for long-term data repositories, which are vendor independent in processing or reviewing of analytical instrument data. Analytical Information Markup Language, AnIML, is being developed by ASTM International Subcommittee E13.15 for the storage of analytical instrument data.⁴ AnIML can also be used to record data from everyday experimental workflows in a laboratory environment.⁵ However, AnIML is yet to be adopted by most ELN vendors.

At the database level, most ELN products support relational databases, such as Oracle and structured query language (SQL) Server. As a result, integration between ELNs and other systems can be achieved using standard SQL or data access tools available in most software development environments.

Benefits of Elns

ELNs are not just simple replacements for paper notebooks. They should support scientific workflow such as routing a page for witnessing or approval, an added advantage compared to paper notebooks. ELNs should permit the entry and relation of unstructured data as well as structured data. ELNs should be secure and meet regulatory and legal requirements. ELNs should have an authenticated electronic signature for an author and witness/approver. Lastly, records in an ELN should be searchable and ELNs should provide collaborative workspaces, an advantage that cannot be easily achieved with paper notebooks.

There are many benefits in using ELN as a tool to record scientific information, including ease of access to research data, increased quality in record keeping, better compliance to IP and regulatory policy, and operational consistency. ELNs also enable better knowledge management and stimulate cultural changes for more open data access and information sharing among scientists. Therefore, the major benefits of ELNs can be grouped as general benefits, benefits for the scientists, benefits for the witness or manager, and benefits for records management.

Key Value Metrics

The benefits of ELNs can be measured and eventually calculated for return on investment (ROI). Without presenting here the complex financial computations necessary for ROI, let us note basic merits of ELNs in bringing financial and organizational gains (Table 2).

Table 2

Measurable benefits of electronic laboratory notebooks

	Reduced turnaround time	Increased productivity	Improved data quality	Cost reduction
Cloning experiment design			–	–
Experiment review and sign-off		–	–	–
Reduced manual data transcription				–
Ability to search data and avoid repetitions of the experiments performed already
Eliminating the need to archive paper notebooks	–	–	–
Integration with instrument and other applications				–
Secure access to data	–	–	–

General Benefits

Electronic Distribution of Scientific Information. ELNs eliminate retransfer of data and provide global access to information across lines and sites. Increased information sharing avoids unnecessary repeats of previously failed experiments and optimizes equipment and resource utilization. ELNs facilitate the propagation of best practices in science and improve experiment outcomes.

Searching and Trending Capabilities. One of the most significant features of ELN is its searching capabilities that are unmatched by a paper-based system. Data can be effortlessly retrieved using multiple criteria, such as compound ID, dates, and names. The time required for compiling support data, such as synthetic procedures and analytical spectra for patent filing, can be significantly reduced. It also reduces the time needed to gather data required for an audit, FDA query, or an inspection.

Most ELNs that are currently available provide trending tools to compile the special reports where the number of experiments designed, started, and completed by a scientist, for a defined period, can be summarized. They can also search and present some statistics such as number of studies reviewed, experiments pending reviewed, and average review time.

Benefits for the Scientist

ELNs would save valuable time by eliminating a number of repetitive tasks.

Eliminating Extra Steps in Notebook Handling. In a GMP regulated environment, the use of paper-bound notebooks requires scientists to follow through the same procedural steps, repetitively. These monotonous procedures can be eliminated in the electronic setting. Steps such as obtaining notebooks or collecting “wet” signatures would not be needed. Time spent in searching for definitions manually could be reduced due to ELNs’ linked acronym database, which can provide the definition automatically.

Simplified Record Keeping. With ELNs, a majority of repetitive steps can be avoided while maintaining a compliant process. For example, there is no need to fill in the “continued from” and “continued to” fields that are located on each page of a paper notebook. The “cut and paste” exercises also will be eradicated. Writing a notebook's page number on attachments, signing, and dating every notebook page of an experiment would be eliminated. Therefore, the task of initialing and dating across the edge of attachments would be removed.

Although ELNs eliminate those steps, it should be noted that ELNs may be configured to capture additional data (such as experimental conditions) that scientists neglect to record in paper notebooks. Although requiring extra data entry may reduce the efficiency gain when the experiment is performed, it helps enhance data quality, a good practice that is hard to enforce in a paper-based system. As a result, the overall R&D efficiency would most likely be increased.

Use of Predesigned Templates. Predesigned templates are commonly used in association with paper notebooks for recording repetitive processes, common reagents and chemical reactions, HPLC mobile phases, and dissolving media. It can be done in a similar fashion electronically using drop-down lists. In some cases, users can also create templates on demand to suit their individual needs.

Importing Capabilities. Many ELNs provide a wide range of capabilities to import pictures (directly, via an interface to another application, or indirectly, as a scanned object) in a variety of formats, which can be easily resized when necessary. Import options that work with multiple files at once make the importing process more efficient. Equipment readouts, such as balance and pH meter measurements, may be automatically recorded in the system.

Other Benefits. ELNs detect typographical errors before an e-signature is entered, saving the time of initialing, dating, and correcting minor typos. They also offer unmatched convenience for objects being inserted anywhere in the experiment at any time. In this case, audit trail and reapproval are required if the experiment has been previously approved. The use of a paper notebook requires much more complicated procedures when another attachment is needed at a later date.

Benefits for Witness/Approver and Manager

Using ELNs instead of traditional paper notebooks has the following advantages when witnessing and reviewing completed work.

Routing for Witnessing or Approval. Authors can choose/define the review flow before starting an experiment and ELNs can be configured to distribute the e-messages to alert witness or approvers of tasks being completed. It reduces the turnaround time needed for the witness or approver to meet the deadlines, by keeping track of what needs to be witnessed or approved.

Reviewing an Experiment. There is no need to waste the witness/approver time required when verifying that the page number from the title page of an experiment is on each subsequent page or checking that the “continued from” and “continued to” number on each page of an experiment is correct or filled in. Many unproductive tasks associated with good documentation practices will not be necessary.

Benefits for Records Management

Tracking Notebooks. There is no need for a separate system for tracking the notebooks and whom the notebooks are assigned to. If only electronic records are maintained, the use of ELNs will eliminate the time and money needed for producing and maintaining the paper and microfilmed records.

Indexing Notebooks. There will be no need for an additional indexing system to transcribe the keywords from the paper notebooks to a database, because all information is captured electronically.

Drawbacks of Elns

As with other technology, ELNs come with drawbacks that must be managed properly. The flip side of information sharing is the potential misunderstanding and incorrect conclusions drawn by scientists who are not intimately familiar with the data. Time must be allowed for cultural changes, related to treating notebook data as corporate asset rather than personal property, to take place. Improved data annotation is essential to minimize the possibility of misinterpretation, and encourage communication between users when the meaning of data needs to be clarified.

Computers are fragile and less portable than bound notebooks, which can be carried around the lab. Although portable devices such as tablet PCs are now available, they are more expensive to maintain and may not be compatible with all commercially available software.

The efficiency gain in data entry using ELNs may not be as large as one would expect. More time may be needed to capture data in the ELNs than by handwriting it in a bound notebook, because keyboard data entry is more cumbersome than handwriting in a laboratory environment. Although optical character recognition technology has become widely available in recent years, the accuracy rate may not be high enough for a laboratory environment, where one does not have much time to correct software errors while performing an experiment. Drawing a chemical structure is another problem. Current drawing tools require the use of a tool kit of structural elements, such as atoms and bonds, an experience that can be improved dramatically if hand-drawn chemical structures can be interpreted into a computer representation. Such a tool has been developed by Brown University⁶ and it would be interesting to see when such technology will be commercialized.

Because electronic format and archiving media technology change rapidly, maintaining electronic records in a readable format for a long period of time is a challenge. The generally long lifetime of a patent, requires ELN data be stored for decades. To preserve data for a long term, care must be taken to ensure that proper standard archiving formats such as PDF/A⁷ is used for data archiving. In the event of standard formats changing, one must be prepared to convert data to the new format.

The cost of implementing a new record-keeping process in the user groups could be significant. In addition to software and hardware cost, costs associated with user workflow changes cannot be underestimated. Implementing and maintaining integration between ELNs and other computer systems can also be of high cost. Integration adds dependencies between systems. If one system is upgraded or modified, proper tests must be performed to ensure that other systems are not negatively affected. Sufficient resources should be allocated to validate the system, to ensure that both user and regulatory requirements are met.

Finally, backup and recovery processes must be established to ensure high availability of ELNs and timely recovery of the system in the event of a disaster. Electronic security is another problem that must be addressed. Procedures must be enforced to prevent password sharing, a common problem in some user areas. Proper infrastructure must be put in place to reduce exposure to computer virus and malicious hacking.

Legal and Regulatory Considerations

There are many requirements for implementing a successful ELN. Meeting legal requirements is the most critical, because laboratory notebooks are the primary records of scientific discovery. Are ELN records legally acceptable as evidence of invention? Regulatory agencies in many jurisdictions accept electronic records. The U.S. Patent and Trademark Office published a notice stating that electronic records are admissible to the same extent as written records if they are created and stored in a way that engenders trust in the records.⁸ Like paper records, electronic records, if kept properly, are acceptable as evidence of invention.

However, to date, there is no case law for successful patent defense based solely on electronic storage of data and electronic signature, due to the extreme few occurrences of patent disputes. Because ELN is a new way of record keeping for scientific inventions, the industry has adopted higher standards than what is required by law. In patent interference, whether a piece of information is admitted as evidence of invention by the Court is governed by the Federal Rules of Evidence⁹ with two major requirements: business records objection and foundation objections. Business records objections accept the records presented by the employer, even if the inventor is no longer employed by the company. Establishing a comprehensive foundation for electronic records to be admitted as evidence of invention is more challenging.¹⁰ A high standard must be set for record keeping, whether on paper or electronically.

Meeting the requirements of regulatory agencies (the FDA in the United States) is critical for ELNs. The FDA has extensive regulations and guidance to electronic records² and electronic submissions.¹¹ It is clear that electronic records are acceptable as proof for the efficacy and safety of a drug if the computer system that manages the data is placed under proper control. Proper testing and documentation must be available to demonstrate that the system meets the user-required functions. An audit trail must be available to track data updates. The validation of computer systems is a rigorous process of testing and documenting that a computer system meets user and regulatory requirements. If the ELN system is used in the GxP area, it must be validated. For non-GxP areas, validation is a good approach to demonstrate that the electronic records are trustworthy and meet the requirements for patent purposes. Given the high risk of ELNs to business, maintaining a validated ELN system is the best interest of discovery scientists.

The ELN should meet the following legal and regulatory requirements:

•

Ensuring electronic records’ relevance, accuracy, authenticity, and trustworthiness.

•

Establishing procedures and enforcing written guidelines to ensure the integrity of the electronic records.

•

Enforcing timely electronic signatures by the author, witness, reviewer, and approvers.

•

Keeping an audit trail for the creation, modification, reviewing, and approval of records.

•

Locking records after they are signed so that they cannot be altered.

•

Keeping amendments to an ELN record as an indelible part of the record with a permanent link between the record and its amendments.

•

Controlling data access with username and password combination.

Industry Benchmarking

Comprehensive surveys of ELNs are available.^{12, 13} In this article, we will provide only some highlights based on these surveys and other published literature. Commercial ELN products became available in the early 1990s, but the market did not take off until after the year 2000. Early ELN technology did not have the flexibility of handling discipline specific information. The new generation of ELNs that have appeared in recent years are equipped with better technology that addresses customer concerns on security, legal, and regulatory requirements, as well as the different functional needs from diverse user areas. Increasing demand in the pharmaceutical industry for ELNs has led to the number of vendors to increase from a few in the 1990s to over 30 currently.¹⁴ However, there are no dominant players among the ELN suppliers currently, although some (CambridgeSoft, Symyx, VelQuest, Waters) are considered the first tier for their wider installation base and larger company size.¹² Collectively, the ELN market penetration is still small and supplier consolidation is expected to occur in the near future.

Pharmaceutical companies that have implemented ELNs include J&J, Merck, GSK, Organon, Berlex, Array Biopharma, Vertex, AstraZeneca, Eli Lilly, BMS, and Schering Plough. Many took a hybrid approach to the new technology and used ELNs for data entry followed by printouts.¹² After records are entered, signed, and witnessed electronically, they are printed out for wet signature and the hard copies are archived. GSK¹² and Array Biopharma¹⁵ are among the companies that keep electronic records only. Recent research indicates that there are a higher percentage of electronic-only implementations among new ELN customers.¹⁴

Across pharmaceutical R&D, information entered in ELN varies greatly. Discovery scientists require a flexible front end that can handle unstructured data and text entry, plus the ability to recognize chemical structures and in vitro and in vivo study images. Development and manufacturing scientists prefer a more structured and organized workflow. To maximize the benefit of ELNs to the different scientists, there is an increasing trend of implementing multiple ELN systems with separate focuses on discovery and development areas and a shared archiving system. After the ELN records are signed and witnessed/approved in separate ELN systems, they are converted to PDF format and archived in acommon enterprise document management systems (such as Documentum).

Integration between ELNs and data systems is critical. Data systems on the top of the integration list for most customers are LIMS, scientific data management systems, image management systems, electronic document management systems, assay management systems, compound registration systems, and data analysis tools.

Recommendations

The ELN technology has progressed significantly recently and is becoming more mature as it evolves in the field of record keeping in pharmaceutical R&D. Benefits of ELNs clearly outweigh the potential drawbacks and risks associated with the changes in record-keeping procedures. Because of its fundamental impact on IP protection and the record-keeping workflow, a comprehensive analysis of requirements should be performed. Ideally, one should start with a small-scale pilot project demonstrating the feasibility of the system in meeting these requirements. Full implementation can follow if the pilot project is successful.

References

Collaborative Electronic Notebook Systems Consortium, www.censa.org (accessed Nov 2006).

Title 21 Code of Federal Regulations (21 CFR Part 11), Electronic Records; Electronic Signature, US FDA.

CTFile Formats, Elsevier MDL, 2005.

Analytical Information Markup Language (AnIML) Workshop, Pittcon, 2006.

Schaefer

; Poetz

; Kramer

Documenting laboratory workflows using the Analytical Information Markup Language. J. Assoc. Lab. Autom. 2004, 9(6), 375–381.

Tenneson

Dana

ChemPad: A Pedagogical Tool for Exploring Handwritten Organic Molecules; Brown University. http://research.brown.edu/btp/technologies.php?id=1113352312 (accessed Nov 2006).

Document Management-Electronic Document File Format for Long-Term Preservation. International Organization for Standardization; 2005; ISO 19005-1.

US Patent and Trademark Office; Official Gazette number 1208 OG 35; March 1998.

Federal Rules of Evidence: The Committee of Judiciary. US House of Representatives; 2004.

10.

Lucas

John

Circling the Electronic Wagons: Protecting Your Intellectual Property in an Age of Automation. IQPC Laboratory Informatics Conference; Feb 2006.

11.

Guidance for Industry Providing Regulatory Submissions in Electronic Format-General Considerations, US FDA.

12.

Electronic Laboratory Notebooks-A Foundation for Scientific Knowledge Management, Edition II. Atrium Research & Consulting: 2005.

13.

Management Best Practices: Update On Electronic Laboratory Notebooks, Vol. 9 Outsell Inc.: 2006.

14.

Elliott

Michael H.

Current State of the Market for Electronic Laboratory Notebooks. IQPC Laboratory Informatics Conference; July 2006.

15.

Rizzi

James

, Biopharma

Array

Expanding the ELN beyond Chemistry: Trials & Tribulations. IQPC Laboratory Informatics Conference; July 2006; John Lucas, Information Circling the Electronic Wagons: Protecting Your Intellectual Property in an Age of Automation. IQPC Laboratory Informatics Conference; Feb 2006.