Abstract
OBJECTIVES
Despite the wide use of medications in clinical practice, graduating medical students often feel unprepared for the task of prescribing upon starting residency. With recent educational initiatives aiming to transform learning modalities, we sought to pilot an interactive textbook on basic pharmacology principles at our institution as a supplement to first-year lectures and assess its subjective impact on students’ knowledge of content as well as confidence to apply material in the real world through pre- and post-intervention surveys.
METHODS
First-year medical students were invited to complete non-validated, voluntary, anonymous, emailed, online surveys consisting of Likert scale and free-text response questions. Our investigation served as a pilot test for future iterations of this research.
RESULTS
Response rates for the pre- and post-intervention surveys were 73/145 (50%) and 38/145 (26%), respectively, with the post-intervention survey further reduced to 13 individuals who indicated use of the interactive textbook. Questions regarding interactive textbook chapters that overlapped with course content were excluded from data analysis due to an inability to separate learning gains from lectures versus the interactive textbook. Post-intervention survey responses all showed significant changes in mean Likert scale scores on student-perceived knowledge and confidence to apply material with P < .001. Free-text response questions revealed limited exposure to the field of pharmacy and interactions with pharmacists prior to medical school.
CONCLUSION
Our pilot study on the initial use of an interactive textbook titled The Medical Student Guide to Pharmacy presented us with valuable insight into providing first-year medical students with a clinically oriented supplemental resource within coursework on basic pharmacology. Challenges for the future include better integrating the interactive textbook into class lectures to facilitate increased use by students as well as developing more targeted, validated assessments of the impact it has on students’ learning.
Introduction
A large portion of a physician's work is the appropriate prescribing of medications to treat diseases and mitigate various conditions. In the United States, approximately 49% of all people used a prescription drug in the last 30 days from 2015 to 2018. 1 This number also reflects an increase in prescription drug use in recent years; when compared to older data from 1988 to 1994, only 38% of people had taken at least 1 prescribed therapy in that same timeframe. 1 Despite the wide use of medications in clinical practice, graduating medical students often feel unprepared for the task of prescribing upon starting residency.2-8 This issue has been acknowledged for decades by both the Association of American Medical Colleges (AAMC) and the World Health Organization (WHO), each of whom has produced multiple guidance documents for teaching reasonable and prudent prescribing practices to medical students.9,10
Learning pharmacology is akin to mastering a second language, with specialized vocabulary, key principles, and abundant minutiae that even experts may occasionally miss. It is niche and almost inaccessible to general college undergraduates unless they choose to study pharmacy or have exposure to the pharmaceutical industry, leaving entering medical students with practically zero familiarity until their introductory lectures in the first year of school.11,12 Thus, the onus of teaching falls on medical schools, which must balance board-type material with realistic clinical pearls essential to the practice of medicine.13-17
Classical teaching methods rely on mass lectures that disseminate general information to a large group with sparse interactions between the educator and learners. 18 Recent trends in scholastic pedagogy have looked to personalize students’ experiences with small group sessions, team-based learning, and interactive educational modules. 19 Topics such as pharmacology, histology, biostatistics, anatomy, and clinical decision-making are just a few examples where material has been successfully taught using innovative online platforms.20-25 Moreover, students often report feeling more engaged with their studies, resulting in either higher or noninferior scores on assessments when compared to methods like passive lectures.20-25 It is also important to note that by providing academic materials in various formats, this supports students who may have diverse learning styles.26,27
At The Warren Alpert Medical School of Brown University, an introduction to general pharmacology block is taught toward the end of the first year, first semester courses. Key components of this block include an overview of pharmacokinetics, pharmacodynamics, the Food and Drug Administration drug development process, medication dosing calculations, and toxicology. This material is intended to prepare students for the United States Medical Licensing Examination (USMLE) Step 1 yet is intentionally broader in scope to be applicable to future practice as a physician. From our internal course evaluations as well as general reports in the literature, students have reported pharmacology as being one of the harder subjects to learn, making this an opportune area to incorporate additional learning modalities.14-17
The goal of this project was to pilot an interactive textbook on basic pharmacology principles at our institution as a supplement to first-year lectures and assess its subjective impact on students’ knowledge of content as well as confidence to apply material in the real world through anonymous pre- and post-intervention surveys.
Methods
Our interactive textbook, The Medical Student Guide to Pharmacy, was created using Top Hat online software and is available for internal use by students at The Warren Alpert Medical School. This resource can be accessed through a computer or tablet connected to the internet. It consists of 7 chapters (Table 1) that incorporate a few elements of active learning such as embedded practice questions, links to outside resources, in-text definitions that appear when a cursor is brought to a blue-colored term, a sidebar navigation system to jump to various parts, as well as an annotation feature where students can highlight text in several colors, add sticky notes, bookmark their reading progress, and type in an electronic notebook paired with the interactive textbook (Figure 1A–D). The main non-interactive feature of note in this book is the clinical correlations that are dispersed within the chapters to provide real-world context; these examples are often paired with a color image to break up dense text or provide the reader with useful learning points in a more accessible manner. For instance, within Chapter 5: Special Populations, under the section on hepatic dysfunction, after a discussion on the impact of cirrhosis on drug dosing, a clinical correlation on dosing prodrugs versus active moieties is presented using cyclophosphamide and doxorubicin as representative medications from each class, respectively.

Screenshots from The Medical Student Guide to Pharmacy. (A) How a cursor placed above select blue-colored terms reveals in-text citations. A clinical correlation on the difference between serum and plasma is also featured in this picture. (B) Interactive question, this one having 1 correct answer. Other question types include selecting multiple right choices or dragging and dropping matching statements. (C) How students can highlight text in 3 colors and add sticky notes for in-text annotations. This information is concurrently saved in a “notebook” in the sidebar. A clinical correlation on how the structure of vancomycin impacts its pharmacokinetics is also featured in this picture. (D) The option for students to read an outside article on race corrections in renal function calculators that relates to a discussion on the Cockcroft-Gault equation. Software copright 2023 Tophatmonocle Corp. Image credits: Components of blood image credit: Wikimedia (public domain). Original figure from https://commons.wikimedia.org/wiki/File:Components_of_blood.png; Chemical structure of vancomycin image credit: Wikimedia (Public Domain). Original figure from https://commons.wikimedia.org/wiki/File:Vancomycin.svg; Arnold Schwarzenegger 1974 image credit: Wikimedia (public domain). Original figure from https://commons.wikimedia.org/wiki/File:Arnold_Schwarzenegger_1974.jpg.
Interactive textbook chapters and content outlines.
Additional information in the form of practice questions, clinical correlations, and outside resources is included within these categories and subcategories. This is not an exhaustive list of all content presented in the interactive textbook. Abbreviations: IV, intravenous; PO, per os / by mouth.
Chapters 2, 3, and 4 are directly reflective of lecture material taught at our medical school, whereas the other chapters contain useful information, but are not necessarily included in required coursework. These chapters were developed in part by viewing standards from the Accreditation Council for Pharmacy Education for Doctor of Pharmacy (PharmD) programs as well as surveying practicing pharmacists at Rhode Island Hospital and The Miriam Hospital, two of The Warren Alpert Medical School's principal teaching hospitals, for ideas on important topics to consider. 28 Other references found within the textbook can be viewed in Supplement 1. We also surveyed third- and fourth-year medical students at our institution for suggestions based on their experiences of navigating clerkships and encountering the clinical side of drug knowledge. Our intention of including a broad base of information with the interactive textbook is to encourage later review of students’ progression in the medical program, an example being the “Special Populations” chapter which discusses pediatric pharmacy principles among other groups of interest. Such a reference may be utilized while on a pediatric clerkship to learn about medication considerations in children.
This investigation sought to pilot our interactive textbook as a supplementary resource for use within a 4-week didactic block that includes several lectures on basic pharmacology in the first-year curriculum and assess its subjective impact on students’ knowledge of content as well as confidence to apply material in the real world through anonymous pre- and post-intervention surveys. The only inclusion criterion was active enrollment as a first-year medical student at The Warren Alpert Medical School; there were no prespecified exclusions for this voluntary study. We did not perform sample size calculations but simply invited all students to participate in completing the surveys; with a class size of 145 students, our goal was to reach a 60% response rate or approximately 87 responses. To encourage participation, raffles for one of two $10 gift cards to Amazon were conducted for both the pre- and post-intervention questionnaires.
Prior to the introductory pharmacology block and access to the interactive textbook, first-year students were invited to complete a voluntary, anonymous, emailed, online survey consisting of Likert scale and free-text response questions to assess the impact of the interactive textbook on their education; this survey was then repeated after the final exam (Supplement 2). Our study was localized to our medical school in Providence, RI and took place from December 2022 to January 2023, a total duration of approximately 4 weeks. These surveys were not validated, and our investigation served as a pilot test for future iterations of this research. Before initiating the survey, students reviewed the consent information; completion of the survey served as their documented agreement to participate in the study.
The first set of 7 questions asked students to rate their knowledge on topics covered in each of the interactive textbook chapters, then the second set of 7 questions inquired about confidence in applying or utilizing such knowledge from each chapter in the real world. Likert scales ranged from 1 (no knowledge or no confidence) to 5 (full knowledge or full confidence). For illustration, the questions related to Chapter 1: What is Pharmacy? initially asks students to “Rate your knowledge on the field of pharmacy and the training path for pharmacists” in the first block of questions then to “Rate your confidence on educating others about the field of pharmacy and the training path for pharmacists” in the second block. Afterwards, several free-response questions on previous experiences with pharmacists or the field of pharmacy as well as interest in interprofessional shadowing were included in the pre-intervention survey. The post-intervention survey had similar free-response questions to assess for changes, but also asked students about their use of the interactive textbook.
Statistical analysis
All analyses were performed using Stata/SE 17.0, StataCorp LLC. For questions that utilized a 5-point Likert scale, means and standard deviations were generated to note overall trends. Free-response questions with similar or identical answers were grouped to form proportions. Nonparametric analyses to determine differences from the pre- to post-intervention surveys were performed using the Wilcoxon rank-sum (Mann-Whitney U) test. Due to the anonymity of the surveys to allow participants to feel comfortable sharing their honest thoughts, we did not match pre- and post-intervention survey responses but pooled the collected data. If students did not report use of the interactive textbook on the post-intervention survey, their responses to Likert scale questions were excluded from analyses. Statistical significance was set at P < .05. All authors attest to the accuracy of the data and fidelity of statistical analyses. When developing this manuscript, we utilized the Standards for QUality Improvement Reporting Excellence in Education (SQUIRE-EDU) checklist to ensure proper reporting of our findings (Supplement 3). 29 This study was declared as curriculum review, not requiring formal informed consent of participants, and consequently exempt from the Brown University Institutional Review Board. This determination has been made according to the definition of research provided in Title 45 CFR Part 46.102(l).
Results
Response rates for the pre- and post-intervention surveys were 73/145 (50%) and 38/145 (26%), respectively, from the surveyed class of first-year medical students. Each survey had the same general structure, with two parts consisting of Likert scale questions regarding self-rated knowledge and confidence on topics covered in each of the interactive textbook's chapters, plus a final section with free-response questions (Table 2). For the pre-intervention questionnaire, these open-ended questions asked about their understanding of pharmacists’ role in healthcare, any past experiences in the field of pharmacy, and whether they would be interested in shadowing a pharmacist at Rhode Island Hospital. Likewise, the post-intervention survey asked the same question regarding knowledge of pharmacists and whether there was any interest in shadowing. Additional questions followed on impact and usage of the textbook, as well as any suggestions for improvement.
Select pre- and post-intervention Likert scale survey results.
Survey questions utilized a Likert scale ranging from 1 = no knowledge or no confidence to 5 = full knowledge or full confidence for each listed statement. A score of 3 indicates neutrality. Questions regarding interactive textbook chapters that overlapped with course content were excluded from the analysis. For the post-intervention survey, only respondents who reported use of the interactive textbook were included for analysis.
Reported using the 2-sample Wilcoxon rank-sum (Mann-Whitney U) test.
The surveys were structured to require completion of all Likert scale questions in the first 2 sections but made the free-text responses optional. Differences in response rates for questions occurred due to initial parts of the survey being completed, but not the open-ended questions. Accordingly, the pre-intervention survey includes 73 numerical and 71 written responses, while the post-intervention survey consists of 38 numerical and 29 written responses. Questions regarding interactive textbook chapters that overlapped with course content (Chapters 2, 3, and 4) were excluded from data analysis due to an inability to separate learning gains from lectures versus the interactive textbook. Additionally, respondents had to indicate use of the interactive textbook for Likert scale responses to be included in the post-intervention survey analyses, further reducing numerical responses from 38 to 13.
When viewing results from the pre-intervention surveys under the section of perceived knowledge, all 4 questions had average scores in the range between 1 and 2, with 1 as “no knowledge,” 3 as “neutral,” and 5 as “full knowledge.” The greatest mean Likert scale score of 1.74 was found under the question on the field of pharmacy and the training path for pharmacists (correlates to material in Chapter 1), whereas the lowest mean Likert scale score was 1.56 for principles of medication use in special patient populations (Chapter 5). Pre-intervention means for perceived confidence to apply knowledge of various pharmacy-related topics were similarly low, with values clustered between 1.30 and 1.48 for all questions, suggesting a preponderance of “no confidence” from respondents on these measures.
Of the 29 students who completed the post-intervention survey, only 13 individuals (45%) indicated use of the interactive textbook during the introductory pharmacology block. Among this group, post-intervention survey responses all showed significant changes in mean Likert scale scores for all questions using the Wilcoxon rank-sum (Mann-Whitney U) test, with P < .001. Average Likert scale ratings were clustered in the range of 3 to 3.38 for self-rated knowledge. Similarly, self-rated confidence had a small range between 3 and 3.54, both of which suggest an aggregation of values in the “neutral” to “some” knowledge or confidence ranking.
Several trends were observed in the free-text response questions, including limited student experiences with pharmacists or the field of pharmacy prior to medical school. Of the 71 survey respondents for this question, only seven noted such previous encounters, approximately 10% of the group. Likewise, students were asked to share their understanding of the role pharmacists have in healthcare. Responses varied, with some citing community pharmacists as examples whereas others had a much broader understanding. This latter group mentioned pharmacists’ role not only in physical settings like a hospital or a retail chain pharmacy but also discussed their actual duties like ensuring the safe and appropriate use of medications. Similar findings were noted across the pre- and post-intervention surveys. Finally, between the pre-intervention and post-intervention surveys, indicating “yes” for interest in shadowing a pharmacist at Rhode Island Hospital decreased from approximately 46% to 28% (Table 3).
First-year medical students reported interest in shadowing a pharmacist at Rhode Island Hospital.
Percentages may approximate 100% due to rounding.
Selected positive and negative feedback on interactive textbook.
Bracketed text is intended to provide clarity in select comments.
Discussion
This study represents a pilot program of an in-house created interactive textbook on basic pharmacology principles for use at The Warren Alpert Medical School of Brown University as a supplement to course materials during a 4-week didactic block featuring several lectures on the basics of pharmacology for first-year medical students. Our use of non-validated, anonymous pre- and post-intervention surveys was to begin to assess the possible impact of The Medical Student Guide to Pharmacy on students, as well as determine areas of improvement for the future. A major limitation in this investigation was the limited uptake of the interactive textbook by students; although we found statistically significant changes in self-reported knowledge and confidence to apply material in the real world, our sample size of 13 greatly limits the strength of our findings, especially due to our lack of sample size calculations for our set significance level. Time, interest in the subjects presented in the interactive textbook, and personal learning styles are likely major contributors as to why most students did not incorporate The Medical Student Guide to Pharmacy into their studies for the didactic block. Furthermore, despite a small gift card raffle to encourage completion of the surveys, we did not reach our set response rate of at least 60% for both the pre- and post-intervention questionnaires. Given the authors’ involvement in creating the interactive textbook, we acknowledge the presence of authorship bias in this manuscript when viewing the impact of our creation.
Moreover, due to an inability to separate learning gains from lectures versus the textbook, we could only analyze survey questions that did not overlap with course material. Questions that assessed the impact of Chapters 1, 5, 6, and 7 were less likely to be confounded due to the content not being formally taught and only presented to students through the interactive textbook. Despite our observed weaknesses, it is notable that all analyzed questions showed increases in mean Likert scale scores between the pre- and post-intervention survey administrations, a period of 4 weeks. These changes were small, essentially increasing from a low level of knowledge or confidence to a neutral or slightly increased amount, which is appropriate given students’ relative inexperience with topics like medication dosing in renal failure or navigating professional drug information resources.
Viewing comments about The Medical Student Guide to Pharmacy proved most helpful in determining attitudes regarding this resource. Table 4 shows a sample of various positive and negative quotes obtained from free-text questions in the post-intervention survey. In general, positive comments centered on the utility of the book as a supplement for lecture material, to allow students to focus on interests and take an active role in their education. For instance, those who wish to learn more about pediatrics could easily read through that section in Chapter 5: Special Populations as a primer for future experiences in that specialty. Moreover, early exposure to topics that will appear in later exams, clerkships, and general practice as a future physician only serves to benefit students.
Common themes in the negative comments included some difficulties with accessing the interactive textbook via a link on the course Canvas page as well as technological issues with the Top Hat interface. Some students also felt overwhelmed by the amount of material present and not having clear guidance on how to best incorporate it into their regular studying for the block exam. In response, with future uses of the interactive textbook, we plan on holding an in-class demonstration of where to find it through Canvas, how to navigate the resource itself and utilize its features, as well as how to best incorporate chapters into studying for the end-of-course exam. With regards to a comment on connecting the book chapters to class lectures, we plan on having specific recommendations in the slides for what sections of the interactive textbook would be most helpful to review. For instance, in the lecture on pharmacokinetics, Chapter 2, Section 2.5: Excretion would assist in providing more information on drug half-life as well as practice questions for self-assessment of knowledge and building confidence for test day. We also plan on reviewing all embedded practice questions for level of difficulty and helpfulness, as some commenters stated a few questions seemed too in-depth or were not as clinically oriented as others. To avoid increasing the risks of authorship bias that are likely already present, a faculty member who did not create the interactive textbook can assist in examining these revisions. Lastly, if this interactive textbook is to be required for students to view, we plan to activate previously unused features on Top Hat to track online “attendance” and verify access.
With all lectures at our medical school recorded and often optional to attend in-person, students have immense power in choosing a learning format that works best for them. Modern educational pedagogy is transitioning from static methods like professor-led lectures to more dynamic approaches like online modules completed at home with a paired problem-solving workshop the next day.30-33 Such blended approaches have shown to improve learning and better engage students. 24 The biggest challenge with incorporating supplemental resources into curricula is considering learners’ often limited time. In the post-intervention survey, a near-unanimous reason for not using the interactive pharmacy textbook was lack of time to view it. Additionally, several respondents commented that they could not spare precious hours learning material that was not going to be on the course exam, even if it was presented in an appealing manner. At this stage in their education, first-year medical students likely do not find clinical pearls and real-world applications as useful as basic science facts that are needed to pass pre-clerkship exams and the USMLE Step 1. Thus, while introducing this textbook early in their career may be helpful to some, its ultimate utility may come later when these students are on rotations and are immersed in clinical medicine. By then, The Medical Student Guide to Pharmacy could serve as a reference book, akin to titles like First Aid or other quick guides.
With approximately 90% of students reporting no interactions with pharmacists or the field of pharmacy prior to medical school, the decreased interest in shadowing pharmacists in a hospital setting by the end of the 4-week block was a surprising result. Several explanations are likely involved, ranging from limited free time to a lessened interest in interprofessional education if students felt they knew about pharmacy and pharmacists well enough through the course lectures and the interactive textbook. Another issue lies with the much smaller response rate on the post-intervention survey, which could have led to a misrepresentation in student consensus on this question. Regardless, upcoming directions with our interactive textbook may incorporate some element of hands-on experience if students desire it. A formal shadowing program between medical students and pharmacists could provide the same real-world advice for learners as seen in Chapter 7 but in the most active way possible. The curriculum at The Warren Alpert Medical School includes interprofessional education workshops with pharmacy, nursing, and physical therapy students from The University of Rhode Island as well as nursing and social work students from Rhode Island College, so the idea of learning from different healthcare professionals is not foreign to our students, though it may not be appreciated as much during the first year of medical school, which is largely isolated from the clinical domain.
Conclusions
Our pilot study on the initial use of an interactive textbook titled The Medical Student Guide to Pharmacy presented us with valuable insight into providing first-year medical students with a clinically oriented supplemental resource within coursework on basic pharmacology. Although limited by small samples and self-reported data from students, we believe further investigation into the use of active learning modalities like our interactive textbook can positively impact curriculums based on our results. Challenges for the future include better integrating the interactive textbook into class lectures to facilitate increased use by students, as well as developing more targeted, validated assessments of the impact it has on students’ learning. At this point, our research is only beginning, but we hope to learn from this experience to further refine the interactive textbook, especially incorporating elements of interprofessional education, to enhance the pharmacology curriculum at The Warren Alpert Medical School of Brown University.
Supplemental Material
sj-docx-1-mde-10.1177_23821205231225589 - Supplemental material for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations
Supplemental material, sj-docx-1-mde-10.1177_23821205231225589 for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations by Kendra L. Walsh and Sarita S. Warrier in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-2-mde-10.1177_23821205231225589 - Supplemental material for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations
Supplemental material, sj-docx-2-mde-10.1177_23821205231225589 for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations by Kendra L. Walsh and Sarita S. Warrier in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-3-mde-10.1177_23821205231225589 - Supplemental material for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations
Supplemental material, sj-docx-3-mde-10.1177_23821205231225589 for The Medical Student Guide to Pharmacy: Piloting an Interactive Textbook on Basic Pharmacology Principles and Clinical Correlations by Kendra L. Walsh and Sarita S. Warrier in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgments
We would like to thank the talented, hardworking pharmacists at Rhode Island Hospital and The Miriam Hospital whose advice and guidance assisted in the creation of this resource for our medical students. We would also like to thank Jillian McGuire Turbitt, M.Ed. for her continual advisement and support with medical education technologies.
FUNDING
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Independent funding for the gift cards was provided by the Office of Medical Education at The Warren Alpert Medical School of Brown University through faculty professional development funds.
DECLARATION OF CONFLICTING INTERESTS
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Author contributions
KLW initiated the research project as well as wrote and edited the manuscript. SSW assisted with general oversight of the research project and provided critical edits to the manuscript.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
