Being There: The Importance of Direct Observation in Resident Assessment

Abstract

There are many methods to assess resident competency. In this report the author describes an experience that strongly suggests the superiority of direct observation compared to other methods. Literature comparing direct observation to other methods is cited. In addition ways to create time for direct observation and how to promote wider use of direct observation are discussed

The primary goal of residency training programs is to produce competent and compassionate physicians. To accomplish this, each program develops unique training features that are encompassed in the 6 core competencies of the ACGME (Accreditation Council for Graduate Medical Education). Program faculty are relied upon to provide formative trainee assessment of all competencies which are then summarized by the key faculty and the program director.

Assessments done with faculty present when residents are caring for patients, are optimal. Although most faculty want to do this, a number of obligations may not allow faculty to be present at these times. Factors impeding direct observation include research commitments and clinical care of their own patients. There also may be insufficient funding for this faculty time. ¹ One consequence of this lack of faculty presence is that some trainees have never been observed performing a history and physical examination. ²

Because of this dilemma, surrogates for direct observation are used frequently. Video-taping of resident care is used in some programs. USMLE Step 2 includes a clinical skills component with direct observation by standardized patients. Other boards have adopted simulation and some have oral examinations.

As a clinician-educator and former program director, I too have been caught in the dilemma of how to fulfill obligations and at the same time, evaluate residents. A recent experience demonstrated to me that we should not rely too heavily on surrogates for direct observation.

After a night on call, a resident presented a patient. The resident was in the Emergency Room evaluating a patient who was sitting upright on a stretcher in obvious respiratory distress. The resident learned that 3 weeks ago the patient had fallen and sustained a non-displaced tibia fracture. She did well until 3 days prior to this presentation when she developed dyspnea on exertion. The dyspnea progressively worsened and her family insisted that she be seen.

On 4 L of oxygen, pulse oximetry was 90%. With any slight movement, further desaturation was seen. The right lower extremity had significant edema. There was no edema of the left leg. A bedside echocardiogram showed right ventricular strain. The patient next underwent CT angiography, showing multiple pulmonary emboli. The largest was in the RV outflow track. As the patient was taken to Radiology and returned to the ER, all movements caused hypotension and further oxygen desaturation.

After her initial assessment, the resident had a high pretest probability for pulmonary embolism. In fact while the chest CT was being done, the resident ordered a thrombolytic infusion.

In our discussion, a number of issues came to light. The resident felt reasonably confident in her decision to administer thrombolytics. She had some background information on the subject and had done a quick internet-based review while on call. But even with this knowledge the resident acknowledged that she struggled. She had never personally been involved in giving thrombolytics and her direct responsibility for the patient’s outcome was a new experience. I then thought about using this personal recount to develop an assessment of the resident. She seemed to have had adequate medical knowledge and demonstrated some competency in professionalism and interpersonal skills. But on further reflection, I concluded that basing an evaluation on post-event information provided by the resident may not be completely accurate. Missing was how did the resident handle the stress for the patient and for herself? Were there others involved in the care of the patient or was this all done independently by the resident? These concerns stem from not being present at the time. Had I been there, I could have directly seen the resident interacting with the patient and other health care professionals. I would also have a better sense of her professionalism and her ability to come to conclusions. Most importantly I could give a much better overall assessment of the resident.

Based upon this notion and a significant number of reports in the literature about the importance and superiority of direct observation, I have changed what I do when evaluating residents.^3-5 On an inpatient service, I minimize all other commitments so that I am present on the ward for almost the entire day and evening. This allows me to see almost all patients on admission with the residents and to be present when acute events occur. I try to do this mostly in an observer role so resident autonomy is not compromised. When precepting residents in the ambulatory setting, I do not have my own schedule of patients so that I am available to residents and can see patients with the residents.

These changes were made at the beginning of the academic year. At the end of the year, I felt that my role as an attending-evaluator was improved greatly. I was able to review resident performance as it occurred and provide immediate feedback. I also felt better integrated in the inpatient team. The feedback I have received from residents is positive.

I believe this style of attending should be more broadly adopted. There are potential barriers. First, there needs to be departmental support for the idea that attending is a full-time activity. It is permissible to have no outpatient clinics scheduled when attending on an inpatient service. It is also permissible in the ambulatory setting to be solely present as a preceptor. I have been fortunate to have this type of support from my department. Second, faculty may resist due to concerns that they may not have the proper skills. To address this, faculty can attend workshops on how to effectively carry out direct observation. Encounter cards and checklists have also been developed to aid in direct observation.^5,6

It is a privilege to be a faculty preceptor. My experience suggests that being present in the training space promotes best practices in all competencies. Taking the effort and having the support to do this promotes our commitment to the training of the next generation of practicing physicians.