Observations: We Need to Stop Drowning-A Proposal for Change in the Evaluation Process and the Role of the Clinical Competency Committee.

In 2012, Nasca and colleagues1 described proposals for the adoption of the Next Accreditation System, including assessment using milestones and a clinical competency committee (CCC). As programs attempted to implement these changes, the time commitment for assessment and documentation has become significant. We need to consider a change in our approach to evaluations, for we are drowning in a sea of increasing real and perceived documentation requirements and have to wade to the shore. I attended a program director boot camp at the American College of Cardiology meeting this past March. Program directors in cardiovascular disease fellowship training talked about the role of the CCC in determining trainees’ progress on the subspecialty milestones. I had attended presentations on this topic sponsored by the Alliance for Academic Internal Medicine in 2012 and 2014. At both meetings, speakers described a system using the transfer of information from postrotational evaluations to a tool for competency-based assessment in entrustable professional activities. A common theme was that the process of incorporating evaluations into milestone-based assessments was complex and sometimes convoluted.2,3 Significant time requirements were reported—as much as 3 to 6 hours of staff and faculty work per trainee were necessary to prepare data for submission to the CCC, with much of this related to culling information from evaluation forms. CCC meetings resulted in even more time commitments, averaging 1 to 3 hours per meeting. This effort likely is not sustainable and, at the least, creates a sense of dread of the evaluation process. We need to change our approach to evaluations and reestablish joy in teaching, including providing prompt formative feedback to our trainees. As program directors and coordinators, we bird-dog faculty to complete multipage evaluations after clinical rotations, requiring supervisors to assess trainee progress in processes ranging from medical interviewing to performing procedures. Evaluations include a space for narrative comments, but after spending 5 to 15 minutes filling in check boxes, formative comments are found only in the minority of evaluations. Yet these comments are the most important tool to direct trainee efforts to improve competency. We need to make evaluations less of a chore, and more relevant in making trainees become better physicians. This means we need to change our approach to evaluations and provide more concurrent feedback. In the current system, the preparation of the material needed for the evaluation of progress along the milestones often is so time consuming that many CCCs will evaluate each trainee only a few times per year. Feedback is most effective if given when areas for improvement are identified. The onus of the process to give milestone-based evaluations limits our abilities to provide effective formative feedback. We need to change this. In our program’s efforts to deal with these issues, we have blown up and reinvented the evaluation system. Instead of asking faculty members to fill in the check boxes on an assessment tool after each rotation, we ask them to write down formative comments. This may be a single line describing strengths and areas for improvement for a trainee, or a paragraph describing the assessment of a core competency. We try to decrease the time requirements and hassle factors associated with performing evaluations. These efforts have resulted in more timely evaluations and higher completion rates. To provide more timely and relevant feedback, our CCC meets every 4 to 5 weeks. We evaluate each class of fellows by academic year. This allows us to compare fellows of similar levels of training. We have a small program, encompassing 7 fellows and 13 key clinical faculty members. In exchange for increasing the frequency of our CCC meetings, we limit meetings to 30 minutes. We have asked that key clinical faculty attend each meeting, and we usually have at least 7 faculty members present. We project onto a screen our last milestone assessments for each fellow, and ask faculty for an evaluation of performance, comparing our current assessment with the last one. Faculty members who have recently worked with the trainee provide verbal comments, and those who have not done so are asked to remain silent. To create normative evaluations, we require a consensus on ratings. This process usually requires less than 1 minute per category per fellow, allowing for rapid documentation and norming. Clearly, when concurrence is not achieved, we spend much more time deliberating. At the end of numeric milestone scoring, we ask for verbal comments on each fellow’s progress. The results are collated by our program administrator, and distributed to the fellow after each CCC meeting. Fellows also perform self-evaluations of milestones competency prior to meetings, and are asked to compare their self-assessments with those of the CCC. The program director does not meet with each fellow after the CCC evaluation, in order to allow the development of self-reflection. However, the program director is available for questions. The frequency of CCC reviews allows performance issues to be addressed early, before more formal remediation processes are required. I recognize that our system would be difficult to use in a larger program; still, the creation of smaller subcommittees to address evaluations in a collaborative, engaged fashion could make this feasible in a program of any size. Our efforts have improved faculty engagement and satisfaction with the evaluation process. By requiring comments, instead of completion of check boxes, we promote more relevant feedback to fellows. The process has resulted in early and less punitive remediation, and allowed for timely, earlier assessment of progress. While joy in medical education remains an elusive goal, we attempt to achieve it.