Among the endless metrics for assessing the quality of health care, one that is exceedingly important for measuring physician quality is on the chopping block.

I’m talking about turning the U.S. Medical Licensing Exam Step 1, which all medical students take at the end of their second year of studies, into a pass/fail test. This proposed change was quietly announced by the owners of the test and has received almost no media coverage. Such a pivotal change, which I find troubling, merits greater attention and debate. If you, too, think it is an unwise move, make your voice heard before comments on the proposal are closed on July 26.

Over the past 20 years, medical schools have mimicked larger trends in higher education to become more diverse by deemphasizing standardized testing for admissions. Once medical students are in the door, administrators have attempted to address disparate levels of achievement by embracing pass/fail grading systems that emphasize experiential learning over foundational concepts.

advertisement

The lack of objective metrics has led to a greater reliance on numerical scores from the Step 1 exam. For the directors of residency and fellowship programs, these scores serve as easily quantifiable proxies of ability. A change to pass/fail would represent the culmination of years of watering down medical education in ways that have significant repercussions for medical training and patient care.

In theory, these well-intended efforts aim to reduce pressure on medical students and increase diversity. Yet they are likely to have the opposite effect.

Without objective standards like standardized testing or grades, residency directors will have to focus on who you know and what you are. Using the who-you-know standard will place even more pressure on students to secure entrance into prestigious colleges and medical schools because the proxy of academic pedigree and network of elite institutions will carry more weight when it comes to placement in residency and fellowship programs. This will increase the medical school rat race by forcing students to focus on accumulating secondary laurels such as pursuing myriad leadership activities or pledging indentured servitude to research professors who will have outsized influence on shaping their residency prospects due to the absence of other objective standards.

Without objective ways like grades and Step 1 scores to measure ability, the what-you-are standard will make it easier to justify race-based quotas in residencies and fellowships. Marginalizing the importance of standardized testing may make it easier to expand opportunities for individuals who can check the right gender and race boxes. But it comes at a cost for people who do not have the “right” networks and fall outside of the “right” categories.

Admission to residency programs based on networks and categories is, by definition, arbitrary. In a world without standards, people may simply assume that medical trainees got into their programs because of their elite connections, race, or gender. This would delegitimize medical students’ efforts and the process itself. Standardized testing, for all of its alleged flaws, is the key to genuine equal opportunity. By having objective metrics, medical students can focus their energies on mastering the material and demonstrating they have the ability to succeed as physicians even if they do not come from the most prestigious institutions or well-connected families.

If elite networks and pedigree had been the key a generation ago, Asian-American students like me would have had limited access to medical school or residency slots and would have been on the outside looking in — just as Jewish students were a generation earlier when elite universities and hospitals had quotas to limit the number of accepted applicants who were Jewish. In a profession where highly refined skills, impeccable judgement, and swiftness of mind can make a difference between life and death, standards that rely on networks and categories of race and gender undercut the point of medical training.

Medicine and merit should go hand in hand. When you face significant illness, you want to see physicians whose decades of rigorous training in medical school, residency, and fellowship serve as proxies for their competence. The recommendation for pass/fail reporting of Step 1 scores represents an unvarnished attempt to abolish the last merit check in medicine. While this approach may open the door for greater diversity in residency and fellowship placements, it perpetuates a pattern of evisceration in quality and standards.

No one wants to see a physician who gets diagnoses right only half of the time. Watering down the system by eliminating numeric scores on Step 1 equates to root rot that will gradually undermine the medical profession.

Kim-Lien Nguyen, M.D., is a cardiologist and assistant professor of medicine at the David Geffen School of Medicine at UCLA. The views expressed are those of the author’s.

Leave a Comment

Please enter your name.
Please enter a comment.

  • Completely disagree. The step scores never correlated to how I did in residency. I struggled on exams more than others and had the stellar step scores we look for. It’s a game. It’s for money. It makes people feel superior and gives them something to hold against those who don’t perform on tests well. It makes the elite with a system designed for the top ten to twenty percent of learners and test takers. That’s okay. I have seen cardiologist with those crazy high step scores miss murmurs because they listened without intent or didn’t listen at all. In the end what’s more important.

  • This editorial brings up two separate issues. First, one that hasn’t been directly addressed is the need for residencies to use Step 1 as a predictor of in-service exam performance. I know this because my specialty is in teaching students (of any age) how to learn and I am often called on by our program directors to help rescue an otherwise clinically competent resident from dismissal due to failing in-service scores. My resident students have been successful in every case mainly because they are all smart enough even though they may not have learned how to learn. Program directors know that while there may be several valid explanations for a low score, e.g. illness, personal life issues, there is only one explanation for a high score. The assumption is that a high scoring student knows how to get it done. This is crucial for a program director who can only hire once a year if they lose a physician.

    The second issue is that Step 1 is, indeed, a snap-shot where real life is a video. However, Step 1 does address a few of the competencies related to diagnosis because MCQs are a metaphor for a patient. A vignette provides the needed patient data while the differential is provided as the answer choices. No student gets away with naming a diagnosis in the clinic, but they try to get away with this on Step 1. Success is guaranteed for a student who can explain why each wrong answer can be ruled out. This is a Step 1 effect at its best.
    When students know how to answer questions the way they will eventually perform on attending rounds, i.e. not with knowing the answers but with knowing the reasons for answers, then the test will measure some of their clinical potential. Step 1 has come a long way, but I still apologize to students when I lecture in biochemistry that much of the garbage that I will be teaching is aimed not at real life, but at Step 1. It just isn’t a clean cut issue. A pass/fail grading would still provide the motivation to learn, and if they follow the advice that I have at my website (just search for “John Pelley SuccessTypes”) they would improve their cognitive skills for the long run. The current system that claims to be able to tell the future for students that are a few points apart (or even many points apart) instills fear and extinguishes true intellectual development. This, by the way, is not an opinion but is backed up by research in learning extending from psychology, to sleep research, and information processing research (all of which are included in my Expert Skills Program).

    Dr. Nguyen, I invite you to visit my website and to communicate with me about any concerns. My intent is not to refute you because I think you speak from experience. My intent is, instead, to add to your experience. There is so much to know that is very exciting about the potential for our students to become the creators of their own understanding.

  • “We need Step 1 scores so that minorities don’t steal your residency spot!” seems like the subtext here. Not a good look.

    • As a medical student of color at DGSOM at UCLA I am very disappointed that the author of this editorial represents this institution. It is very troubling that this kind of rhetoric exists within our faculty physicians who we see as mentors for us physicians in training. The underlying messages here are that #1: because someone does better on Step 1 that automatically makes them a better physician, and #2: minorities do not score well on Step 1, so don’t allow them to take residency spots from those who do.

      Well Dr. Nguyen, as a fourth year I have seen MANY physicians who are extremely booksmart and who I’m sure scored well on their board exams but are lacking when it comes to bedside manner. For example, they know how to manage a patient with CHF but do not know how to manage the circumstances surrounding why the patient received that diagnosis in the first place or how to correctly manage the patient’s emotions. On the other hand, the physicians I consider my role models are also extremely book smart but did not receive the best scores on boards, yet they are the most compassionate physicians who are dearly loved by patients for that reason. Your argument is flawed.

      Secondly, I am a minority student (one of many) who scored well on Step 1. So let’s debunk this notion that minority students equal lower board scores, because that is simply not true. Instead, let’s talk about the institutionalized racism that exists and that our students of color face that may contribute to our ability to study and score as well as our priveledged counterparts. Let’s also talk about how your rhetoric Dr. Nguyen, is the sole reason why our communities of color do not have many doctors who look like and represent them because of people like you, who believe that standardized test scores make a competent doctor.

  • Although this article makes relevant arguments, I don’t agree with your opinion Dr. Nguyen. Step 1 scores will not determine if you are a better physician or not. We still have Step 2. Also, Step 1 scores now determine, in great part, income potential rather than “difficulty” of the residency program or specialty. Step 1 does not measure competence. It measures how much knowledge I had right before the test and how good students are a choosing the best answer.

    • Which part of it does not measure competency? The physiology? Pathology? Pharmacology? Microbiology? I don’t understand the insinuation that if you didn’t know it ‘that day’ that doesn’t imply lower competence? Why would you wait until after the exam to master the content you were asked to master? Step 1 is in fact a measure of not only competence but commitment to granular learning. The
      attention to detail required to consistently outperform your peers over the course of 8 hours of mentally fatiguing work is a virtue understandably valued by PDs. Most competitive specialties are surgical, and the importance of obsessive perfectionism cannot be undervalued in the OR. Even for lifestyle specialties, mental endurance is important. Derm is a great example, and you know all the nitty gritty pathophysiology is important to understand to differentiate seemingly similar skin lesions that may have very different treatments and prognoses.

    • Tyler- I understand your concern but I guess it comes down to what you want this test to measure. The test was originally designed to test *whether* someone was competent to practice medicine, not to differentiate between doctors levels of competency. In fact, the standard error of difference is 8 points, indicating that scores need to be at least 16 points apart to even be able to differentiate between these students (https://thesheriffofsodium.com/2019/05/13/another-mcq-test-on-the-usmle/). I think that the test itself is fine, but using as a residency selection tool is foolish. Meanwhile, being a doctor means a lifetime of learning, and your knowledge grows as you progress through your clinical experiences. Making decisions about ability to thrive in residency based on a test taken 2 years before you even start is also foolish. Residency selection is flawed for sure, but we need to come up with better measurement tools than step 1 to help us determine how we make our rank lists.

  • I don’t agree with this at all. The great majority of physicians I’ve spoken with think that step 1 is pointless or useless and couldn’t pass it at this point in their career if they had to take it again. Step 1 is about basic sciences and which enzymes or G proteins affect what system, a good portion of which we will never need to know again. Step 2 is much more important if you ask me, and the scores for Step 1 don’t seem to really matter as much as we’re led to believe. I know many students who crushed Step 1 and still had difficulty matching.

  • Almost all Medical Schools are pass fail, every deans letter says the candidate will be a great resident, all letters of recommendation say the candidate was superb based on a 4 week rotation, etc.
    Basically leaves residency directors blind as to who is a better candidate.
    The dumbing down of America continues, very sad.

    • We still have step 2. A much better and more clinically relevant test. Step 1 is a pretty useless test. I forgot 99 percent of that when I reached clinical rotations when I was in medical school.

  • Since when did step 1 correlate with how good a resident is? or how they do on their specialty boards? What about step 2CK? I’m in a surgical subspecialty but step 1 was definitely not even in the talks for fellowship. What’s the difference between a 240 and 250 anyway? Almost nothing. It might be a few questions.

  • “Standardized testing, for all of its alleged flaws, is the key to genuine equal opportunity.”

    What about all the $ you can pay for resources/tutors to increase your score Your argument that Step 1 allows students to be compared equally does not make sense. The NBME organization is making a ton of money from medical students that is the sole reason they are so opposed to changing the current test.

    “No one wants to see a physician who gets diagnoses right only half of the time. ”
    USMLE step 1 scores were not correlated to specialty boards scores and there is no evidence that higher-step-1 score students become better residents or attendings.

    Please read the following op-article written by students and residents regarding Step 1…

    https://www.medscape.com/viewarticle/914744

  • When I took this exam over 40 years ago, it was Pass-Fail. It had no effect on residency acceptance/matching. The main criteria for that were grades (how many Honors and in what), Dean’s letters and other faculty letters, and interviews. The choice of specialty was made out of desire to enter that field, not from having a better chance due to Board scores. Obviously in either scenario income potential and ability to pay off loans more quickly remains a factor in specialty choice.

Sign up for our Daily Recap newsletter

A roundup of STAT’s top stories of the day in science and medicine

Privacy Policy