Among the endless metrics for assessing the quality of health care, one that is exceedingly important for measuring physician quality is on the chopping block.
I’m talking about turning the U.S. Medical Licensing Exam Step 1, which all medical students take at the end of their second year of studies, into a pass/fail test. This proposed change was quietly announced by the owners of the test and has received almost no media coverage. Such a pivotal change, which I find troubling, merits greater attention and debate. If you, too, think it is an unwise move, make your voice heard before comments on the proposal are closed on July 26.
Over the past 20 years, medical schools have mimicked larger trends in higher education to become more diverse by deemphasizing standardized testing for admissions. Once medical students are in the door, administrators have attempted to address disparate levels of achievement by embracing pass/fail grading systems that emphasize experiential learning over foundational concepts.
The lack of objective metrics has led to a greater reliance on numerical scores from the Step 1 exam. For the directors of residency and fellowship programs, these scores serve as easily quantifiable proxies of ability. A change to pass/fail would represent the culmination of years of watering down medical education in ways that have significant repercussions for medical training and patient care.
In theory, these well-intended efforts aim to reduce pressure on medical students and increase diversity. Yet they are likely to have the opposite effect.
Without objective standards like standardized testing or grades, residency directors will have to focus on who you know and what you are. Using the who-you-know standard will place even more pressure on students to secure entrance into prestigious colleges and medical schools because the proxy of academic pedigree and network of elite institutions will carry more weight when it comes to placement in residency and fellowship programs. This will increase the medical school rat race by forcing students to focus on accumulating secondary laurels such as pursuing myriad leadership activities or pledging indentured servitude to research professors who will have outsized influence on shaping their residency prospects due to the absence of other objective standards.
Without objective ways like grades and Step 1 scores to measure ability, the what-you-are standard will make it easier to justify race-based quotas in residencies and fellowships. Marginalizing the importance of standardized testing may make it easier to expand opportunities for individuals who can check the right gender and race boxes. But it comes at a cost for people who do not have the “right” networks and fall outside of the “right” categories.
Admission to residency programs based on networks and categories is, by definition, arbitrary. In a world without standards, people may simply assume that medical trainees got into their programs because of their elite connections, race, or gender. This would delegitimize medical students’ efforts and the process itself. Standardized testing, for all of its alleged flaws, is the key to genuine equal opportunity. By having objective metrics, medical students can focus their energies on mastering the material and demonstrating they have the ability to succeed as physicians even if they do not come from the most prestigious institutions or well-connected families.
If elite networks and pedigree had been the key a generation ago, Asian-American students like me would have had limited access to medical school or residency slots and would have been on the outside looking in — just as Jewish students were a generation earlier when elite universities and hospitals had quotas to limit the number of accepted applicants who were Jewish. In a profession where highly refined skills, impeccable judgement, and swiftness of mind can make a difference between life and death, standards that rely on networks and categories of race and gender undercut the point of medical training.
Medicine and merit should go hand in hand. When you face significant illness, you want to see physicians whose decades of rigorous training in medical school, residency, and fellowship serve as proxies for their competence. The recommendation for pass/fail reporting of Step 1 scores represents an unvarnished attempt to abolish the last merit check in medicine. While this approach may open the door for greater diversity in residency and fellowship placements, it perpetuates a pattern of evisceration in quality and standards.
No one wants to see a physician who gets diagnoses right only half of the time. Watering down the system by eliminating numeric scores on Step 1 equates to root rot that will gradually undermine the medical profession.
Kim-Lien Nguyen, M.D., is a cardiologist and assistant professor of medicine at the David Geffen School of Medicine at UCLA. The views expressed are those of the author.
The Steps were not designed to provide any kind of predictive information regarding students’ ability to succeed in residency or beyond. In my specialty (ENT), there have been multiple studies showing that they correlate modestly with passing board exams but not much more. The cutoff for this correlation is significantly below the insanely high scores we see when interviewing applicants. Are they objective? Yes … (well, sort of: some Med Schools have been gaming the system by allowing students to take the test during/after 3rd year or after significant amounts of time off, which undermines the reliability of the test as an objective metric). Do they allow us to rank students? Yes. Do they actually help us determine applicant quality? No. We could also use height or zip code, which are also objective and similarly non-correlative. I empathize with the dilemma PDs face with sifting through the hundreds of applications they receive. They do need an objective measure to meaningfully rank applicants, but we need to be honest with ourselves that that the Steps do not offer this. Med schools need to start developing more meaningful, fair, quantitative ways of measuring performance and aptitude. Further, specialties need to start to developing more comprehensive, sophisticated and quantitative ways of evaluating applicants. There’s no question we need to be able to rank applicant quality, but we shouldn’t pretend that we can do this with a multiple choice exam.
Testing well doesn’t always make someone a better doctor
Some people don’t do well on tests but are great at the bedside
Wow, if these med-students think they are ‘stressed’ now, wait until their millennial dream world (aka nightmare) of socialized healthcare arrives!
https://journals.lww.com/academicmedicine/Fulltext/2017/11000/Why_Not_Wait__Eight_Institutions_Share_Their.11.aspx?utm_source=share&utm_medium=ios_app&utm_name=iossmf
You really don’t think that you’ll do better on an exam when you’ve taken like 8 different shelf exams before that?
Step 1 was a joke after I completed Clerkships. Our class had a 20 plus point jump on average just taking it after clinicals. This nonsense that step 1 is the be all needs to be put away.
First, I will admit I wasn’t aware of those schools moving Step 1. But as a PD myself, I can flatly tell you that most of us are not aware of this.
Second, if you’re arguing to push Step 1 back to M3 I’m not going to fight vehemently against it. I still contend it distracts from your M3 rotations. But, regardless, that’s a DIFFERENT argument than saying to eliminate it completely.
The bottom line is this: We PDs need a 3 digit Step score to determine who to invite for interviews and who not to. If you make Step 1 pass/fail and schools make Step 2 “optional” to complete prior completing M3, then any student who doesn’t have a Step 2 score to submit to me won’t get an interview. Period.
So now everything is riding on Step 2.
Let’s recap…
Before: Step 1 taken before clinicals. If you do well, great. You’re set and you only had to take 1 test before applying to residency. If you do poorly you get to take Step 2 to improve your score from Step 1.
After: Everyone has to take two exams instead of 1 prior to applying to residency. You don’t get to “retake” Step 2 to improve your score if you happen to do poorly.
Or worst possible scenario: Nobody takes Step 2 prior to applying to residency. In which case, I’m only interviewing students from top 20 med schools for my residency program. I hope you’re one of them.
Good luck.
UCSF students take step 1 and 2 after M3, UPENN students take step 1 and 2 after third year, same with Duke, should I keep going?
As a doctor in practice for 30 years, this is by far the stupidest idea I have ever heard! Now your grades in college are more important than ANYTHING! Getting an A in Underwater Basket weaving will be more important than getting a B in advanced Biochemistry? An what about the student who didn’t decide in kindergarten that he wanted to be a doctor? Those students typically go to DO schools or second Tier schools and excel because they didn’t check off all the right boxes. Or what about
the student who didn’t do well their first year of college! You will be eliminating them as well! Do you think people are going to study HARD knowing the exam will be pass fail! How about a student who thinks he wants to go into Dermatology but then doesn’t do as well on his Step 2
Medicine used to be a place where cream rose to the top! I guess this won’t be true anymore
I am just genuinely interested after reading all of the comments below about how people think this change will help medical student stress levels. At the end of the day residencies can’t “holistically” evaluate 1000’s of applications and step 2 CK will become what step 1 was. Dr. Ryan just posted on twitter that he’s started working on step 2 CK material already ffs.
Also blatant nepotism in the admissions process and clinical evaluation process make objective scoring necessary. What we signed up for is competitive and it’s bizarre to me that people don’t realize getting rid of step 1 isn’t going to change that. Just takes away an opportunity for excluded groups to make themselves stand out.
Also for everyone saying step 2 is a better metric, clinical guidelines constantly change, basic science and physiology don’t. Understanding the basic science is the foundation of why people are constantly challenging current clinical guidelines and improving patient care. A bunch of physicians who understand what their supposed to do and not why is kind of a scary concept.
When you took step 1 – your scores were much lower. The average has risen over the past 10-15 years. Our scores would blow yours out of the water. Step 2 is a better exam.
To “Resident” – So what? Many test scores (e.g., SAT) rise over time. That is a reflection of students adapting to the test with better test prep material such as books, online modules, etc.
You can say Step 2 is a better exam. It doesn’t negate the fact that there needs to be an exam that is GRADED (not pass/fail) and taken by ALL students as means of comparing students from different schools. Unless you’re mandating Step 2 be taken before M3 year (which is *not* what has happened), you are basically telling program directors like myself to take students purely based on the reputation/ranking of their medical school.