Exams that US students need to take before being allowed into graduate school are not a reliable way of assessing whether those candidates will successfully complete a PhD. That is the claim of new research, which shows that the over-reliance on the Graduate Record Examination (GRE) in PhD admissions also discriminates against under-represented groups. The study has been carried out by a team led by Casey Miller — a physicist from the Rochester Institute of Technology — who analysed data for almost 4000 students who entered physics PhD programmes between 2000 and 2010, representing around 13% of doctoral enrolments during that period.
The system needs fixing to reach an acceptable level of equity
Casey Miller
In addition to looking at a student’s average grade earned during a degree – known as the undergraduate grade point average (GPA) – many physics doctoral programmes in the US also require applicants to submit GRE scores for a general exam and subject specific tests. However, Miller and colleagues found that undergraduate GPA and the ranking of the graduate programme were the only factors that indicated whether a student would successfully complete a PhD. GRE physics scores, gender and citizenship all had no significant effect on PhD completion. In addition to being poor predictors of PhD success, the researchers say that GRE scores also have large performance gaps based on race, gender and citizenship. This, they claim, is driven by factors like test anxiety and unequal access to expensive coaching or resources.
Assessing competencies
Around a quarter of US physics PhD programmes require a minimum GRE physics score of around 700, which places applicants in the 55th percentile, according to the researchers. They say, however, that the proportion of people that reach this level is not representative of those that take the test. For example, Hispanics account for 6.2% of test takers, but only represent 4.1% of those with scores above 700, while Asians make up 7.8% of test takers and 11.4% of scores of at least 700. Whites account for 78% of both test takers and scores above 700.
“The system needs fixing to reach an acceptable level of equity,” Miller told Physics World. “Programmes need to reflect on what they want to be and what student attributes can help them in that regard, then tie their admissions requirements to those goals.” Miller adds that graduate schools need to start assessing the “emotional-social – or non-cognitive – competencies” that faculty say make for great researchers such as perseverance, adaptability and accurate self-assessment.
GRE scores also act as a mirror that reflects societal disparities
David Payne
Previous studies indicate that around 40% of physics graduate schools require a minimum GRE score for admissions, even though it is recommended against by ETS – the non-profit organization that administers the tests. David Payne, vice president of global education at ETS, admits that an over-reliance on GRE scores – or any single measure – can have negative consequences especially for women and minorities. “It is for exactly this reason that ETS’s communications have, for some decades now, warned against the use of strict cut scores.”
Bringing value
Some US doctoral programmes are moving away from GREs. In 2016 the American Astronomical Society adopted a resolution recommending that graduate schools in astronomy eliminate GRE scores from applications, or make them optional. This was due to previous research indicating that they are poor predictors of success and have a negative impact on under-represented groups.
The physics department at the University of California, Irvine, has recently voted to completely exclude the physics GRE from its admissions process. James Bullock, head of physics, told Physics World that the department had seen “no correlation between physics GRE score and success, as measured by several different measures”, such as securing post-doctoral positions after graduation. Bullock says that he understands from colleagues that physics departments at other universities have also been discussing whether to drop the GRE from their admissions criteria.
Shake-up to US graduate education needed, panel warns
Payne told Physics World that rather than throwing away GRE scores and the “unique value they bring”, schools should take various factors into account. “Students from higher socioeconomic status families are more likely to attend prestigious private undergraduate institutions, have access to prestigious and eloquent letter writers, and have multiple mentors to look over their personal statements,” he adds. “Like these submitted materials, GRE scores also act as a mirror that reflects societal disparities. However, unlike other measures, the GRE tests are designed and reviewed to make the test as unbiased as possible.”
