“Holisitic admissions” and the elimination of standardized tests for graduate school

• Read original article here

“Holisitic admissions” and the elimination of standardized tests for graduate school
August 3, 2022 • 11:00 am
The article below in Science by chemist Anna Mapp (now Associate Dean for Academic Programs and Initiatives at the University of Michigan) is part of the drive to achieve to create a “fairer” process in admitting students to science graduate-school programs. This is to be accomplished, argues Mapp, by eliminating the Graduate Record Exam (GRE) as a required test for admission to graduate school.  (Colleges and grad schools are eliminating standardized tests left and right, despite the recommendations of committees, like one at Berkeley, to keep them as part of an admissions process.)
As Mapp’s title implies, the purpose of eliminating such tests is to achieve equity (i.e., proportional representation of all genders and ethnic groups in grad schools). And indeed, given that the elimination of standardized testing as a criterion for admission will create more equity, she may be right. (Schools vary on how much weight they put on GREs). The question is whether graduate schools, in the face of comparatively poor performance of many minority students on such tests, should be eliminating them as a form of social engineering.
As I’ve said, while I favor some form of affirmative action up to the point of hiring a faculty member, and that includes graduate admissions, I am still conflicted about it; it’s one of the few social issues that I can argue either way on, and I have no concrete solutions.  My real solution, as I’ve said before, is to fix the pipeline entrance: that is, to assure everyone of equal opportunity from birth. Right now we have neither the will nor the funds to do that. All we can do is try to fix outcomes, and that itself is problematic.
From Mapp’s article we learn what we already knew:
Science journals are now promoting the achievement of equity almost as much as they’re promoting the production of good science, even if the two goals conflict.
“Holistic admissions”, favored by Mapp and others, sound good, but really means “admissions that ensure equity”. In fact, grad schools, as I know well, already practice holistic admissions. In biology, for instance, we look not only at standardized test scores, but also grades and letters of admission, previous research accomplishments, and the students’ statements, as well as personal interviews. There is no cutoff for GREs. GREs are probably one of the most unimportant factors we’ve considered.
Never underestimate the ability of social-justice academics to burnish their drive for equity by making assertions that have no empirical support.
Click to read:
The GREs, which comprise a verbal and a quantitative test (there are also subject tests in chemistry, physics, psychology and mathematics ) are described in Wikipedia this way:
According to ETS [Educational Testing Service], the GRE aims to measure verbal reasoning ,  quantitative reasoning , analytical writing, and  critical thinking  skills that have been acquired over a long period of learning. The content of the GRE consists of certain specific  algebra , geometry,  arithmetic , and vocabulary sections. The GRE General Test is offered as a computer-based exam administered at testing centers and institution owned or authorized by  Prometric . In the graduate school admissions process, the level of emphasis that is placed upon GRE scores varies widely between schools and departments within schools. The importance of a GRE score can range from being a mere admission formality to an important selection factor.
Until 2011, the scores ranged between 200-800, but have now been compressed to a 130-170 score for the main two tests. (Subject tests are graded between 200-990). To see the performance of different sexes and ethnic groups on the test, you can consult earlier results from 2001-2002 here , and updated and compressed-range statistics from 2016 here . In general, comparing men and women among all groups, women have a slight advantage in the verbal section while men have a slight advantage in the quantitative section. I don’t know the standard deviations, but certainly men and woman score equally in verbal skills, and the difference isn’t large in quantitative skills.
A bigger difference is seen among ethnic groups; here are the data from 2016:
Those differences aren’t that large—though they’re more obvious if you look at the 200-800 point era—but remember that these are averages. What schools are worried about are the average GRE schools of students who get admitted, which clearly shows the degree of affirmative action, at least in terms of standardized tests.
Do GRE scores correlate with “success” in graduate school? An analysis that I think is pretty objective says yes, but weakly. The problem is that ‘success’ is almost always measured in terms of whether or not you get a Ph.D., not how well you do as a scientist.
Taken as a whole, the evidence suggests that there is some correlation between GRE scores and graduate achievement. But there is widespread disagreement about the degree of correlation. The Educational Testing Service, which funds a considerable amount of research into the validity of the GRE, asserts only that “GRE General Test scores tend to show moderate correlations with first-year [GPA] averages”(ETS 1990). It also admits that there are “critical skills associated with scholarly and professional competence that are not currently measured by graduate admissions tests” (ETS 1989).
The devil is in the details when it comes to GRE validity studies, as relevant correlations are often embedded within distinctions in the data. For instance, there is considerable amount of variability in predictive validity of GRE scores between disciplines (Braun and Jones). This can partly be explained by the fact that the GRE is actually three separate tests: analytic, verbal, and quantitative. Different disciplines demand these skills in different degrees. Thus, predictive validity tends to improve when a particular test is matched to a particular discipline.
In her article, Mapp recommends ditching the GREs because, she argues, they are used as cutoffs or major criteria for graduate-school admission, watering down the “holistic” admissions she wants (and that we already have):
Earlier this year, the University of Michigan became the first US university to remove the requirement that applicants to its nonprofessional doctoral programs take a standardized test—the Graduate Record Examination (GRE). This decision will not, on its own, address inequities in admissions practice, nor the broader education barriers that many applicants face. But it is a major step toward an admissions process that considers all dimensions of a candidate’s preparation and promise—a holistic view that should be adopted by all universities if equity in education and opportunities is to be achieved.
 
The stated goal of the GRE general test is to assess writing skills and quantitative and verbal reasoning. Its actual value as an instrument for determining who is likely to be successful in a doctoral graduate program, however, has not been established. There is a positive correlation across disciplines between GRE scores and first-year graduate course GPA (grade point average). However, although coursework is part of obtaining foundational knowledge in a field, the central focus of a doctoral degree comprises intellectual contributions and research accomplishments (measured in many dimensions, such as publications, presentations, and patents).
But the way that graduate admissions work is already holistic: I have spent hours reading dossiers and discussing admissions in faculty meetings, and GRE, while considered, is just one of many factors used in a “holistic” way: letters of recommendation (often useless), grade-point averages (hard to compare among schools, particularly with grade inflation), and to me, the most important considerations: research experience, knowledge of the subject, and drive to study it, all assessed in the personal interviews we have with students. Because schools accept far fewer graduate than undergraduate students, they are vetted more comprehensively. So what is the point of removing the one single measure that can compare the achievements of all applicants on a single scale, especially since that score is one of many factors weighed, and not used (at least as I’ve seen, as a cutoff to eliminate minorities). Even the ETS itself recommends “holistic admissions”!:
Regardless of the decision to be made, multiple sources of information should be used to ensure fairness and balance the limitations of any single measure of knowledge, skills, or abilities. These sources may include undergraduate grade point average, letters of recommendation, personal statement, samples of academic work, and professional experience related to proposed graduate study.
As anyone in academics knows, the drive is to accept more minorities, not eliminate them! And “holistic admissions” themselves can be dicey: remember that Harvard University seems to have deliberately downgraded the “personality scores” of Asian and Asian-American applicants (without meeting them!), apparently to reduce the proportion of Asian students in their class.
But Mapp implies otherwise—people are still plotting to do down minorities.
Once applications have arrived for review by a doctoral admissions committee, use of the GRE can lead to additional loss of talented applicants. Despite recommendations that GRE scores only be used in the context of an overall evaluation of an applicant, in practice they can readily be employed—implicitly or explicitly—as cutoffs for further consideration of an application. This is an especially problematic practice given that GRE scores consistently correlate with the sex, ethnicity, socioeconomic status, and race of the test taker.
So do grade point averages, by the way! But to claim that GRE scores are employed as a tool to get rid of minority applicants is nonsense, and Mapp gives no evidence for this. I’ve certainly never seen it. Schools are dying to accept minority applicants; the issue, at least in STEM, is finding acceptable minority applicants. And to do that we do pay less attention to their GREs than usual, for that is a form of affirmative action—and one that I don’t oppose. But if the GRE of anyone is extraordinarily low, that does tell us something that should be weighed against an applicant.
Mapp brings up the “GREs are a tool of structural racism” later as well (my emphasis):
Discontinuing the use of the GRE in doctoral admissions is, therefore, a valuable step toward best practices to reduce the barriers—systemic and otherwise—faced by students, to decrease bias in the evaluation process, and to support admission and recruitment of outstanding student cohorts. A concern is that in the absence of the GRE score, admissions committees will rely more heavily on other aspects of a candidate’s application that—like the GRE—are strongly correlated with race, sex, and socioeconomic factors rather than talent and promise. This includes undergraduate institution ranking, prestigious internships, and performance on other standardized tests. It is for exactly this reason that frank conversations about discordance between the stated goals and values of doctoral programs and the common practices in doctoral admissions are indispensable.
If I read this correctly, she says that even if GREs are eliminated, “structural racism” will still rear its ugly head, with committees using proxies like what school the student came from, “prestigious internships” (what about just “internships”?) and so on, all as a way to block minority advancement.  I’m not sure what she’s suggesting we do here, but having a “frank conversation” is not one of them. To argue that the attainment of equity may be inimical to scientific merit, given that the two seem to be at odds and you must prioritize what you want grad-school admissions to accomplish—well, that’s a discussion that is simply taboo.
Two more points. First, Mapp argues that there’s no downside to eliminating the GREs as estimated from data:
In recent years, many individual graduate programs have removed the GRE requirement, with notable success.
What does she mean by “notable success”? If she means “increasing equity,” then yes, she’s probably right. (She doesn’t give the data.) But if she means “quality of scientific work and success of applicants after they graduate,” then I have no idea. Where are the data supporting that claim?
Finally, inequities are, says Mapp, promoted by the costs of test preparation and of taking GREs themselves:
Beyond major questions about GRE utility, there are substantive financial and opportunity costs associated with requiring the GRE: costs for applicants and costs for admissions committees. For the potential applicant, the test is expensive; registration alone has a price tag of US$205, with additional fees for score reporting. (A fee reduction is possible for qualified applicants.) Many feel pressure to enroll in expensive GRE prep courses that teach test-taking strategies and often offer money-back guarantees for competitive scores and score improvements. The GRE 162+ course offered by Princeton Review, for example, guarantees scores of at least 162 (out of 170) at a cost of US$2149.
What are the costs for admissions committees that use the GRE in admissions decisions? In short, the loss of talented applicants at every stage of the process. Students who do not have the financial means to prepare for and/or take the test will not apply. A further loss comes from potential applicants who lack access to the test, either because of the limited availability of physical testing sites or because of lack of access to wireless networks needed for online test-taking options.
Note that she adds that there are fee reductions for taking the tests (not for foreign students, though). But the GRE fees are often lower than the application fees for grad schools, which begin $100 per school—and these fees are often waived, too . Any socioeconomically deprived person who wants to take the test and apply for grad school can find an affordable way to do so. In lieu of tutoring, many simply buy GRE-prep books that have sample questions. A student who wants to go to grad school should at least have the gumption to overcome these minor barriers.  After all, one wants students who will reach out and do what they need to do, at least for these minor matters.
In the end, one has to ask “what is gained by ditching the tests?” As far as I can see, nothing, for it eliminates one piece of a holistic form of evaluation that’s already in place, and the only piece for which all students can be compared against each other. It’s been eliminated for one reason only: if GREs are used at all in admissions, and there’s no “holistic” process, this use can reduce equity. Whether you want that depends on whether you think that graduate schools should be practicing social engineering. Of course they should be reducing bias (and equal opportunity) as much as they can, but if you want to use grad school to make up for past wrongs, well, that’s a discussion that needs to be had, but the one discussion that can’t be had. (I am, however, having it here!)
And there’s one down side that’s very real, but ignored in this article. High GRE scores can help identified good candidates overlooked for other reasons, like having not done well gradewise, or who come from foreign schools where it’s hard to evaluate grades. In other words, GREs are a way to seek out for and help high outliers.
This isn’t just theoretical. I know more than one scientist whose GRE scores have helped them get into grad school, and then become successful scientists. Here’s some testimony by one person I know:
The GRE test can be particularly helpful for foreign students coming from unknown 3rd world countries with grades that do not translate to the US grade system and who submit recommendation letters from unknown people written in a style that is not the US style. As an undergraduate in South America, I had an 8/10 grade-point average and was second in my class, but that translates to only a 3.2 GPA in the US—not an impressive score.  Because my GRE was very high (94th-99th percentile), I had no trouble being admitted to 8 US graduate schools, even though my university and recommenders were unknown.  GRE scores can also help students who did not dedicate themselves in their undergraduate studies, but matured later in life.  By getting a high GRE scores, these students might be able to bypass their poor grades and be accepted in graduate programs.
Share this: