Achievement and Standardized Testing

Back in 2001, Richard Atkinson, then president of the University of California system, sent shock waves through the testing community by suggesting that he would recommend dropping the SAT I in favor of the SAT II. The SAT II, unlike the SAT I, is more closely related to what students actually learn (or should learn, anyway) in high school; the SAT I is the more generalized version that we all love to hate.

Atkinson recognized not only that the SAT I was not a good indicator of college success, but also that preparing for it was distorting the educational process, as its creator, Carl Brigham, had predicted. Atkinson says that his epiphany came when he found his sixth-grade granddaughter studying SAT words to prepare for a test she wouldn’t take until her junior year in high school. He states it this way:

On my way home I stopped in Florida to visit my grandchildren. I found my granddaughter, then in 6th grade, already diligently preparing for the SAT by testing herself on long lists of verbal analogies. She had a corpus of quite obscure words to memorize, and then she proceeded to construct analogies using the words. I was amazed at the amount of time and effort involved, all in anticipation of the SAT. Was this how I wanted my granddaughter to spend her study time?

I think most of us would agree with him on that score, yet as a nation we are strangely reluctant to give up our reliance on these tests. More and more, schools and teachers are being judged by how much test scores go up, not by how well their students do when asked to write essays, do real math, and so on. More and more, classroom time is spent drilling for tests like the SAT and ACT, as well as an increasingly large number of other tests designed to measure incremental progress.

Readers will be familiar with the arguments about testing; my intent here is not to rehearse them all over again. However, I believe that if the testing monster is truly to be crushed or at least tamed, colleges and universities must be the knights in shining armor. As long as they rely so heavily on standardized for admission, and as long as those scores are perceived as indicating “quality” of student and by association of the institution, there will no effective way to put testing in its place.

When Atkinson suggested changing the SAT, the College Board leapt to comply, since the U of C system was one of the largest users of the test. As he wrote:

…[A]dmissions tests should not try to measure “innate intelligence” (whatever that is), but should focus on achievement—what the student actually learned during the high school years. In addition, such tests should have an essay component requiring the student to produce an actual writing sample. And the tests should cover more mathematics than simply an eighth grade introduction to algebra. And, finally, I said that an important aspect of admissions tests was to convey to students, as well as their teachers and parents, the importance of learning to write and the necessity of mastering at least 8th through 10th grade mathematics.

Since the SAT I was a supposed “intelligence test” this statement made the SAT IIs (which used to be called Achievement tests and which closely describes the ACT) more significant. His considered and authoritative decision, based on his own knowledge and research, had an important effect on the SAT. (How it turned out, of course, is another story.)

Since the early 2000s, however, testing for college has only become more fraught, digging its claws even more deeply into the American educational system. Even though it is generally acknowledged that test prep skews the playing field, scores are most indicative of zip code (i.e., income), little predictive value is added, and that admission officers have consistently ranked scores lower than GPA in determining admission, test scores are still the measure of success.

Many colleges have become test optional in their admission processes to recognize these factors, but there has been no coordinated movement among them to acknowledge the issues at hand. Despite knowing that privileged students have access to test prep and that less privileged end up losing significant class time to it, or that scores can be easily mapped to show economic disparities, they have not taken a stand on principle against testing. Each institution is seen as making a calculated move in its own interests instead of making a larger statement.

At the same time, however, it appears that the greatest growth in terms of college staffing is in areas like student academic support systems, pre-freshman summer programs to beef up accepted students’ writing and mathematic abilities, and other activities necessary to do what high schools did not. Test prep hollows out the curriculum, destroying what it is meant to measure. But colleges have helped create this issue by not addressing testing in any significant way. It’s easier to patch things up after the fact than to meet the problem head on.

Colleges could alleviate this situation, or at least address it, by coming out with a unified statement lowering the stakes involved in admission testing. One way would be to repudiate it entirely, although this is highly unlikely. However, as Atkinson suggests, a greater reliance on actual achievement would shift the balance back to substance: reading, writing, math, and so on. Atkinson makes the case for the SAT II based on a study done at UC:

The study examined the effectiveness of high school grades and various combinations of SAT I and SAT II scores in predicting success in college. In brief, the study shows that the SAT II is a far better predictor of college grades than the SAT I. The combination of high school grades and the three SAT IIs account for 22.2% of the variance in first-year college grades. When the SAT I is added to the combination of high school grades and the SAT IIs, the explained variance increases from 22.2% to 22.3%, a trivial increment. The data indicate that the predictive validity of the SAT II is much less affected by differences in socio-economic background than is the SAT I.                                        After controlling for family income and parents’ education, the predictive power of the SAT II is undiminished, whereas the relationship between SAT I scores and UC grades virtually disappears. The SAT II is not only a better predictor, but also a fairer test insofar as it is demonstrably less sensitive than the SAT I to differences in family income and parents’ education.

He makes a strong case for some kind of admission testing that is more directly related to students’ actual high school work. The SAT II (and the similar ACT) might fill that role in an expanded and more equitable version.

Nonetheless, colleges and universities could do more to help schools by making it clear that they want to see more students with better writing and math skills, better reasoning and interpretive abilities, and so on. They could insist on submission of a graded paper from a class or even require an actual written timed essay instead of a multiple choice test. (Perhaps a national essay test in the morning and a math test in the afternoon.) If “critical thinking” is really that important, there’s no defense for a multiple-choice test.

Making the case for real achievement in a unified way would help schools petition for more resources: If colleges required more evidence of substantive academic work in the admission process, cities and towns would have no choice but to make sure their schools were able to provide the kind of teaching and opportunities that would help students, particularly under-resourced ones, get the educations they deserve. Time now used for test prep would have to be used to teach math and writing, not test strategies or vocabulary lists.

Culturally, we are addicted to the shortcut, always looking for the pill that takes off pounds and the easy way to do something that is intrinsically hard. Standardized testing is college admission’s shortcut, the miracle measurement that lets us off the hook when we have to evaluate complex human beings still developing their personalities. No one suggests there is a perfect way to do college admission, but when considered in the context of the American education stem, colleges could do a lot more to make it support, rather than undermine, our students’ educations.

Atkinson Citation: Invited address at the annual meeting of the American Educational Research Association, San Diego (4/14/04). Subsequently published in: A Journal of the Association for Psychological Science, Observer, Vol. 18, 15-22, 2005.

One thought on “Achievement and Standardized Testing

  1. Pingback: Thursday Links | COUNSELOR EVENTS

Comments are closed.