A Barrel of Fish from the College Board

This is really too easy but I’m going to go with it anyway as a way to wake up from a winter’s snooze:

My pal Eric Hoover over at the Chronicle of Higher Ed (check out his blog there) sent me a link to a College Board “report” originally released last fall but just now being publicized. In it, the College Board reveals that it has apparently found the only 900 people in the United States who don’t find the college application process to be complicated or stress-inducing. It tells us that the 600 students and 300 parents it surveyed thought the whole thing was just ducky and not to worry so much. It’s called, in a non-sequiturish sort of way, “Complexity in College Admission: Fact or Urban Myth” (no question mark) and has a sub-heading of “Research Findings of Parent and Student Perceptions of Complexity in College Admission Commissioned by the Task Force on Admissions in the 21st Century.”

So we have a “report” commissioned by a “task force” for the “21st century,” which means we should pay attention, as if the great and powerful Oz has spoken. But the report is embarrassingly shallow and pointless, no doubt causing bitter laughter among the many thousands of students and families who are even now awaiting college admission letters after months of wrestling with essays, fighting with testing agencies to get their scores released to the proper colleges, trying to figure out which college wants which topic addressed where, deciphering the instructions for the FAFSA, and on and on. There’s really only a little accountant behind the curtain…

The stunning obtuseness of the “report” can be summed up in this bit from the executive summary (you didn’t know you were an executive, did you?):

Seventy percent of all high schools offered classes or seminars to students to help explain the college application process, although about one-third did not. These types of classes were even more common in the western region of the country. Of those studentswho attended such a class, most found it helpful in clarifying the college application process.

Well, that’s just fine. I’m sure we can all rest easy now. But did the College Board wizards ask why it was necessary for high schools to offer such a class in the first place? Huh? The rest of the ‘report” is similarly craptacular (thank you, Bart Simpson).

Here’s an excerpt from the press release that accompanies the “report:”

Overall, applying to college appears to be a clear and simple process for most students. Researchers found that 53 percent of students who completed a common application form said that it simplified the college admission process. Knowing how admission decisions were made was the most confusing aspect of the college application process. And, the more colleges that students applied to, the more stressful the experience was for them.

In the College’s Board’s estimation, 53 percent is “most students.” While technically true, those other 47 percent must be scratching their heads. But, right, the College Board forgot to ask them.

Here are some other observations about the “results,” which simply fly in the face of reality. Read the “report” if you must, but don’t say I didn’t warn you.

1. I’m no statistician but it seems they used a very tiny sample (600 students/300 parents) and even though the CB claims to have ensured a representative sample of income levels and first-gen status, I don’t see how. They also claim to have surveyed different “regions” but, come on. And they tout the number surveyed as being “nearly 1,000” people but you could just as easily say that 900 is “just over 800.” So, again, really???? With 3.33 million students graduating from high school in 2008-09 (a peak)? Of course not all of them applied to college, but even if only a third did, seems like a big leap…
2. Many categories of response in the survey don’t really seem to address the question being asked, like “How families make decisions regarding which colleges to apply to.” That’s not really an aspect of the college process, but a pre-existing set of circumstances. And a question like “How complicated was it for students to decide where to apply to college?” isn’t really about the process itself but has more to do with an individual’s background, family expectations, aspirations, etc.
3. There’s no definition of “complexity” against which to measure the responses or evaluate the questions so it’s anyone’s guess what the responses really measure. It seems to be used as a rough equivalent of “confusion” or “stress” but that’s not a very careful use of the concept. There are many things that are complex but not confusing (car engines, science, etc.) and confusing but not complex (why I always seem to get all the red lights when I’m driving down Western Ave., why fractions work, etc.).
4. The perception of “complexity” also depends on the sophistication of the participant, and there’s no baseline of the respondents’ experience with the college process more specific than income or first-gen status.
5.  One could make the case for saying, “Ha ha, all you people! Your confusion and frustration at the college process are false!” but those things can’t be true or false, they can only be perceived or not. At the same time, I guess the CB thinks it’s revealing something amazing and telling us we needn’t fear the Reaper or something. Small comfort to those for whom it’s a truly mystifying experience.
6. There’s no definition of the “college process” or “application process” that would serve as another kind of benchmark against which to measure responses. We all “know” what they’re talking about, but I don’t think any of it rises to the level of analysis a real-life person could understand or make use of.
7. Also plenty of “duh!” “findings” like “First generation students were more likely to apply to in-state colleges and public schools…” Really?? Gosh, I had no idea. And how this applies to complexity I don’t know. Do they do so because it’s too hard to apply elsewhere?
8. Knowing or not knowing how decisions are made isn’t really part of the “complexity” of the process; it’s simply a byproduct and a source of frustration after the fact. I don’t think most people worry about that before decisions are made or if they get accepted. Knowing or not knowing how my car engine works isn’t relevant until it breaks down on the Dan Ryan during rush hour. So there’s confusion about where confusion or complexity or frustration is located in the process.
9. “Fact or Urban Myth” (no ?) seems to me like asking “Do you walk to school or carry your lunch” (no ? just to be consistent). If you’re talking about people’s perceptions, you’re not talking about urban myths, you’re talking about, well, perceptions. An urban myth is “Baby alligators flushed down the toilet live in the sewers and can crawl through your pipes to bite you when you’re on the toilet.” That’s a very different concept from “I’m really afraid that I’ll be harmed by my plumbing.” The former is a non-rational belief that may or may not be true but is accepted as true and which contains superhuman or mystical elements; the latter can be a genuine fear that may have arisen through an early childhood experience or some real event that may have been blown out of proportion but is “real” to the speaker. A myth encompasses much more than simple misunderstandings of fact. If people were saying, “Using the Common Application causes your toes to fall off,” then we’d be in urban myth territory. But just saying it doesn’t cause toe-ectomies because a bunch of people still have theirs doesn’t disprove it.

OK, I told you this was too easy. Sorry for the sarcasm. But I can’t help wondering how people at the College Board can release “reports” like these with a straight face.