No intellectual characteristic is too ineffable for assessment. Some schools use lengthy surveys like the California Critical Thinking Disposition Inventory, which claims to test for qualities like “truthseeking” and “analyticity.” The Global Perspective Inventory, administered and sold by Iowa State University, asks students to rate their agreement with statements like “I do not feel threatened emotionally when presented with multiple perspectives” and scores them on metrics like the “intrapersonal affect scale.”
Surveys can’t tell you everything. So universities assemble committees of faculty members, arm them with rubrics and assign them piles of student essays culled from across the school (often called “student products,” as if they are tubes of undergraduate Soylent Green). Assessment has invaded the classroom, too: On many campuses, professors must include a list of skills-based “learning outcomes” on every syllabus and assess them throughout the semester.
All this assessing requires a lot of labor, time and cash. Yet even its proponents have struggled to produce much evidence — beyond occasional anecdotes — that it improves student learning. “I think assessment practices are ripe for re-examining,” said David Eubanks, assistant vice president for assessment and institutional effectiveness at Furman University in Greenville, S.C., who has worked in assessment for years and now speaks out about its problems. “It has forced academic departments to use data that’s not very good,” he added. “And the process of getting this data that’s not very good can be very painful.”
The push to quantify undergraduate learning is about a century old, but the movement really took off in the 1980s. The assessment boom coincided — not, I think, by accident — with the decision of state legislatures all over the country to reduce spending on public universities and other social services. That divestment continued, moving more of the cost of higher education onto students. (These students are often graduates of underfunded high schools that can’t prepare them for college in the first place.) It was politically convenient to hold universities accountable for all this, rather than to scrutinize neoliberal austerity measures.
In 2006, the Commission on the Future of Higher Education, convened by Margaret Spellings, the secretary of education at the time, issued a scathing critique of universities. “Employers report repeatedly that many new graduates they hire are not prepared to work, lacking the critical thinking, writing and problem-solving skills needed in today’s workplaces,” the commission’s report complained.
Educators scrambled to ensure that students graduate with these skills — and to prove it with data. The obsession with testing that dominates primary education invaded universities, bringing with it a large support staff. Here is the first irony of learning assessment: Faced with outrage over the high cost of higher education, universities responded by encouraging expensive administrative bloat.
Many of the professionals who work in learning assessment are former faculty members who care deeply about access to quality education. Pat Hutchings, a senior scholar at the National Institute for Learning Outcomes Assessment (and former English professor), told me: “Good assessment begins with real, genuine questions that educators have about their students, and right now for many educators those are questions about equity. We’re doing pretty well with 18- to 22-year-olds from upper-middle-class families, but what about — well, fill in the blank.”
It seems that the pressure to assess student learning outcomes has grown most quickly at poorly funded regional universities that have absorbed a large proportion of financially disadvantaged students, where profound deficits in preparation and resources hamper achievement. Research indicates that the more selective a university, the less likely it is to embrace assessment. Learning outcomes assessment has become one way to answer the question, “If you get unprepared students in your class and they don’t do well, how does that get explained?” Mr. Eubanks at Furman University told me.
When Erik Gilbert, a professor of history at Arkansas State University, reached the end of his World Civilization course last fall, he dutifully imposed the required assessment: an extra question on the final exam that asked students to read a document about Samurai culture and answer questions using knowledge of Japanese history. Yet his course focused on “cross-cultural connections, trade, travel, empire, migration and bigger-scale questions, rather than area studies,” Mr. Gilbert told me. His students had not studied Japanese domestic history. “We do it this way because it satisfies what the assessment office wants, not because it addresses concerns that we as a department have.”
Mr. Gilbert became an outspoken assessment skeptic after years of watching the process fail to capture what happens in his classes — and seeing it miss the real reasons students struggle. “Maybe all your students have full-time jobs, but that’s something you can’t fix, even though that’s really the core problem,” he said. “Instead, you’re expected to find some small problem, like students don’t understand historical chronology, so you might add a reading to address that. You’re supposed to make something up every semester, then write up a narrative” explaining your solution to administrators.
Here is the second irony: Learning assessment has not spurred discussion of the deep structural problems that send so many students to college unprepared to succeed. Instead, it lets politicians and accreditors ignore these problems as long as bureaucratic mechanisms appear to be holding someone — usually a professor — accountable for student performance.
All professors could benefit from serious conversations about what is and is not working in their classes. But instead they end up preoccupied with feeding the bureaucratic beast. “It’s a bit like the old Soviet Union. You speak two languages,” said Frank Furedi, an emeritus professor of sociology at the University of Kent in Britain, which has a booming assessment culture. “You do a performance for the sake of the auditors, but in reality, you carry on.”
Yet bureaucratic jargon subtly shapes the expectations of students and teachers alike. On the first day of class, my colleagues and I — especially in the humanities, where professors are perpetually anxious about falling enrollment — find ourselves rattling off the skills our courses offer (“Critical thinking! Clear writing!”), hyping our products like Apple Store clerks.
I teach intellectual history. Of course that includes skills: learning to read a historical source, interpret evidence and build an argument. But cultivating historical consciousness is more than that: It means helping students immerse themselves in a body of knowledge, question assumptions about memory and orient themselves toward current events in a new way.
If we describe college courses as mainly delivery mechanisms for skills to please a future employer, if we imply that history, literature and linguistics are more or less interchangeable “content” that convey the same mental tools, we oversimplify the intellectual complexity that makes a university education worthwhile in the first place. We end up using the language of the capitalist marketplace and speak to our students as customers rather than fellow thinkers. They deserve better.
“When kids come from backgrounds where they’re the first in their families to go to college, we have to take them seriously, and not flatter them and give them third-rate ideas,” Mr. Furedi told me. “They need to be challenged and inspired by the idea of our disciplines.” Assessment culture is dumbing down universities, he said: “One of the horrible things is that many universities think that giving access to nontraditional students means turning a university into a high school. That’s not giving them access to higher education.”
Here is the third irony: The value of universities to a capitalist society depends on their ability to resist capitalism, to carve out space for intellectual endeavors that don’t have obvious metrics or market value.
Consider that holy grail of learning outcomes, critical thinking — what the philosopher John Dewey called the ability “to maintain the state of doubt and to carry on systematic and protracted inquiry.” Teaching it is not a cheap or efficient process. It does not come from trying to educate the most students at the lowest possible cost or from emphasizing short, quantifiable, standardized assignments at the expense of meandering, creative and difficult investigation.
Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning.
That’s how we produce the critical thinkers American employers want to hire. And there’s just no app for that.
Molly Worthen (@MollyWorthen) is the author, most recently, of “Apostles of Reason: The Crisis of Authority in American Evangelicalism,” an assistant professor of history at the University of North Carolina, Chapel Hill, and a contributing opinion writer.
