Beyer, Catherine Hoffman; Gillmore, Gerald M., & Fisher, Andrew T. (2007). Inside the Undergraduate Experience: The University of Washington's Study of Undergraduate Learning. Reviewed by James Soto Antony, University of Washington

Beyer, Catherine Hoffman; Gillmore, Gerald M., & Fisher, Andrew T. (2007). Inside the Undergraduate Experience: The University of Washington’s Study of Undergraduate Learning. Bolton, MA: Anker Press. Pp. 432     $40     ISBN 978-933371-26-9 Reviewed by James Soto Antony University of Washington March 3, 2008 In the perennial conversation surrounding higher education accountability, 2007 will be remembered as a year when colleges and universities were vigorously pushed to consider how to measure student-learning outcomes by the Secretary of Education’s 2007 “Spellings Report”, which called for required data reporting including standardized test results. Though history will tell whether this push, instigated by the Spellings Report, will actually lead to real change one thing is certain: higher education is on notice. Institutions must continue to think strategically about how to assess student learning. Accrediting agencies are increasingly suggesting that institutions make progress toward developing meaningful assessment strategies, with many institutions having received warnings in their accreditation assessments for not adequately addressing the need for meaningful measures of student learning. And the federal government has signaled its interest in the issue in ways that can force institutions to measure student learning on ways that might not be in the best interests of all institutions. The near hysteria stemming from the “Spellings Report” of 2007 led institutions to protest increased reporting requirements, commonly referred to as “IPEDS on steroids.” These efforts at changing the federal government’s understanding of the complexities involved in measuring student learning, and in higher education’s interest in finding a good way to do so, were largely successful in getting the government to soften its approach. Yet the expectation of better student learning assessment has, in fact, always existed. Many colleges and universities have engaged in assessment activities since the late 1980s and, accrediting bodies have asked colleges and universities to report on assessment for more than 10 years—including assessment of student learning. The bottom line here is that assessment of student learning is not a new issue, and the expectation that institutions will find a way to do this will not go away. In response to the climate surrounding student learning assessment, the Association of American Colleges and Universities, in partnership with the National Association of State Land-Grant Universities and Colleges, has pushed a new initiative aimed at creating a Voluntary System of Accountability (VSA) that colleges and universities can adopt as a way of increasing their responses to the call for better student learning assessment. Essentially, the VSA has set forth several possible approaches, and a suggested reporting template, that give institutions choices about what to use when measuring outcomes on their campuses. Though in its infancy, the VSA promises to change the conversation because colleges and universities have come together to define—in a collective fashion—the outcomes worth measuring, and the ways to measure them. They have done this by collaborating with testing agencies, and other research centers and national survey efforts, to identify how existing measurement approaches might be leveraged so that higher education can immediately begin to chip away at the challenges surrounding learning assessment. Though a legitimate argument could be raised as to whether these efforts are, in fact, good representations of best practice in the assessment of student learning. In the process of doing all of this, higher education is confirming a realization it has long held. Namely, off-the-shelf approaches are not adequate in measuring student learning, and institution-specific cultures matter when attempting to accurately define what students learn. This last point is crucial because it illuminates the challenges inherent in designing common systems of accountability that might be used to compare institutions. That is, institutional missions and their consequent outcomes are not always the same and, even when they are (in name), they cannot always be compared in a straightforward manner because of institutional differences in student selectivity, institutional size, and a whole host of other factors. In short, what VSA has done well is stimulated the idea that the measurement of student learning should be driven by the institutions themselves. VSA offers a set of tools—an a la carte menu, of sorts—from which institutions can select those that help tell their student-learning story. But everyone acknowledges that even this toolkit is not enough and, as said above, may employ measures or instruments that never were intended to, and therefore, come up short on the assessment of what students learn. All of this conversation has led many to suggest additional, and alternative approaches, to assessing student learning. These approaches would not negate the contributions that the VSA promise to make, but would build upon the VSA approach (and other similar approaches) by adopting more holistic forms of assessment that higher education has long known, but resisted for a variety of reasons—portfolio-driven assessment. As if almost by design, it is in this climate that the book, Inside the Undergraduate Experience: The University of Washington’s Study of Undergraduate Learning (Beyer, Gillmore, and Fisher, 2007) has been published. As I intend to outline in this review, this book makes a major contribution in our understanding of not only how to best measure student learning (i.e., using a multiple trait, multiple method approach), but what we find out when we do a good assessment of student learning. Throughout this review, with the authors’ permission, I have liberally restated much of what was written in the book in an attempt to accurately describe what was a complex and rich study, and to properly describe the book’s major findings. The book is based upon findings from the University of Washington’s Study of Undergraduate Learning (UW SOUL). UW SOUL was a four-year study conducted from fall 1999 to spring 2003. The study longitudinally tracked 304 students as they moved through their college experience and focused on six areas of their learning: writing, critical thinking/problem solving, quantitative reasoning, information literacy, understanding and appreciating diversity, and personal growth. The purpose of this book is to share with faculty, students, parents, and others what the UW SOUL students told us about their learning through four years of interviews, surveys, focus groups, and submissions of coursework. UW SOUL had several purposes. First, the study aimed to identify what students learned and where they learned it in their undergraduate programs. Because their focus was assessment, the authors also wanted to identify what helped students learn, as well as the obstacles or challenges to learning that they faced. Third, like most undergraduates attending large research universities, University of Washington (UW) students are rarely asked to evaluate their own experience. Therefore, the study was designed to ascertain what students would say when asked to do this kind of reflection. Fourth, the authors wanted to learn what about students’ personal development and the role that the university experience played in that growth. All of this was done with the overarching goal of keeping together a group of students whose opinions on UW initiatives or current issues could be polled. In addition to these purposes, the study was created to maintain enough flexibility in its design so it could respond to ideas, questions, and directions set by faculty and staff at the UW. If a faculty member, for example, had a question that fell within the study purposes, the authors were able to do this, asking study participants questions that came from UW faculty, University Regents, and department chairs, as well as from faculty at community colleges. The UW SOUL allowed the authors to see students’ experiences in two ways: close-up, focused in tightly on individuals and from a distance, giving an aggregate view of the group. These two views gave the authors the opportunity to examine how students grow and struggle, how they display amazing insights at the same time that they may be repeating previous mistakes, and how they learn about themselves and their world, even from thin sources. There Are Significant Gaps between High School and College Learning Experiences Gaps between high school and college experiences were most frequently noted in students’ quantitative reasoning and writing experience, but they were mentioned in other areas, as well. Such gaps remind us that 100- and 200-level courses must teach explicitly into the gap so that students do not get permanently lost there. In terms of quantitative thinking, the gap that students reported was between “plug and chug” equations and using quantitative methods to think and analyze in a wide range of settings. In terms of writing, students reported the gap was between writing for English courses in high school and writing arguments in other academic disciplines in college. All Academic Learning is Mediated by the Disciplines As UW SOUL findings on critical thinking, writing, quantitative reasoning, information technology and literacy, and understanding and appreciating diversity made clear, learning in college is mediated by the disciplinary context in which it occurs. This was not only true for learning in the major, but it was also true for the courses identified as “general education.” Students writing papers for 100-level History courses were expected to adhere to argumentative practices and critical thinking strategies that differed from the papers they wrote for 100-level Philosophy or 100-level Chemistry classes. So pronounced was this reality that when students completed their majors they identified critical thinking with the practices of the disciplines in which they had been immersed for two or more years. Responses of UW SOUL students also suggested that students who came to the UW were not prepared for disciplinarity. Those who came to college directly from high school had not fully experienced the disciplinary nature of knowledge and knowledge creation, and many students coming to the university from community colleges had little experience with disciplinarity, as well. However, UW SOUL transfer students from community colleges often entered directly into majors, where they were immediately immersed in the language and thinking practices of those fields. In contrast, students entering the UW from high schools, often floundered in their first two years, because they were unaware that movement from courses in one discipline to those in another was a move from one set of practices to another. This lack of awareness was exacerbated by the fact that faculty are rarely explicit about disciplinary practices and purposes in courses at the 100- and 200-levels. Learning about Others and Oneself Is a Central Part of the College Experience and May Also Be Affected by the Disciplines of Students’ Majors College changes students. Almost without exception, undergraduates learned who they were, what they hoped to achieve, what they believed, and how they might accomplish their goals. The impetus for these changes did not reside only in the classroom, but it was found in every corner of students’ experience. Often such change was the result of interaction between classroom learning and outside experience. The contribution of study abroad, undergraduate research, internship, service learning, volunteer, and paid work experiences to students’ self-knowledge, self-confidence, and self-awareness cannot be overstated. These opportunities, perhaps because they are coupled with classroom learning that is often challenging students’ thinking, sense of themselves, and understanding of others at the same time, helped students meet their own goals for self-understanding, as well as their goals for better understanding of other people. Another significant contributor to personal growth was self-reflection, as seen in students’ responses to questions about the study’s impact on their learning. And a third contributor may be a students’ major. The authors speculated that students in majors that engaged them in ethical questions, social issues, conversations about diversity, and conflicting viewpoints may foster personal growth more than majors that see themselves as primarily transmitting a body of knowledge and practices. No evidence was found in the UW SOUL that student growth was linear or staged. Furthermore, there was no evidence that personal or intellectual growth was a consistent response to new challenges in which students experienced instability. While often such challenges fostered change, they also fostered withdrawal and retreat. Students’ responses to such challenges depended a great deal upon the contexts in which they occurred—for example, whether students’ felt safe to fail—as well as on the students’ interests and backgrounds. However, the many opportunities that a large research institution located near a metropolitan area offered for exploration, particularly when coupled with reinforcing classroom learning, seemed consistently to foster growth and move students closer to becoming their adult selves. Students Want to Be Intellectually Challenged While challenges that represented too great a leap or—perhaps more to the point—too great a leap without a net sometimes killed students’ interest in subjects, students said again and again that they wanted to be challenged. Many of them had easily succeeded in high school, and they came to the UW hoping that their courses and instructors would no longer permit them to cruise through their courses and assignments with little thinking or engagement. Often the majors students eventually selected coincided with the thinking they described as their most challenging in their first and second years at the UW, and students said in interviews, focus groups, and email that the classes that asked them to think were the ones in which they learned the most and did the best. They universally expressed contempt for classes that made it easy for them to do well in focus through all four years of the study. The key difference between challenges that students valued and those they did not was the support students felt they received for trying to meet such challenges. If they had support—for example, explicit instruction, access to faculty and teaching assistants, the ability to work with peers—then such challenges were exhilarating. If they felt they had no support—for example, if they were presented with a new challenge on a final exam—then they often felt set up to fail. Students’ Learning in College Comes from Many Sources and Their Interaction Student learning came from a wide range of sources, inside and outside the University. Students’ said that they learned the most from faculty and their peers, and it was clear that, often, faculty structured learning peer-to-peer learning experiences. However, students also learned about academics from experience, and they placed a great deal of value on hands-on, experiential learning of all kinds, from lab experiments to assignments requiring them to gather the real-life experiences of others. Hands-on experiences were considered valuable for what they contributed to academic learning and vice versa. The interaction of the two seemed to be key. Students Have Complex Definitions and Multiple Goals for Learning; To a Great Extent, They Meet Those Goals UW SOUL participants’ sense of what it meant to be educated was complex when they arrived at the University of Washington in 1999 and when the study ended in 2003. While they became more focused on their own academic goals usually at some point in their second year than they were on entry, they still retained complex definitions of what it meant to be successful in college. Furthermore, when they left the UW, they generally felt that they had met many of these goals, even though the focus of the University is academics. The authors believe that with increasing emphasis on active learning strategies, peer-learning groups, and student self-assessment—all approaches that research shows contribute to learning—the UW, along with other institutions in higher education, “plays into” students’ non-academic learning goals. In other words, by using methods that are the best for teaching students academic skills and content, faculty can advance students’ complex learning goals. About the Reviewer James Soto Antony is Professor and Director of the Graduate Program in Higher Education at the University of Washington. He also serves as Special Assistant to the Executive Vice Provost, focusing on issues of assessment and accreditation for the university.