Moss, Pamela A.; Pullin, Diana C.; Gee, James P.; Haertel, Edward H. & Young, Lauren J. (Eds.). (2008). Assessment, Equity, and Opportunity to Learn. Reviewed by Orlena Broomes & Ruth A. Childs, University of Toronto

Moss, Pamela A.; Pullin, Diana C.; Gee, James P.; Haertel, Edward H. & Young, Lauren J. (Eds.). (2008). Assessment, Equity, and Opportunity to Learn. New York: Cambridge University Press Pp. xv + 364         ISBN 978-0-521-70659-9 Reviewed by Orlena Broomes & Ruth A. Childs Ontario Institute for Studies in Education, University of Toronto April 23, 2009 Our School of Education occupies a 12-story building. Within that building, the educational sociologists are on the 12th floor, the curriculum experts on the 10th and 11th, the developmental and school psychologists on the 9th, and the educational administration and policy folks on the 6th. If not for the shared elevators, we could go weeks without seeing each other. When, each spring, we gather in the library for a showcase of the books and articles published in the past year, one can hear exclamations of “I didn’t know you were interested in that, too – we must talk sometime” or “I’m still hoping we can find time to talk.” This situation is not unique to the University of Toronto (or to education), which makes this book the more remarkable. It is the result of a sustained conversation and collaboration among psychometricians, sociologists, cognitive scientists, and policy experts. The idea that led to this book originated in a conversation between Pamela Moss and Diana Pullin in a Montreal coffee shop during the 1999 American Educational Research Association conference. With Edward Haertel and James Paul Gee, they obtained funding from the Spencer Foundation for a project they called “The Idea of Testing: Psychometric and Sociocultural Perspectives.” They invited King Beach, James Greeno, Carol D. Lee, Hugh Mehan, Robert Mislevy, Fritz Mosher, and Lauren Jones Young to join them for a series of discussions. This book is the result. The book is organized as a series of chapters authored by different group members and presenting different perspectives on learning, opportunity to learn, and the role of assessment, bracketed by chapters providing a context (Chapter 1) and a synthesis (Chapter 12). It is not a defense of testing: In Chapter 1, Holland, Moss, Pullin, and Gee write, “By and large, the authors are skeptical about the benefits of contemporary accountability testing” (p. 3), including the belief underlying the U.S.’s No Child Left Behind act, that test-based accountability will force schools to bring all students up to a proficient level of achievement. However, lest the reader expect yet another book cataloguing the problems with large-scale assessments, they go on to say: [M]ost of the authors represented here view these concerns and criticisms as almost beside the point. These sorts of problems are serious but in principle remediable. Our book interrogates instead the fundamental premise that schooling is about imparting skills and that [opportunity to learn] is simply a matter of ensuring universal skills acquisition. (p. 3) The main point, explored from different perspectives throughout the book, is that schooling is about much more than teaching facts and skills (what Gee calls the “content fetish”) and that this has implications for assessment and opportunity to learn. As Gee writes in Chapter 8, “Any actual domain of knowledge, academic or not, is first and foremost a set of activities (special ways of acting and interacting so as to produce and use knowledge) and experiences (special ways of seeing, valuing, and being in the world)” (p. 200). In an earlier chapter (Chapter 4), Gee, to describe the process of learning, uses the metaphor of simulations: Children construct mental simulations of the world within which they can try out possible actions and explore the relationships among things and people. This metaphor is particularly compelling when we consider opportunity to learn: If some children have had prior experiences that have enabled them to build more elaborated simulations or if the way in which new information is being presented is more compatible with their already existing simulations, then they will be able to derive more benefit from new learning opportunities. Teachers may tend to assume that children’s mental simulations are like their own and may fail to recognize simulations based on different experiences. To be effective, teachers will need to understand the mental simulations children already have. There are several directions in which discussions about differences among children can proceed. One is to assume that students who are not performing well in school have deficits of ability or experience; based on this assumption, educators try to identify the deficits and provide extra opportunities to remediate those deficits, so that these students can better benefit from the opportunities that schools provide. Another is to assume that schools are not providing the right kinds of opportunities, or are not presenting the opportunities in the right way for some students. Educators then must figure out how to change the opportunities to fit these students, often amid worries that they will hurt the currently successful students in the process, either by lowering expectations or by making the instruction less well-matched to those students. A third – and the view taken by most of the authors of this book – is that “we know much more about how to make educational institutions far more effective for all students than current teaching and testing practices suggest” (p. 338, italics in the original). Pullin, in the concluding chapter (Chapter 12), continues, “The sociological and situative research and perspectives described throughout this volume offer a more complex and nuanced understanding of the teaching and learning process than is represented in many current education policies and practices” (p. 338). In particular, throughout the volume, learning is discussed as something that happens not simply within individuals, but in an activity system. Gee, in Chapter 4, defines an activity system as “a group ... of actors who have a common object or goal of activity” (pp. 89-90). The activity system in a school includes all the students, however successful or not they have been when provided with traditional opportunities to learn, and also, as Moss, Girard, and Greeno make clear in Chapter 11, all the adults in the school. In summarizing the main thrust of three school interventions that they see as consistent with this view, Moss, Girard, and Greeno write, “everyone in the district shares the responsibility for successful learning by others as well as by themselves” (p. 328). So, if learning is about constructing complex but flexible mental simulations and requires collaboration among a group of learners, including teachers, then what is the role (if any) of large-scale assessment? According to Moss, Girard, and Greeno, large-scale assessment can be viewed as a boundary object – that is, “an object that inhabits multiple heterogeneous social worlds [or activity systems] and enables communication and cooperation across these worlds” (p. 300, brackets in the original). To truly understand the uses (intended and not) of large-scale assessments, ideally and eventually, one would want to analyze ... the activity systems through which the assessment was conceptualized, developed, mandated, and implemented; the school and classroom activity systems in which it is responded to, interpreted, and used; the activity systems involving administrators and policy makers at the district, state, and national levels; the activity systems of students’ families and peer groups; the activity systems of professional organizations and teacher education institutions that attend to such information; and the “virtual” activity systems of members of the public who attend to evidence about how their educational systems are functioning. (p. 301) That is not to imply that tests are necessarily benign. In the preface, Mosher and Young summarize the worry about assessments that reappears throughout this book: Current assessments seem to focus primarily on coverage of subject matter content and basic skills. It may be that successful performance on these assessments also requires other aptitudes and capabilities, but if teachers ... think that their main focus is on discrete facts and skills, the tests may “drive” instruction to concentrate on just those things. However, if real proficiency requires students to have additional skills, dispositions, and aptitudes, tests of this sort can deprive them ... of the opportunity to gain this necessary knowledge.... (p. ix) In other words, large-scale assessments may decrease all students’ opportunities to learn what is really valuable. That is not to say that assessments cannot have value. Pullin, in Chapter 12 describes the requirements: “Assessment is productive only when it is deeply embedded in learning activity systems to both continuously test deep conceptual understanding and allow all students, no matter their backgrounds or capabilities, to situate their learning in meaningful ways” (p. 336). Assessments must include not only what the students know, or can do, or how they engage in reasoning and problem solving, but also what they mean when they act and interact. Assessments must explore how they use domain specific-language, representations, and culture, and how they participate in the activities of their local communities, including school. What of equity and opportunity to learn? In Chapter 1, Haertel, Moss, Pullin, and Gee emphasize that “Opportunities to learn do not exist for learners who cannot take advantage of them” (p. 6). This means that, in rethinking how learning happens in schools, educators must consider students’ prior knowledge. That insight is hardly profound. More important is what the authors mean by prior knowledge. Returning to the metaphor of the mental simulation, all children are constantly constructing mental simulations. Some children will enter school with mental simulations that are more similar to those of the teacher; children will also differ in how elaborated their simulations are. One of the dangers is that teachers may not recognize experiences and the resulting mental simulations that are not similar to their own, and so may not build on them. As Pullin writes, OTL and learning in school are not only a function of the school, educators, and students, but also of the organizations and identities historically and developmentally linked with, and beyond, the school. For example, the workplace of students with out-of-school jobs as well as after-school activities, both organized and informal, impact OTL, either diminishing chances for meaningful access to schooling or, conversely, providing richer stimuli for personal growth than is afforded in schools. (p. 341) Equity, for the authors of this book, means providing every student opportunities to learn that acknowledge what they already know, and are meaningfully related to what they already know, and what they will need to know “to effectively participate as a lifelong learner in the practices of modern society” (p. 45). The above summary does not do the breadth and depth of the book justice. The individual chapters are rich with analyses, examples, and insights. Chapters 2, 3, and 4 examine assessment through the lens of opportunity to learn and discuss the theories grounded in sociological perspectives and in a sociocultural perspective. Chapter 5 provides concrete examples of opportunities to learn for students with disabilities. The middle chapters focus on ways of making learning equitable for all students. These chapters discuss making problem-solving explicit in culturally robust classrooms and the implications for assessment. They discuss opportunities to learn in practice, and identify and examine how games can be used to support learning and the implications for opportunity to learn. The sociocultural implications of assessment are further explored in two chapters: Chapter 9 applies a sociocultural perspective to classroom assessment, while Chapter 11 examines the implications of this perspective for schools and school systems. Chapter 10 connects to a view of assessment as constructing argruments about students’ knowledge and skill. In the final chapter, Pullin synthesizes what has come before and offers 24 principles. For example: Content learning in school requires learning new forms of language and the identities, values, content, and characteristic activities connected to these forms of language (e.g., the language of literary criticism or of experimental biology). Every learner has the right for these “new cultures” to be introduced in ways that respect and build on the learner’s other cultures and indigenous knowledge, including his or her home-based vernacular and peer-based and “popular culture” cultures (“discourses”). (pp. 346-347) As an emerging scholar whose research is primarily in the area of sociology and sociocultural theory with a secondary but strong focus on measurement and assessment (the first author) and a psychometrician with strong interests in the effects of assessment (the second author), we believe this book will challenge and inspire new thinking by both sociologists and psychometricians. Too often, assessment is used to enforce a narrow, fixed view of what should be learned and how. Students whose experiences differ from those of the teachers may find those experiences and the understandings they have built ignored. By addressing the role of cultural capital, social capital, language and other aspects of the learner’s environment as integral components in supporting learners and their opportunity to learn, this book provides a valuable resource for sociologists and psychometricians alike, as well as educators and educational policy makers. As the above suggests, we found an enormous amount to like about this book. However, one chapter seems discordant with the rest: Hugh Mehan’s “A Sociological Perspective on Opportunity to Learn” (Chapter 3). Mehan argues that it is necessary in traditional classrooms for students to have the "right kind" of social capital. Many readers will disagree with his suggestion that, in a democratic society, students are more or less responsible for their own success or lack of success. Mehan does not propose that successful schooling involves classrooms adapting to make use of the students’ environments outside of school, but rather stipulates that the working class and minority students (in short, those whose needs are not being addressed by their school), can nonetheless be successful in school and “enjoy some of the same advantages that accrue to their more privileged peers” (p. 68) if they so desire. Mehan provides examples of how, with additional help, students can better fit into the existing educational structures and benefit from traditional learning opportunities. For example, in the third intervention he describes, “the schools and their collective agents act in a deliberate, intensive and explicit fashion to generate a socialization process that produces the same sort of strategies and resources deployed in privileged homes and institutions” (p. 68). This approach is clearly at odds with the vision presented in the rest of the book. Throughout, the book is written in very straightforward and simple language devoid of the technical jargon and formulas that typify much psychometric writing and of the sometimes convoluted, dense theoretical language often found in sociological studies. We strongly recommend it not only to academics and educators, but to anyone who thinks about – and worries about – education. About the Reviewers Orlena Broomes is doctoral candidate in Sociology and Equity Studies in Education. Her research interests include using sociological and psychometric perspectives to explore questions related to large-scale and classroom assessments and the educational outcomes of African Canadian students. Her current research examines factors that predict African Canadian students’ resilience and success in school despite the systemic and institutionalized barriers that they face. The lack of data about African Canadian students continues to be a challenge. Ruth Childs is an associate professor in Human Development and Applied Psychology. Her research focuses on practical psychometric issues that arise in large-scale assessments, such as the advantages and disadvantages of matrix sampling, alternative approaches to scoring tests, and the impact of missing data treatments. She is currently investigating how teachers decide whether to comply with test preparation and test administration guidelines and how equity is considered in teacher education program admissions.