Ryan, Katherine E. and Shepard, Lorrie A. (Eds.) (2008). The Future of Test-Based Educational Accountability. Reviewed by William L. Brown, Michigan Department of Education

Ryan, Katherine E. and Shepard, Lorrie A. (Eds.) (2008). The Future of Test-Based Educational Accountability. NY: Routledge Pp. 316         ISBN 978-0-8058-6470-0 Reviewed by William L. Brown Michigan Department of Education May 13, 2009 According to the preface, The Future of Test-Based Educational Accountability began as a tribute to Robert Linn on the occasion of his “nominal retirement.” The book includes chapters on the context, technical and substantive issues, effects, and future directions of educational accountability, written by a veritable “Who’s Who” in the field of educational measurement. These chapters are grouped into four parts. Part I includes three chapters that consider the context of educational accountability. For those who might be anticipating the demise of test-based accountability, the authors do not see an end to it soon. In fact, a large number of accountability testing programs have been introduced over the past century. These programs reflect the beliefs of politicians, policymakers, and the business community that educational achievement is inadequate, and that some form of outcomes measurement is necessary to prod teachers and school administrators to become more effective. Part II has seven chapters that review the technical and substantive issues of educational accountability. Among the issues discussed are reliability, equating and linking, standard setting, measurement of growth, and fairness. Since the stakes involved in school accountability are high (staff can be reassigned or even lose their jobs), these technical issues are very important. The authors rightly point out that it is problematic to compare two different cohorts of students from two different years (e.g., this year’s fourth graders with last year’s fourth graders), since there is no assurance that the cohorts are in any way comparable. Value added models have been created to address this situation, but they present their own technical issues, some of which are addressed in a chapter on “Causes and Effects.” Part III consists of four chapters dealing with the effects of educational accountability. The chapter entitled “Is Public Interest in K-12 Education Being Served?” begins with an attempt to define the meaning of “public interest.” The very term “no child” indicates that the drafters of No Child Left Behind viewed the public interest as having schools address the needs of all students, bringing every student up to a level deemed as “proficient” by 2014. On the other hand, mere proficiency does not ensure that society will have the highly skilled citizens who can meet the demands of a technologically complex society. “Even with the most optimal system,” writes author Joan Herman, “there are limits to what accountability alone can accomplish.” In a chapter described as “A View from the Teacher Trenches,” concern is expressed that “NCLB champions view teachers with suspicion, if not downright contempt.” The late Al Shanker, AFT President, had embraced the concept of standards based accountability, thinking that it would help secure the future of public education in the face of the advance of private education. “It was time, the union insisted, for politicians and the public ‘to cease acting as though the critics of despair from our poor districts were mere noise and teachers’ advocacy on behalf of their students were mere self-interest.’” Shortly thereafter, in 1990 there emerged a set of eight national educational goals, including the performance standards of Goal 3 – modest compared with the achievement goals of NCLB – calling for improvement in academic performance of students in every quartile and the reduction of the gaps between minority and majority students. Part IV deals with future directions for educational accountability. Eva Baker addresses learning and assessment in an accountability context. Performance assessment has a long history in business, as well as in the military, higher education and as a tool for formative assessment in classrooms. In large-scale assessment, however, early attempts ran into problems of feasibility, cost, technical quality and conceptual sloppiness. This resulted in a loss of public credibility. She describes POWERSOURCE© – a tool that uses the power of examples and student explanations to assess pre-algebra and algebra learning. This tool shows promise as an improved way of assessing student performance on a large scale. The concluding chapter, by Michael Feuer, is subtitled “Notes for a Political Economy of Measurement.” He reasons that there is no test-based accountability system that maximizes a broad set of conditions simultaneously. For example, domain robustness and minimal burden are opposing qualities. Limited budgets and testing time prevent authentic criterion-based representation of complex cognitive functioning based on efficient domain sampling. Eva Baker went so far as to call “specious” the use of correlations between selected-response tests and learning-focused performance assessments to argue for the briefer, less costly standardized multiple-choice formats. As a coordinator for test development in a state which tests nearly a million students annually in multiple content areas, it is easy for me to agree that the limitations of time and funds place severe restrictions on the ability to construct reliable and valid measures of the full range of student performance. The necessity of producing measures and reporting useful results in short timeframes leads to searching for shortcuts to “get the job done” without violating the ethics of assessment that have been agreed upon by the measurement community. Overall, The Future of Test-Based Educational Accountability does an admirable job of distilling the knowledge of many of the nation’s best thinkers in educational measurement, both policy and practice. There are, however, flaws that surfaced because of inadequate editing of the papers that comprise the contents of the book. For example, on page 11 the word “interpretating” occurs. On page 22, it mentions the “polices” of NCER. On page 97, it is stated that 0.13 is five times greater than 0.060. On page 202 we read that “There is evidence that lower performance districts having less access to qualified teachers.” An acknowledgement on page 244 refers to the book’s co-editor Lorrie Shepard as “Lorrie Shepherd.” Although this book may not be the final word on the future of test-based educational accountability (only the future will tell), it does provide a valuable compendium of the best thinking of a group of exceptional leaders who have contributed greatly to educational assessment over the past several decades. Those of us in the trenches of large-scale assessment often wonder if we are contributing to the success of education or to its demise. This book gives us some assurance that our motives have been good. About the Reviewer William L. Brown is Coordinator of Test Development for the Michigan Department of Education’s Office of Educational Assessment and Accountability. He is responsible for developing K-12 assessments in the four basic content areas as required by Michigan Law and the No Child Left Behind Act. He received his doctoral degree in Measurement and Quantitative Methods from Michigan State University in 2003. Email: brownb6@michigan.gov.