Jones, Gail; Jones, Brett; & Hargrove, Tracy. (2003). The Unintended Consequences of High-Stakes Testing. Reviewed by Bob Pletka, University of California, Irvine

Jones, Gail; Jones, Brett; & Hargrove, Tracy. (2003). The Unintended Consequences of High-Stakes Testing. Lanham, MD: Rowman & Littlefield. 192 pp. $18.95 (Paper) ISBN 0-7425-2627-5 $65.00 (Hardcover) ISBN 0-7425-2626-7 Reviewed by Bob Pletka University of California, Irvine March 26, 2005 In The Unintended Consequences of High-Stakes Testing, authors Jones, Jones, and Hargrove pose the question, “What are the effects of high stakes testing on teaching and learning in the k-12 environment?” This question is critical to educators, politicians and parents during a time in which the National Policy, No Child Left Behind (2001) uses high stakes testing as a critical component to its accountability measures. Those accountability measures carry significant consequences in the form of financial incentives and punishments on schools and educators. Even though political advocates of No child Left Behind such as, President George Bush say “We do not need trendy new theories or fancy experiments” (cited in Coles, G. 2001, Bush and “The Basics” section, para. 2.), research is needed to determine the effects of a national policy that reallocates large amounts of resources based on the scores from high stakes testing. Already, some research has emerged (Amrein & Berliner, 2002) that indicates high stakes testing may improve test scores but high stakes testing does little to increase student learning. The Unintended Consequences of High Stakes Testing provides further study that can inform educational stakeholders as they evaluate the effectiveness of this testing policy. The book shows both the intended and unintended consequences to high stakes testing and then leaves the reader to determine whether the outcomes benefit or detract from learning. In addition to the importance to the book’s question about how high stakes testing affects teaching and learning as it relates to national educational policy, the question is based on a more fundamental query posed by educational scholars about the foundations of learning. Is learning a voluntary behavior that is strengthened or weakened by consequences of punishments or rewards as Skinner (1978) suggests? Or, do learners build their own understanding of the world as constructivists argue. Even though the book does not directly answer this question, the book provides a context to explore the implications of a behavioralistic education policy that uses high stakes testing as the instrument to trigger punishments and rewards. With punishments and rewards enacted on educational stakeholders, do the intended learning behaviors result? Do the adoption of the learned behaviors within a bevahioralistic model provide students and stakeholders access to an adequate range of strategies, thinking skills, and problem solving processes necessary to deal with the complexity of teaching and learning? Finding the intended and unintended effects of the punishments and rewards from high stakes testing on learning and teaching, is the focus of this books examination. In the Unintended Consequences of High Stakes Testing, the authors present some of the intended outcomes for those who advocate the use of high stakes testing and later in the book show the unintended effects of this policy . The authors provide studies, anecdotes, and statistics that show limitations, flaws, and failures in educational environments as a result of these types of tests. In the beginning chapters of the book, the authors provide examples of standardized tests that are unreliable and inadequate to deal with measuring the complexities of assessing educational standards. Later in the book, the authors provide evidence of unintended consequences of high stakes testing in the following areas: narrowing curriculum, increasing dropout rates, decreasing of appropriate pedagogical teaching practices, and increasing the inequality of instruction for minority students and students of low socioeconomic status. In the final chapters of the book, the authors give some suggestions for ‘reforming the reform’ of high stakes testing. In chapter 1, the authors summarize some of the arguments that policy leaders make when making the case for high stakes testing. They argue that this policy of high stakes testing is important for measuring student achievement, for providing information about the quality of schools, and holding students and educators accountable. The authors suggest that these arguments have been successful on changing public perception as demonstrated by the large numbers of states that have adopted these types of testing measures and demonstrated by national surveys in which 66% of the public consider testing is at the right amount or even consider that more testing is needed. The success of these arguments are demonstrated further by the adoption of the National Policy, No Child Left Behind that requires a testing policy that not only “must have clear, measureable goals focused on basic skills and essential knowledge” but also establishes heavy sanctions for schools that do not make the performance goals from those tests. The authors attribute the popularity of high stakes testing to the 1983 Department of Education’s report a Nation at Risk in which the US was portrayed at risk economically and that schools were responsible for this danger. From this historical report, the authors argue that this changed the public’s attitudes towards testing. Despite this claim by the authors, little evidence is provided to link this report to the rise in public opinion for testing other than the chronological order of the report that it predicated the increase in testing popularity. The authors acknowledge that even though high stakes testing is increasing in educational adoption in states across the nations, chapter two of the book provides evidence that shows the negative effects of this type of testing on teaching practices. The authors build the argument that because teacher centered practices have been shown to be more effective in teaching lower level thinking skills, and most high stakes testing assesses low level thinking skills, then teachers are changing their practices to be more teacher centered. They follow this argument with a quotation by a teacher from the Barksdale-Ladd & Thomas, (2000) study that suggests that teachers are using more “skill and drill” instructional practices as a result of the high stakes testing. The argument closes by providing various teacher, student, and parent anecdotes illustrating dissatisfaction with teaching practices that focus on these low level skills. The logic of the argument is sound, however, the evidence provided to construct the argument is based on anecdotal evidence. The authors’ point would have been stronger if they had presented the results of the Barksdale-Ladd & Thomas, 2000 study rather than cite an excerpt from one teacher in the study. Chapter 3 of the book provides some studies that indicate positive changes in teacher’s instructional practices as a result of high stakes testing. The author’s point to studies such as the (Bridge, Compton-Hall & Cantrell, (1997) study in which showed that students were spending more time on higher level writing activities as a result of the statewide portfolio assessment system. The authors suggest that factors other than the high stakes nature of testing may cause teachers to reduce the range of teaching strategies in which they engage. They argue that the type of high stakes tests may be a central indicator to what types of skills get taught and in what instructional practices that teachers use. Based on the studies that the authors offer as support, they suggest multiple choice normed-referenced tests encourage teachers to teach low level thinking skills and portfolio and authentic assessments encourage teachers to teach higher order thinking skills. The authors propose that portfolio and authentic assessments may provide assessment options that may minimize the negative impact of high stakes testing. However, they acknowledge that portfolios present their own set of challenges. The inter-rater reliability of portfolios are low. In the Vermont Portfolio program of 1992 the inter-rater reliability ranged from .33 to 43. The authors suggest portfolios as an alternative assessment option but with the low inter-rater reliability cited in the Vermont program, the summative value of the portfolios reduces its assessment viability to educators, parents and policymakers. To better be able to discern the viability of portfolios and authentic assessments, the authors would have needed to provide further studies and data about other implementations of portfolios. In the middle chapters of the book, authors provide studies that show how the effects of high stakes testing narrows curriculum and increases the amount of time spent on test preparation as examples of the unintended consequences of high stakes testing. They suggest that this increased time spent on test preparation is especially high in low performing and in low socioeconomic schools. The authors provide data about time allocations for test preparation. For example, 28% of teachers spend 60% on test prep materials and 80% of teachers spend at least 20% of all time in class on test preparation. Using this data, the authors pose the question, could this time spent on test preparation be better spent on more meaningful learning tasks? In addition to the reported time spent by teachers on test preparation, the authors indicate that teachers that serve disadvantaged populations change their practices to focus on test preparation using rote memorization teaching strategies and that these strategies are associated with higher dropout rates. The authors provide data that cites statistics from the National Center for Educational Statistics that show minorities have higher dropout rates than Caucasians. However, no studies or statistics are provided that show disadvantaged populations engage in higher rates of rote memorization than other populations. Furthermore, the authors fail to provide any data or studies to show that there is a causal or even correlational relationship between time spent in rote memorization learning and student drop out rates. Even though earlier chapters suggest that this relationship between time spent on rote memorization learning and dropout rates could be related, the authors fail to provide empirical support for their argument. In the following chapter, the authors focus on the unintended consequences of retention. The authors suggest that some states such as North Carolina use high stakes testing to determine student retention and that retention is shown to have a relationship with higher rates of student dropout and other harmful effects. Numerous studies are cited to provide corroborating evidence that show a relationship between retention and increased dropout rates. When schools retain students just one time, the dropout rate of these students rise 50% and two or more grade level retentions push the chance of dropout to 90%. After presenting the studies linking retention and increases in dropout rates, the authors suggest that “neither retention or social promotion is a satisfactory answer” but that viable alternatives do exist. They cite Linda Darling’s alternative practices to social promotion such as tutoring, after-school, and multiage classrooms. Even though several alternatives to retention and social promotion are given, none of these are given enough detail to provide a viable option for implementation. In addition, these suggestions are given in an absence of corroborating studies to support the suggested alternatives. Without citing additional studies, or theoretical frameworks, the options to social promotion and retention cited in the text have little value to poliymakers, educators, or parents. The alternatives do provide researchers possible future areas to investigate and so the authors’ suggestions do provide some value for those stakeholders. The final chapters of the book examine ways to ‘reform the reform’ of high stakes testing. To improve high stakes testing, the authors argue that educators, parents and policy makers need to refocus on the primary goal of education which is to improve student learning. They suggest that by focusing on learning, people will “avoid falling into the trap of thinking about test scores as the goal” (p. 156) Both formative assessment and assessments directly connected to the learning are important implementations of testing that are necessary reforms to the current notion of testing. Furthermore, they suggest that testing for comparison only is detrimental to the learning process. These proposed solutions to the current limitations of high stakes testing are in list form. There are few corroborating studies, anecdotes and statistics in which the authors used in previous chapters to cite the effects of high stakes testing. The proposed solutions provide few details about how the solution could be of implemented or how it had been implemented in other places. Despite the weakness of the section on reforming the reform, the other chapters of the book provide sound arguments using various empirical evidence to support the authors’ claims about the effects of high stakes testing on learning and teaching. Because many of the effects of high stakes testing are negative--such as a narrowing of the curriculum--the cons of high stakes testing need to be weighed against the pros by educators, parents and policymakers. The authors’ large number of empirical studies can inform the scholarship of policymakers who are using high stakes testing as a reform effort to improve the quality of education. The educational community of parents, educators, and policymakers need to consider the intended and unintended consequences of high stakes testing documented in this book so that educational stakeholders create a high quality, equal education for all students. References Amrein, A. & Berliner, D. (2002) High-stakes testing, uncertainty, and student learning. Education Policy Analysis Archives, 10 (18) Retrieved February 25, 2005 from http://epaa.asu.edu/epaa/v10n18/ Coles, G. (2001) Learning to read – ‘scientifically’. Retrived Auguest 3, 2002 from Rethinking Schools 15, no. 4 (Summer): www.rethinkingschools.org/Archives/15_04/Read154.htm Skinner, B. F. (1978). Reflections on behavioralism and society. Englewood Cliffs, NJ: Prentice-Hall. U.S. Department of Education, Office of the Secretary. 2001. No child left behind. Washington, D.C.: Government Printing Office About the Reviewer Bob Pletka is a doctoral candidate in the Education Department at the University of California Irvine. His areas of specialization are data-driven decision making, instructional technology and computer supported collaborative learning. He has been the Director of Information and Instructional Technology at Covina Valley Unified School District for 6 years. Previously, he taught middle school science, computers and mathematics.