Carnoy, Martin; Jacobsen, Rebecca; Mishel, Lawrence & Rothstein, Richard. (2005). The Charter School Dust-Up: Examining the Evidence on Enrollment and Achievement. Reviewed by Larisa Warhol, Arizona State University

Carnoy, Martin; Jacobsen, Rebecca; Mishel, Lawrence & Rothstein, Richard. (2005). The Charter School Dust-Up: Examining the Evidence on Enrollment and Achievement. Washington, DC: Economic Policy Institute and New York, NY: Teachers College Press. 186pp. $19.95   ISBN 0-8077-4615-0 Reviewed by Larisa Warhol Arizona State University January 16, 2006 The controversy over the effectiveness of charter schools reached a boiling point in August, 2004, when the New York Times published a front page article claiming that student achievement in public schools was in fact higher than achievement in charter schools. This article was based on a publication of the American Federation of Teachers (AFT), a supporter of charter school reform, that analyzed unreleased data from the federal government’s National Assessment of Educational Progress. Response was immediate from charter school advocates who placed a full-page ad in the Times claiming that the AFT had employed flawed methods of analysis and was biased against charter schools. The Charter School Dust-Up: Examining the Evidence on Enrollment and Achievement comes out of this debate to contribute its own evaluation of charter school student achievement and enrollments. Providing a systematic review of the debate, Martin Carnoy, Rebecca Jacobsen, Lawrence Mishel, and Richard Rothstein seek to establish an unbiased view of charter schools by examining all available evidence for comparing public and charter school achievement and enrollment. They use data based on nineteen studies that were conducted in eleven states and the District of Columbia. They also examine findings put forth by the AFT, the National Center for Education Statistics (NCES), and charter school advocates. Their findings reveal that charter school students do not outperform public school students; and in fact, the impact of charter schools on student achievement is negative. Beyond the data regarding student achievement, the authors also address potential policy implications and discuss commonly advanced rationales for supporting charter schools. However, they caution that the purpose of their book is not to provide an in-depth analysis of reasons for supporting charter schools or attendant policy issues. Rather the book focuses on the controversy surrounding the AFT report and its subsequent publication in the Times. Their analysis seeks to demonstrate that, based on current data, it should not be surprising that charter school student achievement is below that of public schools and that any further debate over charter school policy should proceed from this point. The authors first provide a detailed account of the reaction to the AFT report, the problems with the report, but also the problems with the critique of the report. They pay close attention to the issues that sparked the controversy. Charter school supporters cited conflict of interest on the part of the AFT and flaws in the methods of their reports (i.e., single point-in-time testing and insufficient student background information). The supporters also claim that many charter schools attract more disadvantaged students than traditional public schools, which would account for the low test scores. The authors confirm that the AFT report did have flaws. Yet they also note the hypocrisy of charter school advocates since the latter often put forth the same rationale that was used in the ATF report but in support of charter schools. For example, charter school advocates maintain that charter schools should initially be exempt from standardized testing until they have been able to stabilize their school curriculum and student body. Yet charter school advocates are unwilling to make the same concessions to public schools which might be adopting a new instructional program or have a new principal (p. 23). The authors also point out that charter school advocates have been inconsistent in their views of school assessment. Single point-in-time testing was criticized in the charter school advocates’ advertisement in the New York Times, yet the Charter School Leadership Council initially called for NAEP scores to be collected and analyzed. Only after the test results did not reveal higher student achievement in charter schools was this assessment method considered flawed by charter school advocates (p. 27). Discussion of the controversy also includes evaluation of assessment standards and No Child Left Behind (NCLB). While the authors claim not to be discussing policy issues, they do provide a critique of current NCLB standards and comment on the limitations of standardize testing as the only means of measuring student achievement. Having set the background of the controversy surrounding the New York Times article and ATF report of 2004, the authors systematically go through all available reports and data gathered on both the national and state level dealing with charter school and public school enrollment and achievement. The bulk of the book is dedicated to this evaluation of the data and reports from these various states. This analysis of the data available on both charter and public schools reveals the comprehensive examination that the authors give to support their findings. They also scrutinize specific examples cited by charter school advocates and reports that support the charter school movement. One argument on the side of charter school advocates against the AFT report was that charter schools serve a high proportion of disadvantaged students, thus resulting in the lower test scores. Examining data from each of the various states, the authors could find no evidence to support this claim. They provide the specific example of the KIPP network of charter schools, which has been held up as an extremely effective model for disadvantaged youth. Using both available published data on KIPP enrollment and test scores as well as teacher interviews, the authors could find no evidence that KIPP actively recruits a more disadvantaged student body nor that it has lower test scores as a result of serving disadvantages students (p. 65). The claim by charter school advocates that charter schools served a more disadvantaged population was not supported by available data. Next, examining both the AFT report and the later NCES report on charter schools, the authors found little variation in the analysis of test scores, thus undermining the charter school advocates claim that the AFT report misused the NAEP data. The authors next go through state-level studies conducted on charter school student achievement. Some studies provided data on charter schools that had been operating for over three years, the interval of time that the charter school advocates claim is required for school stability. These data revealed that these students were also under-performing. The authors focus on the Hoxby report. Caroline Hoxby, Allie S. Freed Professor of Economics at Harvard, is a charter schools advocate, and her reports were published after the New York Times article. Many charter school advocates claimed her study was more valid because it had a larger sample, and it also showed that charter schools do have high student achievement (p.98). The authors provide a close analysis of Hoxby’s report that reveals some serious flaws. She violated the evaluation standards that the New York Times charter school advocates had put forth. She also had weaker performance measurements than the NEAP data and used a single point-in-time analysis. Her study also contradicted findings from earlier state studies. Based on their findings from these various reports, the authors’ overall analysis reveals that charter school students are consistently achieving less than public school students. The authors conclude by describing important lessons learned from the debate over charter schools prompted by the 2004 AFT report. They caution that accurate school and student assessments cannot be made using current standardize testing methods. More precise student characteristics need to be taken into account and year-to-year analysis should be done on the same group of students. They maintain that any initial claims made that charter schools will result in higher student achievement have proven groundless. Also, lower student achievement is not based on charter schools having enrolled more disadvantaged children. Charter schools instead do show increased segregation and greater student mobility. The authors acknowledge that some charter schools do produce higher student performance, but they question whether the benefit of maintaining these few schools with current charter school policy is worth the harm caused by the system as a whole. This book is put forth by the authors as an unbiased view of charter schools and as an evaluation of their student achievement and enrollment by the Economic Policy Institute in the wake of the controversy over the 2004 AFT report. Controversy over the validity of charter schools has been widespread since their inception. (Bracey, 2005). The Charter School Dust-Up: Examining the Evidence on Enrollment and Achievement contributes a valuable and clear analysis of current charter school performance. The authors succeed in providing comprehensive evidence to support their final conclusion that charter schools on the whole have been ineffective. They also provide important commentary on current assessment policy and measures although they claim that is not their goal. While the authors claim to maintain an unbiased view of charter schools, the books reads like a critique of charter school advocates and their positions. In the Introduction, the authors explain they will repeatedly be using the term “zealots” to refer to charter school advocates, particularly those who maintain the superiority of charter schools regardless of credible data to the contrary. They claim to use this term to distinguish those who rushed to support charter schools in the wake of AFT report versus those who did not. Regardless of their explanation, when reading the book, the excessive use of the term “zealots” for these charter school advocates influences how the reader perceives charter school supporters and implies a negative opinion of them. Although the authors claim to see some merit in charter schools, they appear critical of the charter school movement in general and the inconsistent opinions and reports of charter school advocates. References Bracey, G. (2005). Charter Schools' Performance and Accountability: A Disconnect. Tempe, AZ: Education Policy Studies Laboratory, Arizona State University. Retrieved January 16, 2006, from http://www.asu.edu/educ/epsl/EPRU/documents/EPSL-0505-113-EPRU.pdf About the Reviewer Larisa Warhol is a PhD student in the Educational Leadership and Policy Studies program at Arizona State University. She formerly worked as an education consultant with many charter schools in the Philadelphia, PA area. Copyright is retained by the first or sole author, who grants right of first publication to the Education Review.