Carlson, Jerry S. and Levin, Joel R. (2005). The No Child Left Behind Legislation: Educational Research and Federal Funding. Reviewed by Wayne White, University of North Carolina Charlotte

Carlson, Jerry S. and Levin, Joel R. (2005). The No Child Left Behind Legislation: Educational Research and Federal Funding. New York, NY: Oxford University Press, Inc. viii + 150 pp. $65.95 (hardcover)   ISBN 1-59311-188-6 Reviewed by Wayne White University of North Carolina Charlotte April 26, 2006 The No Child Left Behind Legislation: Educational Research and Federal Funding, edited by Jerry S. Carlson, and Joel R. Levin, is a volume of 13 articles, centered around an initial article by Valerie Reyna. Reyna is a former senior research advisor for the Institute of Education Sciences. The book is a collection of rich discussions about the scientifically based education research associated with the No Child Left Behind Act and the Education Sciences Reform Act. The title of the book is intended to evoke debate concerning two aspects of contemporary education: the possible impact of scientific research and the proper place of the federal government in planning educational change. The volume supports pros and cons on both sides of the argument and packages them in a very readable style. The arguments of the book deliver sometimes deliberate and sometimes fiery rhetoric concerning the most recent Federal regulations and education. The authors argue the impact to the learning community based on the political, economical, and educational strategies of the new policies. The editors promote the issues surrounding federally funded, scientifically based education research by including authors with countering viewpoints regarding the issue and fresh perspectives on some of the data points regarding the current status quo. The book is divided into thirteen chapters: the first chapter is scribed by Valerie F. Reyna and seeks to define, describe, and defend the No Child Left Behind Act and the Education Sciences Reform Act. Dr. Reyna brings a rich background to writing. Dr. Reyna sites her experience as a senior advisor for the Department of Education and as a well-published scientist as drawing points for the content of the article. In Chapter 1, the No Child Left Behind Act (NCLB) and the Education Sciences Reform Act are introduced as major policy transformations and historic reforms for education based on research. Dr. Reyna points out that the phrase “scientifically based research” is mentioned in the act more than 110 times. Her premise is that fundamental change must take place in the kind of research and also in how colleges and universities prepare educational decision makers. In Chapter 1, Dr. Reyna unfavorably compares the U.S. expenditures on education and the percentage of fourth graders displaying proficiency in reading, twelfth graders in math, and twelfth graders in science. She argues that education’s woes can be fixed by holding educators accountable for results and by applying rigorous research to educational practices. Reyna likens the need of working with the most at risk students to the triage model of emergency medicine. The work stipulates that the four pillars supporting the No Child Left Behind Act are “(1) accountability for results, (2) tempered by flexibility and local control, (3) increases in choices available to parents of students attending Title I schools that fail to meet State standards, and (4) an emphasis on educational programs and practices that have been clearly demonstrated to be effective through rigorous scientific research (p. 7).” The major components of the Education Sciences Reform Act were to replace the Office of Educational Research and Improvement with the Institute of Educational Sciences, and establish three divisions: the National Center for Educational Research, the National Center for Education Statistics and the National Center for Education Evaluation and Regional Assistance. Dr. Reyna argues scientific research defining educational practice should include: applications of rigorous, systematic, and objective procedures; observation and experiment; rigorous data analysis; multiple measurements across evaluators, observations, and studies; experimental or quasi-experimental designs with preference to random-assignment experiments; replication; and peer-reviewed journal entries. She argues that random assignment and “experiments are the ethical choice for responsible educators (p.17).” She points out that the two pieces of legislation passed in 2001 and 2002 could substantially change educational practice in the nation for the betterment of students. She also concedes that some successes have been achieved in the past but does not deliberate on these successes. In Chapter 2, Patricia A. Alexander and Michelle M. Riconscente critique Dr. Reyna’s claims that NCLB and related legislation will positively contribute to learning for all students. Drs. Alexander and Riconscente criticize the tendency of NCLB and Dr. Reyna’s article to equate “achievement” and “learning”. They contend that the current definitions of achievement conflict with the research on learning, and that these conflicts cast doubt on the claims of NCLB and its promise of better learning for all students. Drs. Alexander and Riconscente challenge Reyna’s arguments with seven axioms of their own. They are: (1) Learning is much broader than the achievement of basic skills as indicated by test performance, (2) Assessment of learning does not mean tests results and for this to be the sole indicator of student achievement and learning is to speculate unwisely beyond the data, (3) There are flaws to “teaching to a ….test”(p. 30) as a measure of learning, (4) Higher order thinking skills and content knowledge should be integrated for the onset of instruction, (5) Individual differences in students is human nature in spite of any law, (6) Learning and achievement need to be assessed at the student level not the classroom or school, and (7) Scientific evidence is conditional based on the controls and the conditions established by the controls may not have impact on the populations not targeted by the conditions. While offering these axioms, the authors of this chapter warn that NCLB may weaken the very processes it proposes to strengthen. In Chapter 3, Richard L. Allington critiques Reyna’s point-of-view regarding treating education like the medical profession. Dr. Allington questions randomized trials as evidence in medicine and in education. He asserts that there are no randomized experiments testing many of the premises guiding medicine. He indicates that healthy people are not assigned randomly to groups overindulging in either alcohol, food, or tobacco. However, we do have medical assertions based on consumption of alcohol, food and tobacco. Dr. Allington argues that much of the evidence in medicine is correlated not causal. He shares his belief that to randomize experimental studies regarding risk in some lifestyles would be a risk to the healthy and therefore unethical. Dr. Allington questions Dr. Reyna’s statement that “it is unimaginable to administer untested drugs to patients.” One of his examples of just such practice is in pediatric medicine, where he sites a seventy-five percent use of drugs not tested for use in children. He argues that little evidence exists to indicate that medical practice is evidence based. While giving the research references for his arguments regarding scientific research in medicine, he indicates that Reyna lacks supporting scientific evidence that high stakes testing will improve student achievement. Allington further forwards his argument by indicating that: (1) the existing science recognized by the NRP may be more ideology than evidence, (2) science may discover that children differ, and (3) the assurance of federal policy makers and their advisors in unfounded. In Chapter 4, Robert C. Calfee questions considerations forwarded by Dr. Reyna as he presents a case for considering the history of education. Dr. Calfee advances his argument about history by indicating that student achievement has not remained stagnant, but improved despite significant increases in the portion of students at risk economically. He suggests that the research investment is but a fraction of the overall cost and a more appropriate strategy may be to move to a federally funded system of educational research centers and regional laboratories which interpret research for the practitioner. Dr. Calfee argues that the most daunting challenge for K-12 schools today is equal opportunities for all students, regardless of status and circumstance. Dr. Calfee furthers his position by criticizing Dr. Reyna’s position. He questions the use of random assignment as a means of systematic control, as practical and ethical. The arguments fostered in the chapter continue to question NCLB regarding stilted positions on research, researchers, and the Federal role in public schooling. In Chapter 5, Dr. Earl Hunt continues the critique of Dr. Reyna’s article. Dr. Hunt praises Reyna for calling attention to actions taken in the No Child Left Behind Act and the Education Science Reform Act which he refers to as “The Education Acts.” He suggests that two assumptions are the basics of The Education Acts. The first asssumption is that the highest priority is to assure a basic adequate level of achievement for students currently at the lower levels and to accomplish this especially in the early years. The second assumption is that education reform should be based upon scientific evidence and the scientific method. He applauds the two assumptions and asserts that U.S. educaton is not where most of us want it to be (p. 58). Dr. Hunt raises several concerns regarding the implementation of The Education Acts. The concerns raised include: (1) an analogy between military R&D and education R & D, (2) a critical view of the research funding and time expectations, (3) an implication that politics foul the research process separate from the granting agency ( 4) a question of stressing process over principles, and (5) a concern with bureaucratic tendencies. Dr. Hunt asserts that we have compulsory education and are obligated to conduct and apply research to make it better. He concludes that the effort necessary to forward our obligation includes: moving from research to practice, using science and assessment properly, and retaining human safeguards without stifling relevant research. In Chapter 6, Dr. G. Reid Lyon agrees with the points expressed by Dr. Reyna’s paper. Dr. Lyon praised Dr. Reyna’s efforts with the paper and for crafting the ESRA and other research initiatives. The author warns that educational policies and practices would demonstrate little change without being linked to accountability. He offers an explanatory basis for NCLB and Reading First which include: (a) the struggle in reading, particularly by the disadvantaged; (b) the convergence of science and reading development, difficulties, and instruction; (c) the identification and implementation need for scientific reading research; (d) the establishment and insurance of ensuring accountability; (e) the provided flexibility for specific needs; and (f) the establishment of improved granting process. Dr. Lyon suggests that the quality of current educational research is of uneven merit, and suggests that the rigor of the research needs to be questioned and addressed (p. 84). In Chapter 7, Douglas E. Mitchell acknowledges the “scientific research” movement being applied to education improvement, and equates the public outcry to Nation at Risk and Sputnik. He raises cautions regarding NCLB and ERS in four areas: (a) technical science as the center of the policies; (b) the questions of motivation and morality; (c) the academic standing of the teacher workforce; and (d) schools identities as “social institutions” or “service delivery organizations” (p. 90). Dr. Mitchell suggests that the constructiveness of the “science-based” may be less than imagined by the current enthusiasts. He asserts that science takes a cognitive approach more than a technical approach to application and discovery. The author fears that teaching and learning contain social, motivational and moral considerations which may be ignored by science. Mitchell shares that science and professional endeavors consider very different paths to practice. He further stipulates that scientists simplify to understand and interpret, and professionals simply to take action (p. 93). Dr. Mitchell continues his cautionary remarks by indicating that schools are as much cultural, ethical and social institutions as organizations dedicated to student learning. He points out that this divide and the conflict may afford in little results from implementation of NCLB and ERS. In Chapter 8, Dr. Angela M. O’Donnell considers the consequences if No Child Left Behind works. Dr. O’Donnell attests to the literature supporting the need for change in education and research. She identifies key components in the process of implementing NCLB. The components indicated are (1) focus on scientific research, (2) the notion of accountability is consistent with the concept of improvement, and (3) “qualified” educators. Dr. O’Donnell suggests that the focus on scientific research is an important component in NCLB. However, she further reports the limited knowledge base of strategies, curriculum and professional development. This coupled with an under funded research base casts some doubt on the focus. Dr. O’Donnell indicates that the focus on accountability without the necessary knowledge base may drive professionals away from education, thus widening the gap between the desired outcome and available resources. She also reminds us that there are other forces outside of schools that impact student outcomes. In Chapter 9, Gary D. Phye recognizes that Dr. Reyna provided an excellent synopsis of the federal mandates and credits those mandates as being the most important in the past 25 years. Dr. Phye asserts that his view of education from Iowa is similar to that of Washington. Dr. Phye further explains that the classroom consists of three primary elements: the learner, the teacher, and the environment. He stipulates that the educational psychologist is the professional in the position to bring theory and practice together for educational researchers. Dr. Phye further states that the Washington perspective is valid and subject to three points of emphasis: (1) experimentation is the source of credible data collection, (2) discipline, theory, metaphor, and model validation must evaluate the research design and data, (3) adaptivity of behavior is a premise of psychological inquiry and learning is a change in behavior. The author explains the relationship between learning and experimentation to best practices (p. 107). In Chapter 10, Michael Pressley relates some of the research targeting education for the last 30 years. Dr.Pressley contends that research supporting educational efforts rarely evaluates a whole instructional program. He suggests such research of educational programs would produce results long after the program had become an antique. Dr. Pressley challenges the insider thinking concerning the reliability of single study decision making. He credits his cancer cure on the data of a single study. Dr. Pressley is somewhat skeptical of the sustainable efforts to date and suggests that current policy is focused on R&D may indeed provide some studies. He further speculates that educational and medical decisions will continue to be made on limited data and collective experience of the practitioner. Pressley continues his skepticism by disagreeing with Reyna’s position regarding political determination of instructional science processes. He contends that Reyna seems to think that this determination should be in the political arena. Pressley disagrees and indicates that government has the right and the obligation to demand educational outcomes. He believes the obligation for establishing outcomes extends to providing funds for both the work of establishing how high the potential outcomes should be and implementing them (p.118). In Chapter 11, Dr. Daniel H. Robinson shares his pleasure with Dr. Reyna’s views and the swing from postmodernism/constructivism to experimental methods as a means of addressing relevant issues. He attests to witnessing first had the deemphasizing of experimental research (p. 121). Dr. Robinson expresses his delight with the return of educational research to respectable status. In his paper Dr. Robinson confirms that it has been psychology departments and not colleges of education that have provided most of the scientific evidence concerning learning over the past several years. He reflects rather poorly on the meta-analysts and considers this a type of armchair research. He suggests that educational research is characterized by single study rather than a series of studies. Dr. Robinson supposes based on the suggestions of Glass (2000) and Robinson (2004) that meta-analysis may be more appropriately utilized for educational research involving a series of experiments that replicate and extend (p. 126). In Chapter 12, William R. Shadish praises Dr. Reyna’s paper as a service summarizing the history and substance of the No Child Left Behind Act (p. 129). Dr. Shadish points out that finding the reasonable balance between competing conceptual and practical demands is the core of evidence based practice (p. 130). Shadish suggest a number of avenues to understanding causation. They are: (1) cause precedes effect, (2) the difference between pre and post test on the treated subjects and the control subjects is the effect, (3) cause is a complex package of conditions, and (4) the difference between descriptive causation and causal explanation (p. 131). Dr. Shadish continues to indicate that developing research concerning what works is difficult and is as much conceptual as methodological. However, even with those considerations we need data and observations to help develop concepts of causation. Shadish’s primary wish is to impress readers with the complexity of standard setting and making decisions about what works. He further suggests that our aim should be prudent inquiry. In Chapter 13, Valerie F. Reyna attempts to clarify and expand on her remarks in Chapter 1. She urges readers to consult the ideas and concepts expressed by the critics in the work. Reyna characterized the comments put forward as an enthusiastic augmentation of her arguments or qualifications and additions that warrant attention. Specifically, Reyna identifies four key features of the argument to emphasize in her rejoinder. They are: (1) ensuring a supply of sound research supporting practice, (2) ensuring basic research on learning, (3) converting research into practice, and (4) accountability. Her closing comments are again directed at improving education by research and comparing the research and practice base of medicine to that of education (p. 140). Dr. Reyna urges the audience not to give up and indicates that American education’s dream of compensating for many disadvantages can work for many as it did for her. Dr. Reyna gave examples of research’s impact on medicine and the triage model on patient care (Chapter 1) and the quality of medical research and implied care in America (Chapter 1 and 13). She spoke to the number of criticisms concerning her suggestions for research. With all these criticisms, the basic comparison between the medical profession’s delivery of services to all America and educational profession’s delivery of services was not compared. Examples were given regarding educational research and delivery for the disadvantaged but not for medicine. An argument comparing the medical and educational services delivered to the disadvantaged might be in order. References Glass, G.V. (2000). Meta-analysis at 25. Retrieved from https://gv-glass-archives.blogspot.com/search?q=Meta-analysis+at+Middle+age Robinson, D.H. (2004). An interview with Gene V Glass. Educational Researcher, 33(3), 26-30. Robinson, D.H., Whittaker, T., Williams, H., & Beretvas, S.N. (2003). It’s not effect sizes so much as comments about their magnitude that mislead readers. Journal of Experimental Education, 72, 51-64. About the Reviewer Wayne K. White, Assistant Professor, Department of Educational Leadership, College of Education, University of North Carolina-Charlotte. His interest is in educational finance, school leadership, and technological influences on education. Copyright is retained by the first or sole author, who grants right of first publication to the Education Review.