Osterlind, Steven J. (1997). A National Review of Scholastic Achievement in General Education: How Are We Doing and Why Should We Care? Reviewed by Andreea M. Serban

Osterlind, Steven J. (1997). A National Review of Scholastic Achievement in General Education: How Are We Doing and Why Should We Care? ASHE-ERIC Higher Education Report 25, No. 8. Washington, DC: The George Washington University, Graduate School of Education and Human Development. Pp. 117 + xii ISBN 1-878380-80-Y Reviewed by Andreea M. Serban Rockefeller Institute of Government June 24, 1998 Introduction       Standardized tests have been debated for years. With the advent of the assessment movement in higher education in the 1980s, these debates have intensified but a resolution regarding the fate of standardized tests does not appear any closer. Moreover, although standardized tests have become an integral part of the students' college experience, there are virtually no comprehensive studies analyzing the results of these tests and their implications for undergraduate education. Steven J. Osterlind's study represents a first attempt to analyze nationally the students' achievement in general education based on the results of the College Basic Academic Subjects Examination (College BASE).      The structure of the monograph covers four sections. The introduction provides a description of the study and clarifies the use of the term "general education" within the framework of the study. The author then describes the College BASE instrument and compares it with other commercially available tests of general education. The finding section is a detailed report of the students' achievement both overall and by subgroups of student population and test subject areas. Osterlind concludes the volume with a summary of his findings and a discussion of the implications of the study. The monograph is a valuable addition to the student outcomes literature and a novel effort of reporting what college students know and what skills they possess. As long as standardized tests will continue to be widely used as assessment instruments, such studies are critical for evaluating the quality and effectiveness of college education.      The Study and the College BASE Instrument College BASE is one of the youngest members of the standardized test family geared toward general education. Other tests included in this category are the College Outcome Measures Program and the Collegiate Assessment of Academic Proficiency, developed by American College Testing, the General Examinations of the College-Level Examination Program, developed by the College Board, and the Academic Profile, developed by the Educational Testing Service (for a description of the major commercially available tests of cognitive abilities, see, for example, Jacobi, Astin, and Ayala, 1987).      According to Osterlind, "College BASE is a criterion- referenced achievement test that focuses on the degree to which students have mastered particular skills and competencies consistent with the completion of general education coursework" (p. 11). Of the tests mentioned earlier, the author states that College BASE is the only one that meets the technical criteria for a criterion-referenced test. This characteristic allows the use of the test scores not only for assessment of students' knowledge and skills but also for the evaluation of the quality and effectiveness of academic programs (p. 11). Two-thirds of the test items assess high-level cognitive reasoning skills and the remaining one-third assess important, factual knowledge. The learning outcomes assessed by College BASE were derived from the College Entrance Examination Board's Educational EQuality Project (1983). The test assesses achievement in four subject areas: English, mathematics, science, and social sciences. Subject-areas scores are built on content clusters, which are based on skills (p. 15). The author briefly describes the clusters and skills and the reasoning competencies assessed across disciplines. He also addresses issues related to the test's calibration and scoring, and its reliability and validity. The study is based on the College BASE scores for 74,535 students tested between 1988 and 1993. The students represent 56 colleges and universities. The author acknowledges that the sample of institutions and of students tested within any given campus was not random, but by convenience. However, the study represents a considerable number of students and institutions.      The results are reported overall and by sub-population along four categorical variables: gender, ethnic heritage, class standing, and age. To ensure the integrity of the data, references to scores on particular items are derived from a common set of questions that were included on every form of the test. Findings      The presentation of findings is divided into two major parts. The first presents the overall results for the entire study population and then for the subgroups of the population. The focus of the section is to compare and contrast differences among the major subjects and then the clusters. The second part reports the results in more detail at all three levels - subjects, clusters, and skills - for each subgroup of the population. The number of tables and figures discussed is kept at a minimum such that the reading is not disturbed by unnecessary or redundant data. The statistical analyses performed and presented are basic and easy to understand by scholars and novice as well. Although not surprising, some of the findings of the study are disturbing. They suggest that there are wide differences in the students' achievement in general education, especially by race. Scores in three of the four subjects (mathematics, statistics, and social studies) are very close to each other and may indicate that students' overall level of achievement among these areas is relatively equal. English falls significantly behind the other three areas.      For all four subject areas at all levels of analysis - overall, cluster, skill - there are differences between genders. In English, females outperform males but in the other three subjects, the situation is reversed. Interethnic differences are less consistent than gender ones. Within the Asian sub-population, for example, there are significant disparities between achievement in mathematics and achievement in the other areas, especially when contrasted with English. The Hispanic sub-population scores significantly better in social studies than in other areas. Caucasians tend to be more consistent across the four subjects. The differences between racial groups are a cause for concern. In mathematics, Asian students outperform all other groups and Caucasians lead the other groups in English, science, and social studies. Blacks/African Americans lag far behind all other groups in all subjects. "In some cases, the gap between Blacks'/African Americans' achievement and the other groups is so wide it is more than alarming. It is frightening" (p. v). Older students tend to score better than younger ones. Implications of the Study      Osterlind acknowledges the limitations of his study, including the convenience sample of institutions and students, the difficulty of arriving to a common definition of general education, the challenge of capturing in a test the evolutionary changes in curriculum, the impossibility of inferring conclusions about any particular general education course or class within any institution, and the nature of the test scores of providing a snapshot of achievement. The author also underscores that the descriptive nature of the study leaves important questions unanswered, the most important being why scores vary so widely and what explains the level of general education achievement for the sub- population groups. Nevertheless, the findings of the study pinpoint some important dimensions of students' knowledge and skills and lay the framework for research critical to the understanding of college experiences and their implications for the quality and effectiveness of the undergraduate education. Some of the implications include "the necessity for special programs for low-achieving students and more opportunities to extend the collegiate experience for high achievers" (p. v). A more long-term implication is the need for systematically sampling the achievement of students who are pursuing formal postsecondary education in a nationally based research project. Concluding Remarks      The recently released Carnegie report (1998) on the status of undergraduate education at research universities is the latest example of outcry regarding the quality and effectiveness of undergraduate students' preparation. The report indicates that "Many students graduate having accumulated whatever number of courses is required, but still lacking a coherent body of knowledge or any inkling as to how one sort of information might relate to others. And all too often they graduate without knowing how to think logically, write clearly, or speak coherently." The underlying questions emanating from such reports are: what is the problem with the undergraduate education and how can it be fixed? Osterlind's study attempts to respond partially to the former question, leaving the latter open for further research. General education is an area most colleges and universities agree it is important. However, as the monograph suggests, students have significant deficiencies in one or more general education subject areas. Although this study does not provide any hint as to why these deficiencies exist, it helps identify the content knowledge and skills that higher education institutions should make sure all students possess. References      For product and ordering information of College BASE see: http://www.coe.missouri.edu/~arcwww/prd1.html Boyer Commission on Educating Undergraduates in Research Universities. (1998). Reinventing Undergraduate Education: A Blueprint for America's Research Universities. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching. College Entrance Examination Board. (1983). Educational EQuality. Academic Preparation for College: What Students Need to Know and Be Able to Do. Washington, DC: Author. Jacobi, M., Astin, A., and Ayala, F. (1987). College Student Outcomes Assessment: A Talent Development Perspective. ASHE-ERIC Higher Education Report, No. 7. Washington, DC: Association for the Study of Higher Education. About the Reviewer Dr. Andreea M. Serban is a research associate at the Rockefeller Institute of Government, Public Higher Education Program. She is currently working on projects dealing with performance funding for public higher education, performance measurement and reporting, and state budgeting and its effects on and implications for higher education. E-mail: serbana@rockinst.org http://rockinst.org/higheduc.htm