Meta-analysis refers to the analysis of analyses...the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings (Glass, 1976, p. 3).

This book and accompanying software are intended to help you learn more about meta-analysis by providing you with a step-by-step guide to conducting a meta-analytic study, references for further research and reading, and free, high-quality software.

Gene Glass first used the term "meta-analysis" in 1976 in his presidential address to the American Educational Research Association to refer to a philosophy, not a statistical technique. Glass argued that literature review should be as systematic as primary research and should interpret the results of individual studies in the context of distributions of findings, partially determined by study characteristics and partially random. Since that time, meta-analysis has become a widely accepted research tool, encompassing a family of procedures used in a variety of disciplines.

Meta-analysis responds to several problems in educational research. First, important issues are studied by numerous investigators. The amount of information on a given topic therefore is often overwhelming and not amenable to summary. Even when there are relatively few studies on a given topic, it is difficult to determine if outcome differences are attributable to chance, to methodological inadequacies, or to systematic differences in study characteristics. Informal methods of narrative review permit biases to remain easily undetected. Reviewers' biases can influence decisions about study inclusion, relative weights given to different findings, and analysis of relations between study features and outcomes. These biases can have clandestine effects when reviewers do not systematically seek to reduce them or provide sufficient information for readers to evaluate their extent.

Meta-analysis typically follows the same steps as primary research. The meta-analyst first defines the review's purpose. Organizing frameworks can be practical or theoretical questions of varying scope, but they must be clear enough to guide study selection and data collection. Second, sample selection consists of applying specified procedures for locating studies that meet specified criteria for inclusion. Typically, meta-analyses are comprehensive reviews of the full population of relevant studies. Third, data are collected from studies in two ways. Study features are coded according to the objectives of the review and as checks on threats to validity. Study outcomes are transformed to a common metric so that they can be compared. A typical metric in educational research is the effect size, the standardized difference between treatment and control group means. Finally, statistical procedures are used to investigate relations among study characteristics and findings.

The system provides you with routines for easy data entry, statistical analysis, and graphic analysis. The data can easily be output in ASCII format or in a format ready for use by the Statistical Package for the Social Sciences (SPSS).

Throughout this manual we discuss effect-size meta-analysis. Yet, Meta-Stat also offers correlational meta-analysis. With our easy graphic routines, you may want to use our Plain-Statistics module for analyzing regular, non-meta-analytic data.

Meta-Stat has been under development for a number of years. A prototype was developed in 1989 with Phase I Small Business Innovative Research (SBIR) funding from the National Institutes of Health. We were able to demonstrate that the system had potential and two years latter we were able to secure Phase II SBIR funding. In late 1993, the first commercial system was ready.

Programming for Meta-Stat was conducted by Lawrence M. Rudner, David L. Evartt, and Patrick J. Emery. This manual was prepared by Lawrence Rudner and Dana Sohr of Technical Communications, Inc. in Fulton, Maryland with the assistance of Gerald Bracey, Gene Glass, Caroline Bagin, David Evartt, and Pamela Getson.

The system has been through two field trials. Our field testers caught numerous mistakes, incorrect formulas, bugs, and the like. We gratefully acknowledge David Gibson, Science Applications Inc; Robert Bangert-Drowns, State University at Albany; Philip C. Abrami Concordia University; Michael J. Strube, Washington University; James A. Kulik, University of Michigan; Chen-Lin C. Kulik, University of Michigan; Roger E. Millsap, City University of New York; Michael R. Stevensen Ball State University; Herbert C. Rudman, Michigan State University; Peter A. Cohen, Medical College of Georgia; Norman Miller, University of Southern California; Karl White, Utah State University; Larry Leslie, University of Arizona; and Arthur L. White, Ohio State University for their assistance with one or both of the field tests.

We are especially indebted to Professor Gene V Glass. Gene has been a consultant to this project since its inception. Aside from moral support, he provided a large investment of time. He provided a comprehensive testing of the system catching errors such as the use of 1.96 rather than the appropriate value of the t-statistic, suggested different ways to present information on the screens, suggested new features, helped edit the manual (he was very picky and he caught numerous errors) and most importantly reacted to ideas and answered questions. We were often in daily communication.

I hope you find this book and Meta-Stat useful in your work and as a teaching and learning tool.

-- Lawrence M. Rudner