Recent Papers by Larry
- Use of Response Patterns and Simple Decision
Theory To Classify Examinees Item responses on an operational state
assessment were calibrated and analyzed using decision theory with the size of
the calibration sample being the primary manipulated variable and score-group
classification accuracy being the primary assessment goal. Simple decision
theory was shown to be highly accurate in terms of placing individuals into
the appropriate score categories. If the intent of an assessment is to
classify individuals into discrete categories or determine the proportions of
examinees within each score category, then decision theory provides an
attractive alternative to classical or modern measurement theory. (12/2002)
- Expected Classification Accuracy Every
time we make a classification based on a test score, we should expect some
number of misclassifications. Some examinees whose true ability is within a
score range to have observed scores outside of that range. A procedure for
developing a classification table of true and expected scores is developed for
polytomously scored items under item response theory is developed and applied
to state assessment data. A simplified procedure for estimating the table
entries is also presented. (12/2002)
- Informed Test Component Weighting - (pdf) Testing programs that report a single score based on multiple choice and performance components must face the issue of how to derive the composite scores. This paper identifies and logically evaluates alternative component weighting methods. It then examines composite reliability and validity as a function of weights, component reliability, component validity and the correlation of the components. Weighting can make a big difference when combining a highly reliable test, such as a lengthy multiple-choice test, with a less reliable test, such as a short constructed-response test. A rational process that identifies and considers trade-offs in determining weights is suggested.
(Educational Measurement: Issues and Practice, 20(1), 16-19 , Spring 2001; prepared for MSDE).
- Information Needs in the 21st Century: Will ERIC Be Ready? - Ubiquitous for 35 years, the Educational Resources Information Center (ERIC)
is known for its database and recently for its range of web-based information
services. This paper contends that federal policy with regard to ERIC must
change and that ERIC will need massive restructuring in order to continue to
meet the information needs of the education community. Five arguments are
presented and justified: 1) ERIC is the most widely known and used educational
resource of the US Department of Education, 2) senior OERI and Department of
Education officials have consistently undervalued, neglected, and underfunded
the project, 3) ERIC's success is due largely to information analysis and
dissemination activities beyond ERIC's contracted scope, 4) information needs
have changed dramatically in the past few years and ERIC cannot keep up with the
demands given its current resources, and 5) the ERIC database itself needs to be
examined and probably redesigned.(AERA 2000, submitted to EPAA; shorter version in press by Bottom Line with a non-exclusive copyright release).
- Who is going to mine digital library resources? And how? - As use of the internet grows as a research tool, patrons have
become increasingly less dependent on librarians and other expert
intermediaries. Examining the quality of on-line searches, the author argues
that researchers and other internet users do not look for and hence do not find
the best resources. He concludes that ready access to resources can lead to
decreased research quality and ill-informed practice. Digital resources must be
developed with expert intermediaries and contain pre-selected resources if they
are to be a service. (AERA 2000; D-Lib Magazine, 6(5). [Available on-line: http://www.dlib.org/dlib/may00/rudner/05rudner.html.)
- The Quality of Researchers’ Searches of the ERIC Database - w/ Scott Hertzberg -
During the last ten years, end-users of electronic databases have become progressively less dependent on librarians and other intermediaries. This is certainly the case with the Educational Resources Information Center (ERIC) Database, a resource once accessed by passing a paper query form to a librarian and now increasingly searched directly by end-users. This article empirically examines the search strategies currently being used by researchers and other groups. College professors and educational researchers appear to be doing a better job searching the database than other ERIC patrons. However, the study suggests that most end-users should be using much better search strategies. (EPAA).
- Scholastic Achievement and Demographic Characteristics
of Home School Students in 1998 - This report presents the results of the largest survey and testing program for students in home schools to date. In Spring 1998, 20,760 K-12 home school students in 11,930 families were administered either the Iowa Tests of Basic Skills (ITBS) or the Tests of Achievement and Proficiency (TAP), depending on their current grade. The parents responded to a questionnaire requesting background and demographic information. Major findings include: the achievement test scores of this group of home school students are exceptionally high--the median scores were typically in the 70th to 80th percentile; 25% of home school students are enrolled one or more grades above their age-level public and private school peers; this group of home school parents has more formal education than parents in the general population; the median income for home school families is significantly higher than that of all families with children in the United States; and almost all home school students are in married couple families. Because this was not a controlled experiment, the study does not demonstrate that home schooling is superior to public or private schools and the results must be interpreted with caution. The report clearly suggests, however, that home school students do quite well in that educational environment. (EPAA)
- How to Write a Scholarly Research Report w/ Bill Schafer Researchers communicate their results and help accumulate knowledge through conference papers, reports, on-line journals and print journals. While there are many rewards for having research disseminated in a scholarly outlet, the preparation of a good research report is not a trivial task.
This article discusses the common sections of a research report along with frequently made mistakes.
More papers:
- Rudner, L.M. (1998). An On-line, Interactive, Computer Adaptive Testing Mini-Tutorial,
[Available online: http://edres.org/scripts/cat]
- Rudner, L. (1998). Item Banking. Practical Assessment, Research & Evaluation, 6(4). [Available online:
http://pareonline.net/getvn.asp?v=6&n=4].
- Drake, L, Rudner, L.M. & J. Pierce (1996). Internet resources for assessment and evaluation. Educational Measurement: Issues and Practice.
- Rudner, L.M., Bracey, G. & G. Skaggs (1996) An application of a person fit statistic with a high quality examination, Applied Measurement in Education, Winter.