LSAT Validity and ABA Standard 503
A Foundation of Validity
Beginning with the very first notion of a standardized test for admission to law school, validity was a primary focus. In his May 17, 1945 letter to the College Entrance Examination Board (CEEB) suggesting the development of such a test, Frank H. Bowles, Director of Admissions at Columbia University, stated the seven criteria listed below. (Note that Henry Chauncey, Director of CEEB, in his May 22, 1945 reply to Bowles, acknowledged that some of these criteria may need to be modified in order to achieve others.).
- High predictive value as a measure of success in law school.
- It should be a discrete measure of capacity for law study insofar as that capacity can be isolated.
- High reliability.
- Short enough to be administered in not over an hour and one-half.
- Sensible and observable relation to the study of law.
- Results easy to interpret.
- Low cost to the applicant and to the university.
These foundational criteria demonstrate that validity (1, 2, and 5) and reliability (3) were priorities for the LSAT since its inception.
In the Fall of 1947, the first LSAT validity evaluation was carried out when a three-hour pretest was administered to first-year students at 9 law schools. Pretest performance was compared to participants鈥� first year grades at the conclusion of the 1947-1948 academic year. While these validity results were not available in time to guide development of the first LSAT form administered on February 28, 1948, they did guide the development of the second operational LSAT form, administered in February 1949.
Test validity continued to guide the evolution of the LSAT as law schools considering LSAT scores in their admission process shared first year performance data with LSAT validity researchers. A 1950 report describing results of a series of studies initiated in 1949 noted a correlation between LSAT scores and first year law school grades of .44 and a correlation of .52 when LSAT score was combined with a measure of previous academic success (Schrader & Olsen, 1950). These results are remarkably consistent with results currently observed. Predictive validity is only one aspect of an overall validity evaluation, and the experimental administration and subsequent evaluation of new LSAT question types was common in the early days of the test. Empirical evaluation of such question types, as evidenced by a 1949 research report on question types pretested in the fall of 1948 (French, 1949), led some to be adopted and others to be discarded. The composition of the LSAT evolved to best reflect the skills needed for success in law school.
In the mid-1970鈥檚, 爱游戏体育 President Frederick Hart assembled reports on approximately 105 爱游戏体育-sponsored research studies from 1949 through 1983 into 4 volumes that are still referenced today. A review of the titles in those volumes further demonstrates the value placed on establishing and demonstrating the validity of the LSAT. About half of the reports, 53 to be exact, address the validity of the LSAT in some way (see Appendix A for a list of these reports). The validity research presented includes overall predictive validity, the evaluation of experimental question types, differential predictive validity for population subgroups, validity for repeat test takers, and technical aspects of validity analyses. Note that 爱游戏体育-sponsored research on the best representation of undergraduate grades in predicting law school performance were also noted but not included in the count of 53.
In 1965, 爱游戏体育 (then known as the Law School Admission Test Council) made the evaluation of validity for individual law schools a priority by committing to provide this service free of charge to all member law schools. In June 1968, having carried out 128 validity studies for 97 law schools over the prior four years, the Council voted to make the validity study service permanent (see an excerpt from the memo to law schools in Appendix B). This service, now known as the LSAT Correlation Studies, is still provided today free of charge to 爱游戏体育 member law schools. Participants receive a school-specific report detailing the predictive power of LSAT score and undergraduate grade point average, both individually and in combination, for predicting law school first year average. The results of the Correlation Studies are regularly summarized and published (see Summary of 2017, 2018, and 2019 LSAT Correlation Study Results (PDF)). Technical reports addressing repeat test takers and differential prediction for gender and racial/ethnic subgroups are also regularly shared (see 爱游戏体育 Research Library)
Was 爱游戏体育 required to demonstrate LSAT validity for the ABA?
In a review of the ABA Standards archives available on the organization鈥檚 website (), the first endorsement of LSAT usage in law school admission appears in 1969 as, 鈥淭he Council is of opinion that LSAT, administered by Educational Testing Service of Princeton, New Jersey, should be required of all applicants for admission to the approved schools.鈥� By the time this statement appeared, the LSAT had undergone numerous validity evaluations of various types to support its usage for this purpose. In fact, 20 of the 53 research studies mentioned above had been carried out by this time. 爱游戏体育 had also permanently committed to providing free validity studies to all member law schools, having already provided this service for four years.
Continuing LSAT Validity Evaluation
Validity continues to be the first priority for the LSAT as it was in Mr. Bowles letter of 1945. Annual LSAT Correlation Study participation remains high with 163 law schools having participated in the latest cycle. As mentioned above, LSAT Technical Reports on various aspects of validity are regularly made available. The skills evaluated by the LSAT were also recently validated through the 2018 Skills Analysis Study (see 爱游戏体育 2018 Skills Analysis Study: Content Validity of the LSAT (RR 19-01)).
A Word on Reliability
Unlike validity, reliability is evaluated for an individual test form and can be represented as a single statistic with a theoretical ceiling (never observed in practice) of 1.0. LSAT reliability is consistently above 0.90, indicating that the LSAT is a highly reliable test.
Appendix A
Early Validity Research Reports
French, J. W. (1949). Validation of Experimental Law School Admission Test Administered in the Fall of 1948 (爱游戏体育-49-1)
Schrader, W. B. & Olsen, M (1950). The Law School Admission Test as a Predictor of Law School Grades (爱游戏体育-50-1).
French, J. W. (1952). Validation of the Practical Judgment and Directed Memory Experimental Sections of One Form of the Law School Admission Test (爱游戏体育-52-1).
Johnson, A. Pemberton & Olsen, M.A. (1952). Comparative Three-Year and One-Year Validities of the Law School Admission Test at Two Law Schools (爱游戏体育-52-2).
French, J. W. (1953). Validation of the Study of Arguments and Vocabulary Interest Test Experimental Sections of One Form of the Law School Admission Test (爱游戏体育-53-1).
Coffman, William E. & Papachristou, J. (1955). Experimental Objective Tests of Writing Ability for the Law School Admission Test (爱游戏体育-55-1).
Olsen, M. A. (1956). The Effectiveness of Tests of Writing Ability and of Directed Memory for Predicting Law School Grades and Ratings of Writing Ability (爱游戏体育-56-1).
Hills. J. R. & Raine, W. J. (1959). Keying Pairs of Items for Predicting Law School Grades (爱游戏体育-59-1).
Miller, Paul V. (1962). A Study of the Relationships Existing Among Scores on the LSAT, Writing Ability, and General Background Tests and between These Scores and Certain Other Candidate Variables (爱游戏体育-62-1).
Pitcher, B. (1962). The Prediction of First-Year Law School Grades from Experimental Tests of Writing and Reasoning Ability and from the Law School Admission Test, 19459-1960 (爱游戏体育-62-2).
Miller, P. V. (1963). A Repetition Study of the Relationship Existing Among Scores on the LSAT, Writing Ability, and General Background Tests and between These Scores and Other Candidate Variables (爱游戏体育-63-3).
Winterbottom, J. A., Pitcher, B., & Miller, P. V. (1963). Report on the Readministration of the LSAT to Third-Year Students (爱游戏体育-63-3).
Pitcher, B. (1965). The Law School Admission Test as a Predictor of First-Year Law School Grades 1962-1963 (爱游戏体育-65-2).
Boldt, R. F. (1966). Predicting Law School Achievement with Discrepant Predictor Scores (爱游戏体育-66-1).
Pitcher, B. & Schrader, W. B. (1866). Performance of Experimental Section (爱游戏体育-66-2).
Schrader, W. B., Pitcher, B., & Winterbottom, J. A. (1966). The Interpretation of Law School Admission Test Scores for Culturally Deprived and Non-White Candidates (爱游戏体育-66-3). (for the Ad Hoc Committee on Student Background Factors, Frank K. Walwer, Chairman).
Klein, Stephen P., Rock, D. A., & Evans, F. R. (1967). Predicting Success in Law School with Moderators (爱游戏体育-67-1).
Boldt, R. F. (1968). Extension of the Discrepant Predictor Study 鈥� Final Report (爱游戏体育-68-1).
Klein, S. P. & Evans, F. R. (1968). An Examination of the Validity of Nine experimental Tests for Predicting Success in Law School (爱游戏体育-68-2).
Wills, Thomas A. (1968). Programmed Study Experimental Item Types (爱游戏体育-68-4).
Boldt, R. F. (1970). Discrepant Predictor Study of LSAT and Writing Ability (爱游戏体育-70-1)
Boldt, R. F. (1970). Discrepant Predictor Study Using Validity Study Results (爱游戏体育-70-2)
Carlson, A. B. (1970). Factor Analysis and Validity Study of the Law School Admission Test Battery (爱游戏体育-70-3).
Linn, R. L., Klein, S. P., & Hart, F. M. (1970). The Nature and Correlation of Law School Essay Grades (爱游戏体育-70-4).
Schrader, W. B. & Pitcher, B. (1970). Interpreting Performance of Foreign Law Students on the Law School Admission Test and The Test of English as a Foreign Language (爱游戏体育-70-6)
Angoff, W. H. & Herring, C. L. (1971). Study of the Appropriateness of the Law School Admission Test for Canadian and American Students (爱游戏体育-71-1).
Schrader, W. B. & Pitcher, B. (1972). Discriminant Function Analysis in the Prediction of Law School Performance (爱游戏体育-72-4).
Schrader, W. B. & Pitcher, B. (1972). The Interpretation of Law School Admission Test Scores for Culturally Deprived Candidates: An extension of the 1966 Study Based on Five Additional Law Schools (爱游戏体育-72-5).
Evans, F. R. & Rock, D. A. (1973). A Study of the effects of Moderator Variables on the Prediction of Law School Performance (爱游戏体育-73-2).
Pitcher, B. (1973). An Exploratory Study of the Possibility of Curvilinearity on Law School Validity Study Data (爱游戏体育-73-3).
Schrader, W. B. & Pitcher, B. (1973). Predicting Law School Grades for Black American Law Students (爱游戏体育-73-6).
McPeek, W. M., Pitcher, B. & Carlson, A. B. (1974. The Predictive effectiveness of Several Experimental Item Types and the Operational Item Types in the Law School Admission Test in 1970-71 (爱游戏体育-74-1).
Pitcher, B. (1974). LSAT Part-Score Validity Study (爱游戏体育-74-2).
Pitcher, B. (1974). Predicting Law School Grades for Female Law Students (爱游戏体育-74-3).
Schrader, W. B. & Pitcher, B. (1974). Extension of the Study of the Possibility of Curvilinearity in Law School Validity Study Data (爱游戏体育-74-7)
Schrader, W. B. & Pitcher, B. (1974). Prediction of Law School Grades for Mexican American and Black American Students (爱游戏体育-74-8).
Linn, R. L. (1975). Test Bias and the Prediction of Grades in Law School (爱游戏体育-75-1).
Pitcher, B. (1975). A Further study of Predicting Law School Grades for Female Law Students (爱游戏体育-75-3).
Carlson, A. B. & Werts, C. E. (1976). Relationship Among Law School Predictors, Law School Performance, and Bar Examination Results (爱游戏体育-76-1).
Linn, R. L. & Pitcher, B. (1976). Predictor Score Regions with Significant Differences in Predicted Law School Grades from Subgroup Regression Equations (爱游戏体育-76-2)
Pitcher, B. McPeek, W. M., & Binkley, M. (1976). The Validity of Two Experimental Item Types 鈥� Issues and Facts, and Artificial Language (爱游戏体育-76-3)
Pitcher, B. (1976). Subgroups Validity Study (爱游戏体育-76-6).
Schrader, W. B. (1976). Summary of Law School Validity Studies, 1948-1975 (爱游戏体育-76-8).
Powers. D. E. (197). Comparing Predictions of Law School Performance for Black, Chicano, and White Law Students (爱游戏体育-77-3).
Linn, R. L. (1977). On the Treatment of Multiple Scores for Law School Admission Test Repeaters (爱游戏体育-77-4).
Pitcher, B. (1977). The Validity of the Law School Admission Test Scores for Repeaters (爱游戏体育-77-5).
Boldt, R. F. (1977). Evaluation of Three Methods for Treating Repeaters鈥� Scores (爱游戏体育-77-6).
Pitcher, B., McPeek, W. M., & Binkley, M. (1977). The Validity of the Usage experimental Item Type (爱游戏体育-77-7).
Rubin, D. B. (1978). Using Empirical Bayes Techniques in the Law School Validity Studies (爱游戏体育-78-1).
Powers, D. E. (1980). A Consideration of the Possibility of Differential Predictive Effectiveness of Writing Ability Scores for Minority Groups (爱游戏体育-80-1).
Powers, D. E. (1981). Predicting Law School Grades for Minority and Nonminority Students: Beyond the first-Year Average (爱游戏体育-81-1).
Evans, F. R. (1982). Recent Trends in Law School Validity Studies (爱游戏体育-82-1).
Linn, R. L. & Hastings, C. N. (1983). A Meta Analysis of the Validity of Predictors of Performance in Law School (爱游戏体育-83-1).
Appendix B
Excerpt from Memo to Law Schools Announcing Permanent Validity Study Service
Additional reports in this collection
