Recommend This Page to a Friend Printable Version
Return to the Education Consumers ClearingHouse
www.education-consumers.com Phone & Fax - (423) 282-6832
     
Charles Arthur, M.S.Ed. / 
President and Executive Director /  Mastery Learning Institute /  Portland, OR

Virginia P. Baxt, Ed.D. / 
President /  Education Agenda, Inc.

Wayne Bishop, Ph.D. / 
Professor of Mathematics /  Department of Mathematics /  California State University-Los Angeles

William L. Brown, Ph.D. / 
Director of Institutional Assessment /  Lansing Community College

Guy Bruce, Ed.D. / 
President, APEX Consulting /  Assistant Professor, St. Cloud State University

Louis Chandler, Ph.D. / 
Professor & Chairman /  Department of Psychology in Education /  University of Pittsburgh

Andrea Clements,Ph.D. / 
Professor /  College of Education /  East Tennessee State University

Donald Crawford, Ph.D. / 
Education Specialist /  Otter Creek Institute

Nathan Crow, B.A. / 
School Administration Consultant /  Education Consumers Consultants Network

George K. Cunningham, Ph.D. / 
Professor /  School of Education /  University of Louisville

Mary Damer, M.Ed. / 
Instructor & Student Teaching Supervisor /  Northern Illinois University

Jerome Dancis, Ph.D. / 
Associate Professor /  Department of Mathematics /  University of Maryland-College Park

Edwin J. Delattre, Ph.D. / 
Professor  /  College of Arts & Sciences /  Professor & Dean Emeritus /  School of Education /  Boston University

Benjamin F. Eller, Ed.D. / 
School of Education /  Western Carolina University

Lucien Ellington, Ed. D. / 
Professor & Co-director, Asia Program /  College of Education and Applied Prof. Services /  University of Tennessee at Chattanooga

John Eshleman, Ed.D. / 
Apex Consulting

David R. Feeney, Ed.D. / 
Director of Digital Education /  Fox School of Business & Management /  Temple University

Carol C. Gambill, M.Ed. / 
Director-Special Projects /  Curriculum & Instruction /  Tennessee Department of Education

Patrick Groff, Ed.D. / 
Professor of Education Emeritus /  San Diego State University

Bonnie Grossen, Ph.D. / 
Professor, College of Education /  University of Oregon

Richard Gruetzemacher, Ed.D. / 
Director /  Planning, Evaluation, and Institutional Research /  University of Tennessee at Chattanooga

Mark Y. Herring, Ed. D. / 
Professor & Dean of Libraries /  Winthrop University
 
Featured Research:

Can Teacher Quality Be Effectively Assessed?
 (click title for full article)

By Dan Goldhaber
University of Washington Center for Reinventing Public Education
Emily Anthony Urban Institute


Briefing:

Teachers pay $2,300 to be assessed by the National Board for Professional Teaching Standards (NBPTS). They earn pay increases up to $7,500 per year if successful. To date, NBPTS has certified over 30,000 teachers.

Despite widespread reports to the contrary, the evidence brought forth by Goldhaber & Anthony does not strengthen the claim that NBPTS-certified teachers are substantially more effective than their colleagues in bringing about student achievement. If anything, the case for NBPTS certification may now be weaker.

Prior to Goldhaber & Anthony, only two small-scale investigations had examined the link between NBPTS certification and student achievement. Bond, Smith, Baker, & Hattie (2000) compared 31 NBPTS-certified teachers with 34 teachers who had applied for certification and failed. There was a slight edge in achievement favoring the NBPTS-certified group, however, the study was fraught with methodological difficulties (see Podgursky, 2002).

Stone (2002) examined the value-added achievement gains of the 16 NBPTS-certified teachers for whom data was available in Tennessee’s state database. He found both above and below-average performance, and determined that none of the 16 would have qualified for the performance-based salary increase awarded by one of Tennessee’s urban school districts. Stone’s report was the first empirical study to raise doubt about the value of NBPTS certification; and as such, stirred considerable criticism.

Findings
Goldhaber & Anthony’s study compared all of North Carolina’s 3rd, 4th, and 5th grade NBPTS-certified teachers to their colleagues around the state. It analyzed the achievement test scores of over 400,000 students taught by more than 16,000 teachers and concluded that NBPTS-certified teachers produced “significantly” greater gains in student achievement.

Its conclusion, however, referred only to the “statistical significance” of the results. In fact, the differences found by Goldhaber & Anthony were quite small and of doubtful scientific, educational, or practical importance (see Table 1, p. 29).

Less than one point (.49) in annual reading score gain separated NBPTS-certified teachers from nonapplicants (6.18 versus 5.69), and an even smaller difference (.35) separated them from those teachers who applied but were not selected (6.18 versus 5.83).

A similar pattern of small differences was observed with respect to the math scores. NBPTS-certified teachers differed from nonapplicants by less than one point (.46, 10.21 versus 9.75) and from teachers who applied but were not selected by slightly more than one point (1.07, 10.21 versus 9.14).

Statistically significant does not mean important
“Significance,” as the term is used in statistical reports, refers only to whether a numerical result is of sufficient size to be considered something more than the product of random error. It refers to a minimum or threshold level of outcome. It is not a synonym for “important.”

To their credit, the authors acknowledged that the observed differences were “. . . relatively small; the largest differential [being] in math between certified and non-certified teacher applicants, at just over a point on the exam or roughly 14% of a standard deviation in the growth of math scores”(p. 14).

What their comments did not make clear, however, is just how small these differences are relative to accepted benchmarks for "effect size."

Effect size is the difference between two groups stated as a percentage of a standard deviation (i.e., “14%”). It is the appropriate statistic for gauging the importance of results such as those found by Goldhaber & Anthony.

Jacob Cohen (1969, 1988)--perhaps the foremost authority on effect size--offers the following guidelines: greater than 50% = “large,” 50-30% = “moderate,” 30-10% = “small,” and less than 10% = “insubstantial, trivial, or otherwise not worth worrying about.” Other authorities (Glass, McGaw, & Smith, 1981) caution that Cohen’s categories may be too liberal because they fail to take cost-effectiveness into account.

Goldhaber & Anthony found one effect size of 14% and three others in the 6-8% range. Clearly, effect sizes of this magnitude are of doubtful importance.

Another way of judging effect size is by comparison to the effects of other policies and interventions. Goldhaber and Anthony employ this practice when they compare their results to the effects of class-size reductions and to the effects associated with teachers’ holding a bachelor’s degree in their subject area (p. 18).

Mark Lipsey and David Wilson compiled a list of effect sizes drawn from hundreds of educational and psychological studies. The median was 34%.

Of particular relevance to Goldhaber & Anthony, Lipsey & Wilson found substantial differences in student achievement effects associated with various teaching methodologies. For example, the use of positive reinforcement produced an average effect of 117% whereas the “open classroom” approach to teaching produced effects ranging from plus 1% to minus 13%.

Plainly, such a range of effect sizes suggests that choice of teaching methodology has a far greater impact on student achievement than does NBPTS certification.

Conclusion
Rather than demonstrate the superiority of NBPTS-certified teachers, Goldhaber & Anthony showed that NBPTS-certified teachers are nearly indistinguishable (see chart) from their colleagues.

Given that this same conclusion was suggested by Stone’s earlier study, policies that reward teacher quality on the basis of NBPTS certification should be reconsidered. As shown by Goldhaber and Anthony and by Stone, statistical analysis of test scores is not only a viable alternative, it is evidently more objective, accurate, and cost-effective as well.


The Education Consumers Consultants Network is an alliance of experienced and credentialed educators dedicated to serving the needs of parents, policymakers, and taxpayers for independent and consumer-friendly consulting. For more information, contact J. E. Stone, Ed.D., at (423) 282-6832, or write: professor@education-consumers.com

Daniel Hursh, Ph.D. / 
Professor of Educational Psychology /  College of Human Resources & Education /  West Virginia University

Carol Jago, M.A. / 
English Teacher, Santa Monica High School /  Director, California Reading & Literature Project /  UCLA

Jerry Jesness, M.A. / 
ESL Teacher & Author /  Los Fresnos (TX) Schools

Daniel Konieczko, M.Ed. / 
Science Teacher, Brunswick High School /  Brunswick, ME

Martin Kozloff, Ph.D. / 
Watson Distinguished Professor /  School of Education /  University of North Carolina at Wilmington

Rob Kremer, M.B.A. / 
President /  Oregon Education Coalition

Elaine K. McEwan-Adkins, Ed.D. / 
Author & Consultant /  McEwan-Adkins Group

Richard P. Phelps, Ph.D. / 
Economist & Author

Michael Podgursky, Ph.D. / 
Professor and Chairman /  Department of Economics /  University of Missouri

J. Martin Rochester, Ph.D. / 
Curator's Distinguished Teaching Professor /  Department of Political Science /  University of Missouri-St. Louis

Linda Ross, Ph.D. / 
Director & Instructional Design Specialist /  Archimedia eLearning Solutions

Valerie Copeland Rutledge, Ed.D. / 
Associate Professor /  College of Education /  University of Tennessee at Chattanooga /  Member, Tenessee Board of Education

Douglas Sears, Ph.D. / 
Professor & Dean /  School of Education /  Boston University

Mark C. Schug, Ph.D. / 
Professor & Director /  Center for Economic Education /  University of Wisconsin-Milwaukee

Lewis Solmon, Ph.D. / 
Dean Emeritus /  UCLA Graduate School of Education /  Executive Vice President, Education /  Director, Teacher Advancement Program /  Milken Family Foundation

Robert Spangler, Ed.D. / 
President /  Spangler & Associates, LLC

J. E. Stone, Ed.D. / 
Professor /  College of Education /  East Tennessee State University

Sara Tarver, Ph.D / 
Professor /  School of Education /  University of Wisconsin-Madison

John Towner, Ph.D. / 
Professor Emeritus /  Woodring College of Education /  Western Washington University

Herbert Walberg, Ph.D. / 
Research Professor of Education & Psychology /  Emeritus /  College of Education /  University of Illinois-Chicago

John T. Wenders, Ph.D. / 
Professor of Economics, Emeritus /  University of Idaho

Richard Western, Ph.D. / 
Professor (retired) /  School of Education /  University of Wisconsin-Milwaukee