J. E. Stone, Ed. D.
House of Representatives
Testimony to the Select Committee on House Resolution
Tuesday, September 19, 2000
My name is John Stone. I am a licensed educational
psychologist and licensed school psychologist,
and I have taught prospective teachers for the
past 30 or so years. I am a graduate of the
University of Florida and currently a professor
in the College of Education at East Tennessee
As I am sure you will notice, unlike Vice President
Gore, who is also from Tennessee, I was not
raised in the Washington DC area.
My purpose today is to talk about accountability
and value-added assessment. Since the early
nineties, Tennessee has used a system of educational
accountability that serves the needs of both
educators and the consuming public.1 It is called
the Tennessee Value Added Assessment System
Instead of looking at school system averages
as does the Pennsylvania System of School Assessment
(PSSA), value-added assessment compares each
student to his or her own track record of achievement
and summarizes that information by teacher,
school, and school district. School and school
district reports are made public. They enable
parents to see not just which schools have high
test scores, but which schools are helping students
the most. Parents want schooling that optimizes
their child's talents and value-added assessment
encourages that kind of schooling.
A critically important feature of Tennessee's
system is that expert help is not required to
interpret differences between schools and districts.2
The scores are analyzed and reported in such
a way that the influence of all preexisting
differences is removed. I mean that everything
from socioeconomic factors, to differences in
talents, to differences in initial skill levels
are removed so that everyone including the lay
public can accurately understand the outcomes.
Just as the newest generations of computer programs
use very sophisticated programming to make software
user-friendly, Tennessee's system of accountability
uses some very sophisticated statistics to create
user-friendly educational data.
Teacher reports are sent to teachers and principals.
They are enormously helpful because the parties
on the business end of schooling need to see
how well classroom practices and policies are
working, and without the uncertainties created
by student differences. Tennessee's system,
in effect, creates a level playing field for
Before I continue, I should make clear that
my point of view may a bit different than that
of other witnesses. I head an online service
called the Education Consumers ClearingHouse
(www.education-consumers.com). The ClearingHouse
is a kind of Consumers Union for the consumers
of public education.
It is a paid subscription service that serves
parents, employers, school board members, lawmakers,
taxpayers, and all who are on the consumer side
of the education marketplace. We afford our
subscribers access to consumer-friendly, information
and expert advice. The ClearingHouse also supports
the development of local Education Consumer
Associations (they are like PTAs) and a Consultants
Network. I am appearing here today as a representative
of our Consultants Network.
All of this is to say that my remarks will
not be from the perspective of the education
community or an education interest group or
from that of a vendor who is marketing something
to the education community. Rather I am here
to talk about accountability from the standpoint
of what parents and taxpayers would like to
see in their public schools.
Two Key Issues
I want to begin by saying that I am very much
in sympathy with your effort to strengthen your
state's accountability system. There is a rising
tide of public concern about whether students
are getting what they should get from their
school experiences. I believe that an effective
accountability system can address that problem
but a flawed system will produce change too,
and maybe the wrong kind.
No system of summarizing the results of an
enterprise as large and expensive as a school
will reflect everything that the institution
has produced and do so to everyone's satisfaction.
In truth, there is no perfect system of auditing
or accounting whether we are talking about industrial
outputs or financial management. Yet independent
auditing is as necessary to public confidence
in the schools as it is to consumer confidence
in banks and corporations.
I am going to take the next few minutes to
talk about two issues I believe are crucial
to an accountability system that will build
confidence in public education. They are:
- How well does it respect the public's educational
aims and priorities, i.e., does it make them
accountable for what the public wants?
- Does it produce results that can be used
to improve events at the classroom level?
How well does it help teachers and administrators
to see the results of their efforts?
These are by no means the only issues that
you will want to consider but they are ones
that I believe are essential to the kind of
accountability desired by parents and taxpayers.
I will close my remarks by presenting a brief
description of Tennessee's value-added system
in the hope that it will suggest some refinements
that you will want to consider.
Let me begin by talking about the public's
aims and priorities. Parents and taxpayers value
education primarily because of what it does
for a child's future. Responsible adults understand
that kids grow up and that the knowledge and
skills they acquire in school are critical to
their becoming independent, self-sustaining
members of society.3 Adults also understand
that the opportunity to fully participate in
and benefit from school does not last forever.
They recognize that the competing responsibilities
of adult life have undermined many well-intentioned
efforts to return to school.
Educators believe that knowledge and skills
are important to adult success but they do not
necessarily agree that teaching focused primarily
on knowledge and skills provides the most desirable
sort of learning experience.4 Rather, virtually
all educators are taught that the ideal learning
experience is, first and foremost, pleasant,
engaging, interesting, and enjoyable; and they
are given to believe that such experiences automatically
produce optimal learning outcomes. These outcomes
may not be outcomes that will contribute to
the child's long-term welfare but educators
are given to believe they are what is best for
the child. Many parents have their doubts but
they are supportive and cooperative in the hope
that the desired outcomes will eventually emerge.
In my judgment, this misalignment between educational
ideals focused on the immediate experience and
parents' desires for their child to acquire
recognized knowledge and skills is the root
cause of the public's dissatisfaction with public
schooling. Establishing clear accountability
is needed not so much as a guide to adding resources
or a means of identifying slackers; it is needed
to redirect the very considerable effort that
is already underway.
Let me digress a moment to clarify my meaning.
Several recent studies have identified inner
city public schools whose learning outcomes
are equal to or better than most suburban schools.
Kipp Academy in the Bronx, P. S. 161 in Brooklyn,
and Wesley Elementary in Houston are three that
have recently gotten attention in the national
media. One would suppose that schooling methods
used by the highly successful schools would
be widely imitated but they are not. Why? Because
their more structured and orderly approaches
to teaching sharply disagree with the mainstream
pedagogical wisdom, i.e., the educational ideal
I am describing.
The ideal to which I refer is the foundation
of a wide variety of educational practices,
present and past.5 Diane Ravitch's recent Left
Back: A Century of Failed School Reforms offers
an excellent description. They include fads
such as the "open classroom" concept
of some years ago and the current approach to
reading instruction called "whole language."
They include "discovery learning,"
"developmentally appropriate instruction,"
and a number of other popular approaches to
teaching that have been around since the sixties
and seventies. Collectively they are called
student-centered or learner-centered schooling.6
Student-centered is the term applied by Jeanne
Chall in her posthumous The Academic Achievement
The learner-centered ideal is an appealing
but unsubstantiated concept that requires teachers
to approach the task of teaching as though the
student's immediate satisfaction is more important
than the long-term outcome. By contrast, the
more traditional teaching methods used by Kipp
Academy and other successful schools focuses,
first and foremost, on the demonstration of
knowledge and skills, not the student's immediate
reaction. They seek to be stimulating interesting,
and engaging but not at the expense of the outcome.
As teachers at Kipp Academy and other successful
educators have demonstrated, learning requires
students to behave themselves, pay attention,
and make an effort. With students who already
fit that description, learner-centered approaches
can work well. With the other 95%, however,
a more structured and orderly approach is usually
Learner-centered thinking is at odds with the
public's educational priorities in a subtle
but critical way. Instead of teaching in a way
that is hoped to be entertaining, teachers are
encouraged to entertain in a way that they hope
will be educational. Learner-centered approaches
typically result in kids loving school and parents
being favorably impressed when they visit the
classroom. It's only later that the substantive
results are noticed.
My reason for addressing this issue is to make
it clear that an accountability system that
will build public confidence in education must
be one that makes the public's desire for improved
achievement its top priority. Yet it must also
be sold to an education community that is not
convinced of the public's aims. The task will
require extraordinary leadership and, indeed,
The learner-centered pedagogical orthodoxy
is widely held and many educators view the public's
insistence on objectively measured achievement
as an infringement on professional prerogatives
and a requirement that is contrary to good educational
practice. Many have been trained exclusively
in learner-centered thinking. Many have been
given to believe that it is the only valid if
not the only ethically defensible approach to
teaching. In truth, many teachers do not know
of, much less believe in, result-oriented approaches
I will give you one quick example. The largest
educational experiment in history was Project
Follow Through of the sixties and seventies.7
It was part of the Johnson administration's
War on Poverty and it cost over a half billion
dollars. Follow Through compared 11 of the major
approaches to teaching by field-testing them
at sites all over the U. S. Nine models competed
the study and seven of the nine were learner-centered.
Five of the seven learner-centered models produced
less achievement than the comparison schools--schools
using traditional teaching methods. The two
non-learner centered models greatly outperformed
the field beating both the other models and
the comparison schools. Yet as of the present,
the more effective of the two successful models-Engelmann
and Becker's Direct Instruction-is included
in the curriculum of only one teacher training
program, that of the University of Oregon.
For all of these reasons, I would urge you
to be patient and to recognize that change will
take time and significant rethinking by many
in the education community. Change will have
to be addressed on many levels. Professional
development, for example, will be a key factor.
Open-minded teachers-especially the newly minted
ones-will need to be introduced to resulted-oriented
teaching methods.8 The learner-centered view
has been around since the early nineteen hundreds
and it is ardently held by many fine, intelligent
educators. Many feel morally obligated to oppose
more result-oriented approaches because they
have been told that they are a threat to student
self-esteem or student creativity.
Teacher accreditation and licensure standards
must also be reviewed and aligned. The new standards
promoted by the National Commission on Teaching
and America's Future, for example, continue
to favor learner-centered teaching. The Commission's
proposals agree with accountability, in principle,
but urge fuzzy assessments of learning, in practice.9
All of this is to say that implementation an
accountability system that makes objectively
measured learning a clear and unrivaled educational
priority will be a long-term undertaking and
one that will have to address more than the
mechanisms of accountability.
Individual classrooms and individual schools
are where the rubber hits the road in education.
There are many teachers and principals-especially
experienced ones--who recognize that learner-centered
teaching doesn't really result in the kind of
learning wanted by the public or by educators
but they have to keep their skepticism under
wraps. They know that education's experts do
not support their views and they are reminded
of that fact every time they attend an inservice
training session or continuing education course
or teacher conference. Moreover, teachers and
principals know they are expected to cooperate
with various learner-centered reform initiatives
that are implemented in schools. These research
projects and educational innovations are almost
all externally funded and administratively supported.
In such an environment, deviation from learner-centered
orthodoxy can be career threatening. Teachers
who want to implement result-oriented instruction
frequently have to do it behind closed doors.
In California, for example, the legislature
mandated a return to more traditional methods
of reading instruction after it was found that
bottom had fallen out of reading scores. The
essence of the problem was the learner-centered
"whole language" reading instruction
that had been promoted by the State Department
of Education in the nineteen eighties. Despite
the legislative mandate, teachers who were trying
to improve results found it prudent to hide
their phonics-based instructional materials.
They knew that many of their peers and supervisors
did not agree no matter what has been mandated.
Teachers and principals hear educational leaders
talk about aims and policies but they see what
is rewarded. Good teaching is theoretically
that teaching which produces learning. In truth,
however, actual measured learning plays little
or no role in decisions about which teachers
are given awards for excellent teaching. Popularity
with students, parents, and other teachers is
far more important. The same holds true with
principals who are given promotions and pay
increases. Measured learning plays only a small,
if any, role in these decisions.
Two factors are primarily responsible for this
inattention to results. One is the learner-centered
orthodoxy. It practically obliges educators
to be dismissive of results as indicators of
true educational quality. The other factor is
the difficulty of accessing, summarizing, and
interpreting achievement data. Objective test
score differences among teachers and schools
have to be interpreted in light of all kinds
of mitigating factors such as differences in
entry level skills, socioeconomic differences,
and myriad other factors. Differences on fuzzy
outcome measures such as student portfolios
are, as a practical matter, not interpretable.
Kentucky, for example, tried to use portfolio
assessment as part of its high stakes accountability
system. After several years of effort and millions
expended, it was discovered that the assessments
The point I am getting to is that an accountability
system that supports effective teachers and
principals must permit educators to see not
only differences among schools but differences
among teachers. Some teachers are doing an effective
job of getting students to learn. Others are
not and they need to know what is working and
what is not. Rarely is the case that they are
insufficiently dedicated to serving students
or not working hard.
The Educational Empowerment Act applies various
consequences to schools and school districts,
thus principals and superintendents will have
to know which teachers are producing results
and which are in need of help. Given the vagaries
of test score interpretation, both administrators
and teachers would be better served by an accountability
system that provides fair, accurate, and objective
data at the teacher level in addition to that
which available for the larger units. Teacher
data would not need to be made public (it isn't
in Tennessee) and its role in performance evaluations
could be subject to negotiation. For example,
it could be used in peer evaluations. In any
case, improvement will require both administrative
decisions and classroom practice to be tightly
coupled with achievement data.
A study by Sanders and Rivers in Tennessee
found that a student who had the good fortune
to have effective teachers three years in a
row averaged 50% more learning than one having
poor teachers over a three-year span.10 As presently
configured, the PSSA reports data at the school
and system level yet learning is mainly governed
by events at the classroom level. Ideally, parents
want assurance that their school is providing
an effective teacher, not just a popular one.
Tennessee's Value-Added Assessment System
Now I will outline the workings of the Tennessee
Value Added Assessment System (TVAAS) and try
to show something of how it addresses these
two issues.11 I will minimize details about
how the scores are derived, but I will furnish
you several articles that contain a fuller description
and reference sources.12 Incidentally, Tennessee's
system has been subject to review by some of
the foremost authorities in educational measurement
and evaluation and it has performed as expected.
TVAAS measures teacher, school, and school
system performance by looking at the year-to-year
increases in achievement test scores earned
by individual students. Every regular education
student in every grade (2-8) takes an annual
achievement test-currently a custom version
of McGraw-Hill's Terra Nova. Fresh items are
written annually and the scores are reported
on a 1000-point scale. The basic datum on which
the value-added analysis is performed is the
annual gain in individual achievement.
The annual test scores are analyzed using a
statistical procedure called "mixed model
analysis." It is a type of analysis previously
known mostly to geneticists and it has now been
applied to educational achievement by Dr. William
Sanders-developer of the Tennessee Value Added
Assessment System. The Sanders analysis estimates
the degree to which students have improved in
a given year over their previous years of schooling.
In other words, Sanders' value-added analysis
looks at each student relative to his or her
track record and estimates the individual's
gain over his or her history of academic achievement.
Such an analysis is a very close approximation
of the informal appraisal of student's test
scores that might be made by a teacher or parent.
Although the actual computations do not produce
individual student gain scores, teacher, school,
and school district scores are the average of
the gains earned by the students who were served
by a given teacher, school, or school system.
Unlike assessment systems that compare scores
to a norm or fixed criterion, value-added assessment
encourages schools to maximize the talents of
each student, not just those that are below
a certain level. Such an approach is very much
in agreement with the aspirations of most parents.
There is one other equally important advantage
to Dr. Sanders' "mixed model" methodology:
Through a process statisticians call "blocking,"
Sanders analysis excludes the influence of all
preexisting differences among students in the
estimates of gain. These include any differences
that might be due to race, socioeconomic status,
previous learning, intelligence, and any other
biasing factors, known or unknown. These are
the same kind of biasing factors that PSSA sought
to eliminate from its school reports in the
early nineties through a procedure called multiple
regression analysis. The important difference
is that Sanders' analysis removes all such biasing
factors and without having to individually measure
The attached charts illustrate the ability
of TVAAS to level the playing field. One shows
the relationship between school district gains
in reading and the percentage of students receiving
a free or reduced lunch. The other shows the
relationship between school district gains in
reading and percent minority students.
The importance of a level playing field
I want to take a minute to elaborate on the
importance of a level playing field. I have
taught college freshmen for thirty years and
my wife is in preschool special education. We
have both seen instance after instance of something
that frustrates most teachers (whether they
say so or not): The teachers with the most talented
students are almost always thought of as the
best teachers. Obviously, the most talented
and advantaged students tend to learn the most
and be most satisfied, so it is not surprising
that their parents and most everyone else think
they are doing a great job.
When school administrators say things like
"everyone knows who are the best teachers,"
these are the teachers they have in mind. In
truth, the teachers who work with the weaker
students may be working much harder and, in
fact, may be contributing more to their students'
educational development. They rarely are identified
as the best teachers, however.
Value-added assessment of teacher performance
changes the accepted image because it looks
achievement objectively. Even modest gains can
be very impressive with a student that has not
done well previously. Conversely modest gains
with talented and advantaged students may not
be satisfactory. In Tennessee, one of the most
disquieting findings in the six years that TVAAS
data that has been collected is that students
with high achievement scores are in many cases
making less than average gains. In other words,
Tennessee has many bright students who have
been permitted to coast toward the average,
i.e., they have been "dumbed down."
When parents who have sacrificed to live in
the "good" school districts learn
the truth, they are very unhappy.
Other aspects of Tennessee's value added
Testing all students annually is more expensive
and time consuming than the selective sampling
used by the PSSA, but the added cost is minor
relative to state educational budgets. Additional
time is used but it too is a small amount relative
to the days lost on account of weather and extracurricular
School and school system performance data is
publicly reported in the form of a 3-year rolling
average for each of the five subjects measured
by the annual test-officially known as the Tennessee
Comprehensive Assessment Program or TCAP. I
have a copy of a TVAAS Report Card Supplement
available for your perusal.
Teacher performance scores are based on all
students taught by the individual teacher for
at least 150 days in the course of the school
year. Teacher performance is also reported in
the form of a 3-year rolling average in each
of the 5 subjects but unlike the school and
system reports, it is not made public. Rather
teacher reports, by law, are sent only to the
teacher and the school system.
Although Tennessee has not yet done so, the
teacher reports can be used as an indicator
of teacher training effectiveness. The performance
of novice teachers during their first 3 years
can serve as an objective indicator of how well
a given program is preparing its graduates.
TVAAS report cards are freely available to
the public although many parents and taxpayers
are only beginning to recognize their importance.
Schools typically try to control their impact
by spinning the annual results to the media.
Unfortunately, education reporters often file
a rewritten press release. However, Education
Consumers Associations and real estate agents
are beginning to read and use TVAAS data to
advise parents and prospective homebuyers about
the best schools and the best neighborhoods.
Summary and Conclusion
Renewed public confidence in education will
require accountability that is fair and useful
to educators and convincing to parents and taxpayers.
Value-added assessment can meet all of these
requirements, but the education community will
have to be sold on the idea of teaching dedicated
to achievement. If they accept that concept,
classroom practices that work can be developed,
identified, or adopted on the basis of their
value added outcomes.
- For more complete information about value-added
assessment, see www.isc-erh.com/vaas.html#TITLE1
Dr. William Sanders-the creator of Tennessee's
value-added assessment system may be contacted
at the SAS Corporation <Bill.Sanders@sas.com>.
- J. Stone, "Value-Added Assessment: An
Accountability Revolution" in M. Kanstoroom
and C. Finn, Jr., eds., Better Teachers, Better
Schools (Washington, DC: Thomas B. Fordham Foundation)
- J. Johnson and J. Immerwahr, First Things
First: What Americans Expect from the Public
Schools (New York: Public Agenda) 1994.
- J. Stone, "Aligning Teacher Training
with Public Policy," The State Education
Standard, 1(1), pp.34-38, (2000).
- D. Ravitch, Left Behind: A Century of Failed
School Reforms, (New York: Simon & Schuster,
- J. Chall, The Academic Achievement Challenge,
(New York: The Guilford Press, 2000).
- C. L. Watkins, "Project Follow Through:
A Story of the Identification and Neglect of
Effective Instruction," Youth Policy, (July
- D. Carnine, Why Education Experts Resist
Effective Practices (April 2000). Available
at the Fordham Foundation website: www.edexcellence.net
- J. Stone, "The National Council for
Accreditation of Teacher Education: Whose Standards?"
in M. Kanstoroom and C. Finn, Better Teachers,
Better Schools (Washington, DC: Thomas Fordham
- W. Sanders & J. Rivers, Cumulative and
Residual Effects of Teachers on Future Student
Academic Achievement, 1996 (Available from UTVARC,
225 Morgan Hall, P. O. Box 1071, Knoxville,
TN 37901-1071). See also, J. C. Rivers Sanders,
The Impact of Teacher Effect on Student Math
Competency Achievement, unpublished doctoral
dissertation (Knoxville, TN: University of Tennessee,
- Ibid. J. Stone, "Value-Added Assessment:"
- D. Hill, "He's Got Your Number,"
Teacher Magazine, Vol. 11, number 8, pp. 42-47
(May 2000); J. Archer, "Sanders 101,"
Education Week, (5 May 1998); J. Archer, "Putting
Value Added to Good Use," Education Week,
(5 May 1998).