2006 Value-Added Achievement Awards

VALUE-ADDED DATA

Users with difficulty opening the files may also right-click, select "save target as," and download the file.

How the Winners Were Determined:

Introduction
From the data available through the Tennessee Department of Education's website, the Education Consumers Foundation selected the first, second, and third highest performing elementary and middle school principals in Tennessee's East, Middle, and West Grand Divisions. Only principals who have served in their present position for five or more years were considered.

Teachers and parents are the spear tip of education, but schoolwide results require teamwork led by a principal. Our purpose was to identify the principals of those schools that were doing the best job of bringing about high achievement gains regardless of achievement test levels. Some of our winners have students with high achievement test scores and others have averages that are relatively low. The mission of the public school is to do the best they can to help the children that are enrolled. Parents from across the board want schools to help their children to "be all they can be." Of the over 1300 elementary and middle schools in Tennessee, these are the students that are excelling in this primary mission.

In Tennessee and in most others states, a wide variety of grade configurations are found among elementary and middle schools. Among elementary schools, they range from pre-kindergarten to grade 4, to preK-5 or 6, and from kindergarten to grade 8. Among middle schools, they range from grade 5 or 6 to 8 and a few contain only grades 7 and 8. Because of these differences, any comparison among like schools is to some extent arbitrary.

After examining the data, we concluded that combining schools into two groups-grades 4-5 and grades 6-8 would render the fairest comparisons. Schools with grades in both groups (e.g., traditional K-8 schools) were included with the 6-8s if they had two or more grades in that group.

We chose to focus on two of the 4 subject areas that are tracked by Tennessee's educational accountability system: Reading/Language Arts (i.e., reading and writing) and Math (i.e., arithmetic). Public schools seek to teach much more than basic skills, but basic skills are indispensable. Students who fail to learn them are handicapped for life. Schools that fail to teach them cannot be described as effective regardless of their other virtues.

Data and Computations

The data that we compiled is publicly available through the Tennessee Department of Education's (TNDOE) website: http://www.state.tn.us/education/. Tennessee is to be commended for having the most sophisticated educational accountability system in the Nation. Thanks to the creativity of Dr. William Sanders (then of the University of Tennessee) and the far sighted work of former Governor Ned Ray McWherter and the Tennessee General Assembly, Tennessee has been tracking value-added achievement gains by student, teacher, school, and district levels for over 10 years. Only school and district data is public.

The TNDOE provides a wealth of school performance data in addition to value-added reports. The website offers a good bit of guidance, but it seems to assume that readers are educators and that it is up to the reader make interpretations of that which is available. For additional explanation of terminology and of how Tennessee's Report Card may be interpreted, see TDOE's Glossary at http://www.k-12.state.tn.us/rptcrd05/rptcrdterms.htm and the Help tab at the top of each page.

To access the data on which this report is based, follow these steps:

Step 1
Visit the TNDOE website, see the navigation bar on the left and click on "Report Card." A page allowing access to the State's school report cards is opened. The full report card is a multipart document that may be accessed by using the menus in the center of the page.

Step 2
See the navigation bar on the left and click on the "TVAAS Public Site." The page that opens presents a menu listing all Tennessee school systems. In order to view the data on which the Value-Added Achievement Awards are based, roll your mouse to "Reports" tab at the upper right and, first, click on a school system. Then click on the school in which you have an interest.

The page that opens "2005 TVAAS School Report for _____ School in ______ System TCAP CRT Math" displays both the value-added achievement gains and the achievement test averages for the selected school in Math. To see the value-added gains and achievement test results in Reading/Language Arts, roll your mouse across the "Subjects" tab at the top of the menu and click the subject of interest.

The "TVAAS Restricted Site" is for teachers and administrators who need to drill down into the data at the classroom and student level. It provides the kind of information educators need to pinpoint curriculum, teaching, and student learning problems.

Step 3
In the "Estimated School Mean NCE Gain" section of the report (i.e., the value-added achievement gain section), see the row headed "3-Yr-Avg" and select the relevant grades (4-5 or 6-8) for Math and then for Reading/Language Arts.

These are the basic data on which the awards were based.

If you are not familiar with the value-added reports, see the listing of terminology and descriptive information at the end of this document. They are in addition to the help files associated with the value-added data and all of the other information furnished by the TNDOE.

Step 4
With help from the TNDOE, the 3-Yr-Ave data available from the individual school reports was compiled into separate groups for grades 4-5 and grades 6-8. In order for us to combine the Math scores with the Reading/Language Arts scores and to compare schools, the following calculations were undertaken. They are a bit technical but were necessary to ensure accurate comparisons.

For each group (i.e., elementary and middle schools), a normalized score was calculated for each grade and subject (i.e., math and reading/language arts) within each school. A normalized score is a score stated in standard deviation units. It permits scores drawn from differing tests to be combined.

The normalized scores in the two subjects and for the number of grades covered by the school were averaged into a single indicator for each school. This indicator was used to rank the schools. It may be interpreted as part of a distribution with a mean of zero and a standard deviation of one.

Terms and Abbreviations Used in Tennessee's Value-Added Assessment Reports

TVAAS = Tennessee Value Added Assessment System (The name of the accountability system created for Tennessee by William Sanders)

TCAP = Tennessee Comprehensive Assessment Program (The Tennessee-customized version of McGraw Hill's nationally published Terra Nova test.)

CRT = Criterion Referenced Test (A test that provides scores referenced to set criterion; as contrasted to a NRT or Norm Referenced Test that references scores to the average of some group such as national sample of test takers. Tennessee now uses CRTs referenced to statewide averages that were reported in 1998. The purpose of using CRTs is to give schools a constant benchmark against which annual performance can be compared.)

NCE = Normal Curve Equivalent (A test score reported on a scale that ranges from 1 to 99 with an average of 50. For example, if a student earned an NCE score of 70 on a math test, it would mean that his or her performance is equal to or greater than approximately 70% of the other students who took the test. NCEs are approximately equal to percentiles.)

Growth Standard = The 1998 statewide average achievement gain in a given subject and grade. The Growth Standard is the criterion or benchmark against which achievement gains all subsequent years are now judged.

State 3-yr-Avg = The average achievement gain in a given subject and grade for Tennessee's most recent 3 years. (Due to a variety of factors including curriculum changes, the statewide 3-year averages are up in some grades and subjects and down in others. By comparing a given school's present NCE gains with the 3-year average, it is possible to see how far the school has come since 1998 as compared to the progress made by other schools statewide. The state 3-year average was the benchmark against which schools were compared for the present awards.)

Estimated School Mean NCE Gain = The average gain in student achievement made by a given school expressed as an NCE score. (For example, if a school has a "2003 Mean NCE Gain" in Grade 4 of 5.2, it means that on the average, 4th grade students in this school gained 5.2 NCE points more than 4th graders statewide gained in the baseline year of 1998, i.e., the "Growth Standard" year. As a different example, a school with "2003 Mean NCE Gain" in Grade 4 of -3.0 would indicate that students were learning 3.0 points less per year than they were learning in 1998. The school reports are color coded to reflect degrees of growth or decline.

Std Error = Standard Error (The range of points above or below the reported score within which there is a 68% chance that the reported score may vary due to error. All averages are fallible estimates. If it were possible to remove all error from an average, a so-called true score would be obtained. The standard error associated with a given NCE score enables the user to know the accuracy with which it estimates a true score. For example, if a school's "2003 Mean NCE Gain" in Grade 4 was 5.2 and the "Std Error" was 1.0, there would be a 68% chance that the true NCE score for the school is somewhere between 1.0 point greater than 5.2--i.e., an NCE score of 6.2--and 1.0 point less than 5.2-i.e., an NCE score of 4.2. The number of points between 6.2 and 4.2-i.e., 2 points-is called the "confidence interval." A confidence interval of 2 standard errors-4 points in the present example-would be the range within which there is a 95% chance that the true score is to be found. Fallible estimates and standard errors notwithstanding, the "best estimate" of a true score is the average, and thus averages may be used to assess school performance.)

3-Yr-Avg = The average value-added gain of the most recent 3 years expressed as an NCE score. (These scores represent the average number of NCE points by which a school exceeds or falls short of the student achievement growth that occurred statewide in 1998-the benchmark year, a.k.a., the Growth Standard. These are the scores on which the Value-Added Achievement Awards are based.

Estimated School Mean NCE Scores = A school's average TCAP achievement test scores arranged by grade and year, and expressed as NCE scores. (Scores greater than 50 indicate that the school's TCAP scores are above the 1998 statewide average-i.e. above the State Base Year. Scores less than 50 are below the 1998 average. For example, a school with a "2005 Mean" for grade 4 of 64.7 may be said to have TCAP scores that are approximately at the 65%ile of Tennessee's 1998 TCAP achievement test distribution.)

State Base Year (1998) = The statewide TCAP average in 1998. (Test score averages in all subsequent years are reported with reference to the 1998 benchmark. The 1998 statewide average is defined as an NCE score of 50.)

State 3-Yr-Avg = The statewide average TCAP score for the most recent 3 years. (For example, if a school's 5th grade 2005 Mean NCE score is 70 and the state 3-Yr-Avg is 52, the school's score is equal to or greater than those of 68% of comparable schools.)