What Education’s Policymakers Can Learn from the Enron Collapse

An Occasional Paper by J. E. Stone

Education Consumers ClearingHouse

November 2002

Conflicts of interest, biased research, and bad advice to investors led to Enron’s financial collapse. Investments in education have failed for essentially the same reason. It is a lesson education’s policymakers need to learn. Agencies that produce and disseminate educational research cannot be faithful to both investors and providers. Providers want rosy assessments and investors need candid ones. The Bush administration’s new Institute of Education Sciences is supposed to correct the problem but it faces the same pressures that have shaped the other publicly funded research and policy sources. Policymakers need watchdogs that bark. They should pay attention to performance, not rhetoric.

Conflicts of interest, biased research, and bad advice to investors led to Enron’s financial collapse. Investments in education have failed for essentially the same reason. The Bush administration’s new Institute of Education Sciences is an attempt to correct the problem.

The Institute replaces the U. S. Department of Education’s Office of Educational Research and Improvement (OREI). It joins a new What Works Clearinghouse and an Office of Innovation and Improvement. All three aim to increase the “scientific rigor” of the research that informs education policy decisions.

Without question, improvement is needed. Says Rep. Michael N. Castle, chairman of the U. S. House’s Subcommittee on Education Reform, “I want quality educational research, not fads or anecdotes.”

But will the new offices avoid the pitfalls that undermined the efforts of their predecessors?

There is reason for optimism, but change will ultimately require policymakers to gain a new understanding of why educational research has failed them in the past.

Serving Two Masters

In the 1970s and ‘80s, a federal entity whose mission it was to disseminate effective programs to the schools — the National Diffusion Network (NDN) — examined the outcomes of the Follow Through project. Follow Through was the very kind of rigorous, large-scale trial that the new Institute of Education Sciences hopes to promote. However, instead of encouraging schools to adopt the best programs, NDN recommended that schools consider both the effective and the ineffective programs, and that the poor performers be given added funding because of their greater need to improve!

The problem was conflict of interest. Many teachers disagreed that their favored programs were ineffective, and strongly defended them. Policymakers, by contrast, lacked the information and expertise to disagree with the NDN assessment. By recommending both the effective and ineffective programs, NDN avoided jeopardizing its support among educators or policymakers. The new Institute of Education Sciences will face similar pressures and incentives.

Merrill Lynch and Enron

An instructive parallel can be drawn between conflicts of interests in the education arena and the recent financial collapse of the Enron Corporation. In that situation, firms that provide financial research to investors were profiting from sales of the stocks that were being researched.

In 1998, a Merrill Lynch securities analyst named John Olson lost his job because he refused to give Enron a “buy” rating. Now a widely respected analyst, Olson explained that many members of his profession rendered excessively optimistic assessments because they were subjected to incentives and pressures from within their brokerage firms.

Olson and his colleagues were responsible for appraising stocks like Enron. At the same time, other divisions within the same company were trying to promote and sell them. “Buy” ratings were encouraged because they promoted brokerage business from corporations like Enron. “Sell” ratings did the opposite, so they were rare.

Investors were taught a hard lesson. Owners and buyers have competing interests. Owners benefit from rosy assessments, and buyers benefit from candid assessments. Brokerage houses cannot be faithful to both.

It is a lesson that education’s “investors” — i.e., policymakers — need to understand.

Conflicts of Interest in Research and Evaluation

The information furnished to policymakers is portrayed as impartial and objective, yet the parties that produce and disseminate it often benefit from affirmative reports. The conflict may be subtle, but the temptation to “put lipstick on a pig” is substantial. If a source of research-based guidance rarely finds fault with policies and innovations (i.e., never issues “sell” ratings), it is unlikely to be serving the interests of education’s investors and consumers.

A few years ago, Professor Herbert Walberg raised this issue in an essay questioning program evaluations conducted by their sponsors and developers. Self-evaluation is desirable, but his point was that inherent conflicts of interest are unnoticed and self-assessments are mostly unaudited.

A less obvious but equally potent form of conflict exists when programs hire outside contractors to assess their performance. For example, the National Board for Professional Teaching Standards (NBPTS) recently granted $6.6 million to 22 organizations to study various aspects of its teacher certification process.

Exhibiting some sensitivity to the issue, NBPTS engaged the services of the RAND Corporation to vet the proposals. Still, the transaction was one of a well-funded program seeking external validation by contracting with researchers who will benefit if the program prospers.

If favorable or even mixed results are found, NBPTS will remain a “golden goose.” Refinements will be implemented and additional assessments will be needed. However, if findings are negative, not only will NBPTS suffer, the contractor may become an undesirable choice for other programs seeking self-evaluation.

See No Evil

Conflict of interest does not inevitably result in bias. Rather it creates an incentive for bias, and it is the bias that becomes visible over time.

In the course of the past 30 to 40 years, policymakers have invested in countless educational innovations and reforms. Few have produced substantial improvement and many have been conspicuous failures. Virtually all, however, have been accompanied by “buy” recommendations from educational researchers.

Perhaps policymakers should have been more skeptical, especially about positive self-evaluations. However, policymakers rarely act without information and advice from one or more of the governmentally funded organizations and agencies that study, digest, and report on education research and policy. So with all of this guidance, why have so few failed initiatives been questioned, opposed, or subjected to a post mortem? The short answer is that most have been deterred by the same pressures that sidetracked the NDN.

Organizations such as OREI, the regional education laboratories, the state education agencies, and the many independent and university-affiliated research and policy groups have been counseling policymakers for decades, yet they have rarely detected or questioned what have proven to be badly flawed programs. To the contrary, they have mostly endorsed, disseminated, and implemented anything that generates funding. In fairness, however, studies with critical or negative findings do not often make it into print.

Reviewing the archives of several education news sources, I was struck by the paucity of references to cautionary reports regarding the many educational fads of the last 20-30 years. Neither were there references to studies of their financial or human costs. Surely policymakers would benefit from knowing the who, what, where, and why of past mistakes.

Was there lack of editorial interest? Not likely. Negative assessments by major organizations typically stir controversy and news coverage. For example, there was extensive coverage of OREI’s criticisms of bilingual education and early intervention programs in the days when Chester E. Finn, Jr. headed the agency. Rather, the absence of references seems to accurately reflect a benign silence toward fads and failures by the governmentally funded, grant-seeking organizations that give guidance to policymakers.

An Illustration

Earlier this year I authored a study reporting that 16 Tennessee teachers who are both certified by the National Board for Professional Teaching Standards (NBPTS) and who have scores recorded in the state’s value-added assessment system were merely average performers in the classroom. It was a small study — the state database contained only 16 teachers — but it remains the only study that examines the linkage between NBPTS certification and objectively measured student achievement. And it is the only one that suggests NBPTS should be temporarily suspended and reexamined.

Given my findings, I expected NBPTS to be displeased. What I did not expect was a hurried announcement from the Education Commission of the States (ECS) that it would form a panel of “unbiased, distinguished educators and researchers” to review my study.

Figuratively speaking, ECS pulled off its rose-colored glasses and took out its microscope. However instead of considering both the evidence for and against NBPTS, only the study’s disquieting results were subjected to a hairsplitting analysis.

The panel concluded that my results might not be widely applicable, yet the central finding remained: In 16 of 16 cases, NBPTS certification overestimated teacher effectiveness; and the implication for education’s investors was obvious.

The conflict faced by ECS was obvious as well. In a letter accompanying the panel report, ECS president Ted Sanders acknowledged that there was no research showing that NBPTS-certified teachers are more effective; yet he carefully avoided suggesting that the absence of evidence was any reason for policymakers to resist committing millions to NBPTS initiative.

By taking a wait-and-see approach, organizations like ECS appear prudent and cautious. In truth, they are making a virtue of necessity. Straddling the line between investors and providers permits them to avoid controversy.

Policymakers Need to Assess Conflicts, Track Records, and Loyalties

It is inevitable that educational initiatives that spend millions of dollars attract lots of friends. Resisting their influence will be a stern challenge for the new Institute of Education Sciences and its sister agencies. In Britain, Chris Woodhead, Her Majesty’s Chief Inspector of Schools, was wholly independent yet he was virtually driven from office by educational interests that resented his uncompromising assessments.

What can increase the new agencies’ chance for success is increased policymaker awareness of how research and policy agencies have failed in the past and a more sophisticated approach to finding investor-friendly sources of educational guidance.

Like the now-chastened Enron investors, policymakers need to give greater weight to sources that are transparent with regard to conflicts of interest, that have a sound record of policy assessments, and that are demonstrably faithful to investor/consumer interests, i.e., sources that issue both “buy” and “sell” recommendations.

Here are questions that might be asked:

· Have they issued any “sell” recommendations in the past — i.e., questioned the merits of any of the education fads of the last 30 or so years, particularly before the money ran out?

· Have they endorsed or promoted any of these same fads?

· Have they assessed the human and/or financial costs of fads and failures, or taken other steps to help policymakers (and themselves) to avoid the same mistakes?

· Who pays for their information and advice and who has been helped by it: investors or providers?

Public education is a regulated monopoly staffed by professionals whose aims and priorities do not necessarily match those of parents, policymakers, and the public. The notion that this gap can be bridged by rhetorical commitment to what is “best for the children” or to a “balanced” or “unbiased” view is a convenient fiction.

In truth, policymakers need watchdogs that bark, not agencies whose need for self-preservation overrides investor interests and deters all but the most oblique and reserved criticisms. If policymakers want to know whether the new agencies are loyal to education’s investors, they should pay attention to their performance, not their rhetoric.

J. E. Stone (professor@education-consumers.com) is a professor in the College of Education at East Tennessee State University and principal partner of the Education Consumers Consultants Network (www.education-consumers.com) – an affiliate of the Education Consumers ClearingHouse.