SFU Dean Paul Shaker slams Fraser Institute

SFU’s dean of education, Paul Shaker, doesn’t mince words when it comes to the Fraser Institute’s ranking of B.C. elementary and secondary schools. During a recent debate over these “report cards”, Shaker claimed that the right-wing think tank’s research wouldn’t stand up to scrutiny if its findings were subjected to peer review.

“The gloss or the veneer of science is put on these claims, but real science isn’t there,” Shaker said.

The British Columbia Association for Supervision and Curriculum Development hosted the event on January 18 in Langley, pitting Shaker against Peter Cowley, the Fraser Institute’s director of school-performance studies. Cowley opened the discussion by saying it was “critically important” to determine if a school is achieving its stated goals.

“How do we know if we’re improving without measurement?” Cowley asked.

The discussion focused on the Fraser Institute’s 2006 “Report Card on British Columbia’s Elementary Schools”, which is based on provincial Foundation Skills Assessment tests for numeracy, reading, and writing. Each school’s students are tested in grades four and seven. The Fraser Institute then ranks the schools based on average marks. Public schools in wealthy neighbourhoods and private schools usually register higher test scores.

Cowley, a former businessman, said he became interested in school performance while volunteering on his daughter’s school-consultative committee in the early 1990s. None of the data in the Fraser Institute’s report card was available to him, even though the Ministry of Education was turning this information over to school districts. “Once we did know, as the result of the judicious placing of a report on a table by a sympathetic teacher”¦it confirmed some of the concerns that we had about individual results at that school,” he said.

Shaker said that measurement has its place in the evaluation of schools, emphasizing that it is just one of several accountability tools. He warned that too much emphasis on a single ranking, however, can result in educators narrowing the curriculum so their students will do well on tests, sometimes at the expense of learning in other areas that are more difficult to measure, such as the arts.

“Measurement is not a cure, and measurement is not a comprehensive method for evaluation,” Shaker said.

Cowley told the audience that the institute applies several criteria to the data: it must be objective; it must be collected by a third party (by that, he meant the ministry); it must be easily comparable between schools; it must be generated annually; and, ideally, it should be available from a central source, such as the Ministry of Education.

Cowley defended the report card as a good “networking tool”, enabling educators to learn which schools are successful. “I’ve had the occasion to be the brunt of anger, tears, sorrow, a feeling of distress, and diminishment from very good people,” Cowley admitted at one point. “What I believe that they misunderstand is that in the rankings, no one is saying that the people at the low-ranked schools are bad people. No one is saying that they are bad teachers.”

Shaker pointed out that there is no correlation between the Fraser Institute rankings and parental satisfaction, as monitored by the ministry. “There is highly significant median correlation with parental education levels: 30 percent of the variance in the Fraser Institute rankings can be correlated at a significant level with parental education levels,” he said.

The U.S.–based National Assessment for Education Progress, unlike the Fraser Institute, embeds demographic variables into its evaluations, Shaker noted. This “value-added approach” takes into account such factors as the percentage of poor people in a school’s catchment area, the percentage of single-parent families, and the number of times poor people only encounter other poor people in their neighbourhood.

Shaker also lambasted the Vancouver Sun for devoting so much editorial space to the Fraser Institute’s report card. “It’s unprecedented that you can see such a simplistic kind of ranking used to such an extent in the media,” Shaker said. “It troubles me that the Sun has no ombudsperson or public editor who can call the editor to account and look more deeply into this issue, like the public editor would at the New York Times.”