Coop média de Montréal

Journalisme indépendant

More independent news:
Do you want free independent news delivered weekly? sign up now
Can you support independent journalists with $5? donate today!
Not reviewed by Coop Média de Montréal editors. copyeditedfact checked [?]

Lies, Damn Lies and La Presse Statistics

Blog posts reflect the views of their authors.
And crooked like La Presse poll results?
And crooked like La Presse poll results?

I support the student strike. I think other people should too. I'll argue with people who are in favour of tuition fee increases; I think it's a legitimate discussion though, and believe there are intelligent arguments in favor of the hike. I just don't agree with them.

I wanted to get that out of the way so there are no pretensions here. Each journalist has their opinion about the strike and fee increases, but we don't hear them because these opinions either a) are not supposed to exist or b) are not supposed to matter. I think there are ways to produce accurate and fair reporting even through your own bias; I try to do it myself.

But those biases are there, and we all have opinions. There is nothing more frustrating, though, than when opinions and biases are passed off as scientific fact.

And that appears to be what is splashed across pages two and three of La Presse today. While some La Presse columnists have been sympathetic to the strike, their editorial pages have not. This does not de facto deligitimize their coverage of the strike. But it does raise some serious questions.

Today's article states the the Quebec population are 60 per cent in favor of the tuition fee increase, while 60 per cent also believe the government should negotiate with students. Maybe those numbers are correct, and that would be disappointing, but that's fine. The issue, though, is that La Presse and polling agency CROP can't say whether or not the numbers are correct, and that is highly problematic.

In the fine print of the infographic they have showing the poll results, they state that:

Méthodologie: Le sondage a été mené en ligne auprès de 800 répondants les 28 et 29 mars derniers. Les résulats ont été pondérés afin de refléter la distribution de la population adulte du Québec selon le sexe, l'âge, la région et la langue maternelle des répondants. Compte tenu du chractère non probabiliste de l'échantillon, le calcul de la marge d'erreur ne s'applique pas. [emphasis mine]

This means that 800 people were surveyed online; the results were sorted to reflect the Quebec population, but it is a non-probabilistic sample. That's really important, because it means they cannot state the margin of error within their results (normally written along the lines of: these results are accurate within plus or minus 2%, 19 times out of 20). The reason they cannot state the margin of error is because the sample does not allow them to accurately state that these numbers are representative of the Quebec population's opinions. They can only say with any certainty that these numbers reflect the opinion of their sample group.

According to Statistics Canada:

Reliability cannot be measured in non-probability sampling; the only way to address data quality is to compare some of the survey results with available information about the population. Still, there is no assurance that the estimates will meet an acceptable level of error. Statisticians are reluctant to use these methods because there is no way to measure the precision of the resulting sample.

I wouldn't argue that we should place much faith in many polls; statistics are useful tools, but are often twisted. But in this case, La Presse is using numbers that cannot be accurately verified to pronounce upon what is one of the largest questions facing Quebec society right now.

I don't know the opinions of the two journalists who wrote this piece, and I don't adhere to the theory that editors are breathing down reporters' necks, ensuring that they publish stories towing the party line. But that doesn't eliminate the questions in my head about whether a similar survey presenting different results, but using the same method, would have either been minimized or even pulled.

Maybe that's extreme though, and leaning towards conspiracy theory. There's another reason to be concerned though: it's lazy, inaccurate and I'd argue irresponsible journalism. According to Statistics Canada, one of the reasons researchers use non-probabilistic sampling is because it is "quick, inexpensive and convenient." So there's just as strong a case to be made that this is a situation of a newspaper chasing a story, going for a scoop, and publishing numbers and a major article, the validity of which cannot be ascertained.

La Presse should pull the story, and agree to no longer use such studies when reporting on major policy issues.

Want more grassroots coverage?
Join the Media Co-op today.
739 words
Join the media co-op today
Things the Media Co-op does: Support
Things the Media Co-op does: Report
Things the Media Co-op does: Network
Things the Media Co-op does: Educate
Things the Media Co-op does: Discover
Things the Media Co-op does: Cooperate
Things the Media Co-op does: Build
Things the Media Co-op does: Amplify

User login

Subscribe to the Dominion $25/year

The Media Co-op's flagship publication features in-depth reporting, original art, and the best grassroots news from across Canada and beyond. Sign up now!