Skip to content


How objectively do you suppose politicians evaluate data?

For politicians, the more data, the more they ignore

You can guess the result of a Danish experiment, but it’s still interesting.

https://arstechnica.com/science/2017/08/how-objectively-do-you-suppose-politicians-evaluate-data/

Snippets:

“Politicians aren’t the easiest group to study, but a group of researchers at Aarhus University, led by Martin Baekgaard, gave it a shot by sending surveys to Danish city councilors—a large enough group (950 people) for a meaningful study. The surveys (which were also filled out by a representative group of about 1,000 Danish citizens) were aimed at evaluating the subjects’ tendency toward motivated reasoning.

Motivated reasoning is what we all do with information that either confirms or threatens our previous opinion: we tend to buy into the confirming evidence and explain away any contradictory evidence. It would obviously be nice if politicians were particularly skilled at processing information objectively. But the researchers’ hypothesis (and probably your hypothesis right now) was that the politicians would fall short of being shining paragons of reason, just like the citizens they surveyed.

In the first experiment, survey subjects were asked to evaluate fictional customer-satisfaction-style data for parents rating schools or citizens rating road maintenance options. On some surveys (for the first example), schools were identified simply as “A” and “B.” But on the other surveys, one school was identified as public and the other as private.

This was the crux of the experiment, as the merits of government services vs. private suppliers is a source of lively political debate. Subjects were asked to evaluate which supplier was doing a better job based on the data provided. When this merely pitted A vs. B, politicians and citizens alike had little trouble identifying the winner in the data. But once the question became public vs. private, interpretations were skewed by how people felt about public vs. private suppliers in general.

If the data showed the public school getting better ratings, nearly all those who preferred public suppliers had no problem giving the correct answer, but half or more of the fans of private suppliers claimed the data showed the opposite. The same thing was true (in reverse) if the data leaned toward the private school.

Losing motivation

The next experiment is where things get a little weirder. A follow-up survey included the same sort of data for providers of post-surgery rehab, but with varying amounts of data. Some participants received data for patients in one type of rehab, some got three sets of data, and others got five sets. Research has shown that our motivated reasoning can hit a breaking point (in the right situation) when we have a large enough pile of unambiguous information in front of us. At some point, it becomes too obvious that we’re trying to explain away uncomfortable facts, and we process the information without ideological blinders.

We hope that politicians would do this, and the researchers’ hypothesis was actually that they would see this in the survey results. That is, motivated reasoning would be a little weaker for those given three and five sets of data that all pointed to the same conclusion.

But that is… not what happened. The politicians showed no improvement with additional information. If anything, they might have even done the opposite. Motivated reasoning was slightly stronger for the subjects who received more data. If the public provider (for example) got better ratings, those who preferred public providers were even more likely to give the correct answer when they had multiple data sets in front of them, while supporters of private providers were a tiny bit less likely to interpret the data correctly.”

The researchers also point to a study that suggests that highly partisan people will interpret questions about data as straight opinion questions when motivated to. (For example, when asked to assess economic trends, answers fundamentally reflect whether one’s political party is in power rather than the data provided.) And it’s worth noting that survey respondents knew the data they were given was fictional in this case, so perhaps they felt free to disregard the numbers they were asked to interpret.”

Stephen

Posted on: September 15, 2017, 6:00 am Category: Uncategorized

0 Responses

Stay in touch with the conversation, subscribe to the RSS feed for comments on this post.