It’s a tricky problem meant to exploit a common mental shortcut. A glance at the numbers leaves most people with the impression that the skin cream improved the rash. After all, more than twice as many people who used the skin cream saw their rash improve. But if you actually calculate the ratios the truth is just the opposite: about 25 percent of the people who used the skin cream saw their rashes worsen, compared to only about 16 percent of the people who didn’t use the skin cream.
This kind of problem is used in social science experiments to test people’s abilities to slow down and consider the evidence arrayed before them. It forces subjects to suppress their impulse to go with what looks right and instead do the difficult mental work of figuring out what is right.
In Kahan’s sample, most people failed. This was true for both liberals and conservatives. The exceptions, predictably, were the people who had shown themselves unusually good at math: they tended to get the problem right. These results support the Science Comprehension Thesis: the better subjects were at math, the more likely they were to stop, work through the evidence, and find the right answer.
But Kahan and his coauthors also drafted a politicized version of the problem. This version used the same numbers as the skin-cream question, but instead of being about skin creams, the narrative set-up focused on a proposal to ban people from carrying concealed handguns in public. The 2x2 box now compared crime data in the cities that banned handguns against crime data in the cities that didn’t. In some cases, the numbers, properly calculated, showed that the ban had worked to cut crime. In others, the numbers showed it had failed.
Presented with this problem a funny thing happened: how good subjects were at math stopped predicting how well they did on the test. Now it was ideology that drove the answers. Liberals were extremely good at solving the problem when doing so proved that gun-control legislation reduced crime. But when presented with the version of the problem that suggested gun control had failed, their math skills stopped mattering. They tended to get the problem wrong no matter how good they were at math. Conservatives exhibited the same pattern — just in reverse.
Being better at math didn’t just fail to help partisans converge on the right answer. It actually drove them further apart. Partisans with weak math skills were 25 percentage points likelier to get the answer right when it fit their ideology. Partisans with strong math skills were 45 percentage points likelier to get the answer right when it fit their ideology. The smarter the person is, the dumber politics can make them.
Consider how utterly insane that is: being better at math made partisans less likely to solve the problem correctly when solving the problem correctly meant betraying their political instincts. People weren’t reasoning to get the right answer; they were reasoning to get the answer that they wanted to be right.