Attention conservation notice: 3000 words about how smart people who ought to know better are reading way too much into a poll.

Last May, NCSE reported on a poll on evolution conducted by Virginia Commonwealth University. The results were, to my eye, in line with most of the other polling out there, so I never wrote about it here, other than a passing mention in a post about my WaPo review of Elaine Howard Ecklund's Science vs. Religion.

Anyway, a month and a half later, Jerry Coyne and Jason Rosenhouse have discovered the poll, and each has found different questions in the poll that they find worthy of comment. Before we delve into this, I'll suggest that folks read professional pollster George Bishop's essay on the vagaries of polls on evolution and creationism, and the pitfalls to beware.

Polling data on evolution from Gallup and VCU surveys: Paste this image in your site, Myspace, Facebook, Ebay copy this code...

<a href="http://www.weeklybits.com/polling-creation-2fevolution_article_11_74723.html"><img src="http://scienceblogs.com/tfk/images/gallupandvcupolls.png" border="0" alt="Polling data on evolution from Gallup and VCU surveys en Tech WeBlogs WEEKLYBITS.COM"></a>

Which of these statements comes closest to your views on the origin of biological life: biological life developed over time from simple substances, but God guided this process, biological life developed over time from simple substances but God did not guide this process, God directly created biological life in its present form at one point in time?

These results are nicely in line with historic answers to Gallup's surveys. For the last 30 years, Gallup has asked the similar question:

Which of the following statements comes closest to your views on the origin and development of human beings? (1) Human beings have developed over millions of years from less advanced forms of life, but God guided this process. (2) Human beings have developed over millions of years from less advanced forms of life, but God had no part in this process. (3) God created human beings pretty much in their present form at one time within the last 10,000 years or so.

As Jason notes, the big differences are VCU's significantly higher support for "did not guide" than Gallups for "had no part." It's a 9 point difference between VCU's result and Gallup's latest poll, and a 6.5 point jump from the long-term Gallup average. The "God guided" option took the hit in VCU's poll relative to Gallup, as the "directly created" is basically unchanged.

Jason looks at that difference and tries to develop it into a falsification of accommodationism (a term which I've noted could do with a clear definition some day). This is a deeply unsatisfying argument, as it glosses over some pretty important issues.

Not least among the problems, I don't know that anyone from the "accommodationist" camp has ever proposed the hypothesis Jason is testing. Without some clearer citation, it's hard to know if this is a legitimate extension of ? I'm guessing ? Chris Mooney's writings. I can see where someone might get it, but I don't know that Chris has ever claimed explicitly that New Atheism will actively decrease public acceptance of evolution or increase acceptance of creationism. I certainly know I haven't made that claim. Most significantly, I wouldn't claim that because I think the main effect of that fracas is to simply turn people away from experts whose opinion they might otherwise listen to. This could swell the ranks of the undecided, but I doubt it will move many people out of that broad middle. My concern, and what I understand Chris (and his co-authors Matt Nisbet and Sheril Kirshenbaum, at various times) to have principally argued, is that the New Atheism is likely to turn people away from folks they'd otherwise regard as trustworthy experts, and that this could hold back ongoing outreach efforts. These data don't really allow testing of that hypothesis. (I've hedged my language here because it may be that there is such an explicit prediction, and this paragraph would be moot.)

The more significant error I see in Jason's post is that he's comparing apples and oranges. Gallup asks about human evolution, VCU asks about "biological life." Framing the question in terms of humans tends to reduce support for evolution while framing it in terms of all life tends to boost support for evolution. I don't know of other polls using the "biological life" phrase, but I'd imagine that mentioning biology would also cue respondents to answer in terms of their understanding of the science, rather than their personal beliefs.

As I noted in Jason's comments, Harris tends to ask a question about human common ancestry in parallel with a question about common ancestry of all life, which lets us get a feel for the magnitude of that effect. The most recent such survey found:

In reply to one question almost half (45%) of adults say they believe humans were created directly by God and only 29% say they evolved from other species. In reply to another question 53% of these same people say they believe that ?plants, animals and human beings have evolved over time,? and only 21% say they do not believe this, with fully 25% who are not sure or decline to answer.

Furthermore, Jason compared VCU's question in 2010 to Gallup's from 1998. But Gallup has been asking the question for 28 years. So why compare Gallup 1998 to VCU 2010? When we look at Gallup's results, we find a fairly steady increase in acceptance of "God had no part," from 9% in 1982 to 14% in 2008. But the Gallup poll's margin of error is 3 points, making it statistically impossible to say that the 1982 result is different from the 2008. I could probably do a time series analysis to figure out if the trend from all 9 data points is statistically significant, but it doesn't matter for our purposes here.

It doesn't matter because the rise is fairly steady over time, with no obvious inflection when the "New Atheists" started publishing. Since 2004, when Harris published The End of Faith and not long before Dawkins published The God Delusion, support for "God had no part" has moved from 13% to 14%. Since 1999 (a low outlier from the general trend, selected by Jason for no obvious reason), it has risen 4 points, but since the Four Horsemen began their march, it's risen 1 point, statistically no different than noise. If there's an inflection point ? and I don't think statistical analysis would find one ? it would be between August 1999 and February 2001, with (if anything) a flattening since 2004. If there's a statistically significant trend to the data at all, though, I don't think there'd be statistically significant inflection points. Certainly not from 9 data points (6 in Jason's discussion).

Finally, Jason should have compared VCU's 2010 poll with their poll from September, 2005. I plotted those results above, and you can clearly see that the big shift was not in the "directly created" line (which moved only one point, as did "God did not guide"). If "New Atheists" had an effect on "God did not guide," you'd expect to see some of that signal since 2005, but you don't. Not in Gallup, not in VCU. In VCU, but not in Gallup, we see a sizeable jump in the "guided by God" line, an 8 point shift, mostly switching from the large undecided/decline to answer category in the 2005 poll. If this tells us anything about accommodationism, it says that when you push people without strong feelings to weigh in on evolution, they'll tend to pick the "God guided" option if it's available. It'd be interesting to have data where a similarly ambivalent population was forced to make a binary choice, and see how many break for "God created" versus "evolution only." I would guess they'd pick "God only."

In short, there's simply nothing in Gallup's data, or in an informed comparison with VCU's data, that supports Jason's claimed "doubling in ten years," nor is there justification for his attempt to attribute causality, one way or another, to the rise of the "New Atheists." Jason is a mathematician, the author of a good popular book on statistics, and is working on a book that will (among other things) examine public polling on creationism and evolution. I'm confident he'd never try to read causality or claim both a trend and an inflection point based on 9 data points in his professional writing, nor would he claim equivalence between differently worded questions from different pollsters, and I don't know why he's doing it here.

Turning to Jerry Coyne, we find a different question being discussed. He's interested in this question:

In general, would you say the theory of evolution conflicts with your own religious beliefs, or is mostly compatible with your own religious beliefs?

Anyway, Coyne takes the VCU result and opines:

The large chunk who see conflict is bad news for accommodationists. But the accommodationist response?at least that of the National Academy of Sciences, the National Center for Science Education, and the American Association for the Advancement of Science?is this: You don?t understand your own faith, because if you did, you would see that there?s really no conflict. They have a big theological task in front of them.

To the first point, then, Coyne could be making either of two arguments. Either a) it's bad news because when 42% of people see a conflict, it means that there really must really be a conflict (while when 43% say there's no conflict, it means there could still be a conflict) or b) it's bad news because there are so many people who have to be convinced that evolution is not incompatible with their religion. Option a makes little enough sense that I'll assume he means option b. And yes, it is a problem, but it isn't news (see the Pew survey from a year earlier). Furthermore, it's a problem of exactly the same proportion as the one facing anyone who, like Coyne, wants to convince Americans that evolution is incompatible with their religion, not to mention the further challenge such a person would face of having to get people, once they accept that conflict, to reject religion and accept evolution. In what sense is the news worse for "accommodationists" than for the "New Atheists" and creationists trying to woo that 43% while squabbling over the 42% who already see a conflict?

As for the second objection, many members of religious groups that have officially endorsed evolution still believe that their religion is incompatible with evolution. It seems reasonable to tell such people that they are at odds with their own church's doctrine. That's not insulting, and it isn't a theological task. It's an educational task. Telling someone that the Methodists have endorsed evolution is no more advocacy for Methodist theology than telling them that Methodists believe Jesus is their Lord and Savior. One can disagree with either belief while still accurately informing people that those are the official views of United Methodism. Given that most people cannot identify Genesis as the first book in the Bible and 2/3 know that Jesus delivered the Sermon on the Mount (and I can), I feel comfortable telling people about the basic doctrines of those denominations whose leadership has issued clear statements in support of evolution. Where my own knowledge gets thin, I can rely on advice from a trained theologian on NCSE's staff, our Director of Religious Community Outreach.

To the third point, I'll note that these public polls are less informative about the truth of this matter than a review of the number of religious scientists, a statistic Coyne has derided. There's no question that it signals a problem when 31% of nonscientists think scientists have doubts about evolution, but it does not signal that those people are right. If you want to understand a technical topic, you consult relevant experts. In the case of evolution, you talk to biologists, and get near unanimity about the central role of evolution in biology. In the case of theology, you talk to theologians, and in the case of the intersection of science and religion, you talk to people who study science and religion, who agree that there need not be conflict (though conflict is, of course, possible).

And the fourth point speaks for itself. Why is this bad news for "accommodationists" but not for "New Atheists"? The numbers are nearly identical, so it would seem that the bad news is symmetrical as well.

Then Coyne makes an argument that never ceases to amaze me when I find it in the wild:

How do we solve the problem? Many scientists?atheists and accommodationists alike?are trying to educate people about what evolution is and how much evidence supports it. Sadly, that doesn?t seem to be working very well, at least judging by how little acceptance of evolution has budged over the last few decades.

What other effects? Answers in Genesis has a budget that dwarfs NCSE's, and AiG is only about 2/3 of the creationist marketplace. Toss in funding to natural history museums, NSF educational funding and funding for "broader impacts" of research grants, and more may be spent on evolution outreach than on creationist outreach, but creationists also have a lot of volunteers, and a lot of churches and other groups promoting creationism outside of official channels. Note also that creationist funding seems to be rising more rapidly than NCSE's funding.

We also know that 1 in 6 high school biology teachers are creationist, and 1 in 8 of those teachers spend at least an hour advocating creationism. Teachers are rarely required to have a college class in evolution, and those who do not have such a class are less likely to spend time on evolution. These are all serious issues, and there's no way that someone as versed in experimental design and data analysis as I know Coyne to be would ignore those. The science education efforts Coyne describes are not happening in a vacuum, and any evaluation of their success given public polling data has to account for everything else happening in society.

Given what we'll call a rough parity of concerted effort, is it surprising that there's little to no trend in the polls? No. All it means is that science education efforts are canceling out the effects of creationist efforts. Note also that polls today will not reflect data from students currently in school, and that there's far less outreach to adults than to school children. This would introduce a significant lag into any signal one might find in the polls. Assuming kids take high school biology in 9th grade, when they're 14, they are unlikely to be polled until they're 18, so that's a minimum 4 year lag, and longer until enough students reach the adult population to show up in polls to significant degrees. Naively waving poll numbers around without any sort of mention or consideration of these factors is not the solution.

Now I'm sure that someone with the right data and some serious statistical models could tease out these effects. I don't know that the right data exist at all, but maybe the GSS have data with which we could isolate a signal and identify the effects of creationist outreach vs. scientific outreach, and see what works and what doesn't. But Jerry cites no such research, he simply asserts that a flat poll line means that what is currently being done is not working. I know he wouldn't approach data from an uncontrolled field study so blithely, and I can't imagine why he'd treat these data that way.