“Study 2 was a field experiment in which we posted ads to a social-networking Web site and recorded click-through rates….
We launched six advertisements for 1 day each on a
social-networking Web site in a 2 (God vs. no God) × 3
(immoral risk vs. nonmoral risk vs. no risk) design. The
ads ran 452,051 times on accounts registered to users
over 18 years of age residing in the United States.”
Here’s what their ads looked like:
They further explain their rationales for the blurbs:
“We expected that the relevance of risk to skydiving would be self-evident,
but we emphasized the risky nature of engaging in bribery to ensure we were capturing risk-relevant behavior.”
Which is a hell of an assumption in their research. People may assume skydiving is risky, but until they get some decent data backing that up, they are making the proverbial ass out of u and me. And people don’t think bribery is risky, so they need reminded? So they’re comparing something which is explicitly labeled a risk versus something they assume people thing is risky and are judging it as if it’s apples to oranges.
“We subtly manipulated the salience of God by
stating either “God knows what you’re missing!” or “You
don’t know what you’re missing!” at the beginning of
each ad.”
So apparently by salience they mean literally saying. That’s a pretty damn liberal use of the term. Too liberal, in fact. Bernie Sanders levels of liberal. To the point where I’m calling it out. They’re using an idiom “God knows”. I’m am a full blown nihilist and I use that idiom. At a conscious level, it has zero salience for me. And I’d wager I’m not alone. And I realise they can argue for subconscious effects, but without knowing to what extent the subconscious picks up on neutral idioms, they’re argument is useless. At this point, they are essentially determining the effect of phrasing on advertising response.
Which is all well and good, but that has nothing to do with God and risk anymore.
“We computed a binary logistic regression model to pre-
dict clicking behavior using priming condition (God = 1,
control = 0), activity condition (dummy coded with video
games as the comparison condition), and Priming
Condition × Activity interaction terms for each of the
dummy-coded activity variables.”
If you don’t know what logistic regression is, you can try to read about it on Wikipedia, but it’s probably not going to clear a lot up. This information is pretty much completely useless to anyone without a masters in statistics, and it’s a problem endemic in the sciences. Again, most scientists know fuck-all about statistics beyond the basics. What they care about are p values < .05, which flags statistical significance (which is not necessarily, let’s say, salient). This is there because that’s the way the industry works. “Look at all this complicated math! We’re smart! We know way more than you. Trust us. We proved it with math.”
An easier visualization of the data is their figure 3:
Much better. Data appropriate for kids and adults of all ages!
Let me first draw your attention to the y-axis. The click-through rate is simply how many people on a site click on an ad. Take a look at those numbers and the bars. Their most popular ad had a click-through rate of .08%. That is, only 8 in 10,000 people clicked on it. The take away from this is “damn near nobody clicks on those stupid adds on social media.” Society isn’t as stupid as we think? So if you’ve ever wondered aloud “who the hell clicks on this stupid crap!?”, it’s less than 1 in a thousand people. But those assholes are ruining it for the rest of us. This is well known in the industry, so it’s not as if this study can publish that as a novel finding. But I don’t think it’s exactly common knowledge either.
And in the advertising world, getting 8 in 10,000 clickers up to 16 in 10,000 is potentially a big deal. So it makes sense for business majors to be looking into it. What it doesn’t do a good job of doing is telling us anything about society at large. The people who click ads are major outliers. Speaking in absolute terms, not change in percentages, basically nobody gives a shit about ads. That’s what their data overwhelmingly show. For literally more than 99.9% of the population, it doesn’t make a difference whether or not you mention God in an ad. Mentioning God in a skydiving ad increases the click rate from 3 in 10,000 to 6 in 10,000. That is a statistically significant change, because they aren’t comparing clicking with not clicking, they’re comparing clicking column A with clicking column B, and not-clicking was ignored. But when you’re talking about so few people, it seems basically meaningless.
To reiterate, statistically significant does not mean “meaningful”.
They also show the click-through rate dropping from 5 to 1 in 10,000 for the bribery component. This is used as a corroboration for their assertion in the introduction that studies show a link between moral behavior and religious verbiage.
This study makes several major assumptions:
Literally mentioning God at all means that this has a salient effect on people.
Non-psychologically-salient phrasing has no effect on advertising effectiveness
Skydiving is inherently risky, therefore it is not necessary to mention risk when presenting it as a risky behavior.
Skydiving is a fair model for all “nonmoral” risky behaviors.
Crimes are not considered inherently risky, so it must be mentioned explicitly.
A difference in fewer than 10 in 10,000 people is a meaningful segment of the population from which to extrapolate psychological analysis for the population at large.
God forbid you have a hang-up on any one of those.