Correctly Sizing Up “Concern for the Environment” by American Teens

By David Wigder
Vice President / Director, Digitas

This week, Jupiter Research released the results of their latest online survey: 38% of online teens are “concerned about the environment.” Interestingly,
MediaPost reported that JWT’s March 2007 survey indicated a much higher
response by online teens: “more than 80% of American teenagers are
concerned about the environment and the role of the United States that
is causing pollution”. 

At first glance, it seems that these two companies have published dramatically dissimilar responses to a similar audience. (“Green Teens”, Jupiter Research, July 13, 2007; “Advertisers: Teens Value Environment, Buy From Socially Responsible Companies“, MediaPost, March 20, 2007)

If that were the case then how did “concern for the environment” by American teens fall so dramatically in four months? How do marketers and strategists interpret seemingly disparate results from a seemingly similar online audiences?

It is presumed that both surveys
generated statistically significant results, though likely with
different levels of confidence as JWT surveyed 767 teens to Jupiter’s
2,091. Moreover, Jupiter polled teens ages 13-17, while JWT polled teens 13-19. But,
it is reasonable to assume that this does not account for the
dramatically different survey responses as the mean age for teens
polled by JWT was 14.6 years old.

The simple answer may be that survey questions and the reporting of survey results can sometimes be misleading. JWT’s survey provides a representative case study. Here is the timeline:

1) In mid-March JWT and
RelightNY, an organization trying to “educate and inspire” people to
take action to protect the environment, released a survey in which it
reported that “Nationwide, 79% of teens are bothered by the fact that
America has emerged as the world’s leading pollution source.” (“Ten Stats about Teens and the Environment”, March, 2007; Advertisers: Teens Value Environment, Buy From Socially Responsible Companies)

One interpretation of this statement is that the US has only recently become (“emerged” as) the leading polluter in the world.  As it is worded, the question introduces a bias by implying that the US has changed its status and is now the world’s worst polluter. This
is likely to have influenced more teens to respond affirmatively to the
question than if the question had simply asked teens if it “bothered
them that the America was the world’s leading pollution source.”

 2) On March 19, PRNewswire published a story entitled “JWT Survey Reveals Four of Five U.S. Teens Share Concern for the Environment”. In the body of the text, PRNewswire clarified the survey results (and their headline) stating that “Over 80% of American teens are bothered by the fact that the U.S. represents one of the world’s leading sources of pollution”.  

rewording the question in the article title as “concern for the
environment”, PRNewswire effectively changed how readers interpret the
survey results (and especially for those who only scanned headlines). Only below in the copy did PRNewswire clarify that the question was referring to the US as a major polluter. Moreover,
note the subtle but effective changes in wording of the original survey
results by PRNewswire that makes the high level of response by teens
all the more dramatic: “Over 80%” vs. “79%” and “represents one of the
world’s leading sources of pollution” vs. “the world’s leading
pollution source”.   

3) On March 20, MediaPost filed its story on JWT’s survey.  In this
article, JWT writes that its “online study discovered that more than
80% of American teenagers are concerned about the environment and the
role of the United States in causing pollution”.

This rephrasing of the original survey question can be interpreted in several ways, albeit incorrectly in each case. First, it implies that the original question included the phrase “concern for the environment” which it did not. Perhaps MediaPost incorrectly repurposes wording from PRNewswire’s story the previous day.

Second, it shifts the emphasis of
the question towards “concern for the environment” (because it is read
first in sequence) rather than what was actually stated in the original
question regarding pollution.  Finally, it decouples
“concern for the environment” and “the role of the United States in
causing pollution,” implying that they are two separate survey
responses and that teens responded equally to each of them.

Again, you see subtle rephrasing of survey questions or perhaps again incorrect repurposing of PRNewswire’s story. MediaPost state that “more than 80%” of teens responded affirmatively to the survey question vs. “79%”.

The lessons are clear: surveys and the reporting of survey results can be misleading. Marketers
should be weary of very high (and low) responses to questions (or any
result that does not pass the gut check), as they are often a sign that
the question was leading or unclear to respondents.

Moreover, marketers should always go back to the original source to review questions and responses first-hand. Try to understand in what context the question was asked and how it could have been interpreted by respondents. Finally, understand at what confidence level are the results valid.

In a
world where environmentalists have been accused of inflating statistics
to bolster their case, it is an imperative that this data is gathered,
reported and interpreted accurately. Our understanding of consumer attitudes on the environment depends on it.

No comments yet... Be the first to leave a reply!

Leave a Reply