Posted on May 20, 2014
Between July 2013 and February 2014, Humane League Labs carried out a large-scale study to determine what elements would make a pro-vegetarian booklet more effective at inspiring young people to reduce their consumption of animal products. This study examined:
- whether it is more effective to focus primarily (three out of four relevant pages) on the cruelty done to farm animals, or to focus primarily on the health benefits of eating vegetarian (cruelty vs. health)
- whether it is more effective to discuss all common farm animals (pigs, cows, and chickens), or to discuss only chickens (all animals vs. chickens)
- whether it is more effective to focus primarily (four out of six relevant pages) on the reasons why to go vegetarian, or to focus primarily on how to eat vegetarian (why vs. how)
The study was carried out as a full factorial study, meaning a booklet with each possible combination of the above three variables was created and used in the study, for a total of eight booklets. As a result, in addition to providing general answers to the above questions, the study also suggested which combination of elements might make for the most persuasive leaflet.
Young individuals were approached and asked to fill out a survey on food choices. A total of 3,233 individuals agreed to take part. Those individuals then filled out a questionnaire on how often they ate red meat, chicken, fish, and eggs. They were then provided with one of eight different eight-page printed booklets about vegetarian eating, modeled after Farm Sanctuary’s Something Better vegetarian advocacy booklet. After reading the booklet for as long or short a period as they desired, participants were asked to fill out another survey asking how much they anticipated they would eat red meat, chicken, fish, and eggs in the future. A control group of participants filled out both surveys but was not provided with any booklet.
Approximately 40% of participants were surveyed at the Warped Tour Music Festival at a variety of cities in the northeastern U.S. The remainder of participants were surveyed at colleges and universities in the Boston, Philadelphia, and Dallas regions.
Three months after receiving the leaflet and filling out the initial surveys, all participants were emailed and called with a follow-up survey to measure how frequently they now ate red meat, chicken, fish, and eggs. They were offered a chance to win a $500 gift card if they completed the follow-up survey. A total of 569 participants completed the three-month follow-up survey.
Data was analyzed to measure how much each participant’s diet had changed between the time they filled out the first survey (prior to seeing a booklet) and when they filled out the follow-up survey three months later. Results were then translated into the number of farm animals spared per year by the reported diet change.
There was a sharp difference between the overall impact of different types of booklets:
- Cruelty vs. Health – This category made the least difference overall. Those who received a booklet that was primarily cruelty-focused reported diet changes that spared slightly more animals than those who received a primarily health-focused booklet (3.27 animals spared per cruelty-focused booklet vs. 2.94 animals spared per health-focused booklet).
- All Animals vs. Chickens – This category made the greatest difference overall. Those who received a booklet that discussed pigs, cows, and chickens reported diet changes that spared 150% more animals than those who received a booklet that only discussed chickens (4.41 animals spared per all animals booklet vs. 1.79 animals spared per chickens-only booklet).
- Why Focused vs. How Focused – This category made a significant difference. Those who received a booklet that focused primarily on how to go vegetarian reported diet changes that spared 50% more animals than those who received a booklet that focused primarily on why to go vegetarian (3.75 animals spared per how-focused booklet vs. 2.45 animals spared per why-focused booklet).
The data for the individual booklets is not reliable due to small sample sizes (45-95 participants per booklet completed the three month follow-up survey). Even so, the success of the individual booklets seemed to more or less conform with the above results.
The two best-performing booklets were:
- Booklet I – how-focused, all animals, cruelty-focused (6.44 animals spared)
- Booklet K – why-focused, all animals, health-focused (5.56 animals spared)
The two worst-performing booklets were:
- Booklet L – why-focused, just chickens, health-focused (1.24 animals spared)
- Booklet H – why-focused, just chickens, cruelty-focused (-1.13 animals spared)
Once the (smart) decision has been made to use booklets that feature all farm animals, the impact of the other elements measured in the study changes significantly. Among booklets that featured all animals:
- Why-focused booklets actually performed slightly better on average than how-focused booklets (4.85 animals spared per why-focused booklet vs. 3.97 animals spared per how-focused booklet)
- Cruelty-focused booklets significantly outperformed health-focused booklets (5.29 animals spared per cruelty-focused booklet vs. 3.53 animals spared per health-focused booklet)
These numbers are not completely reliable because of their smaller sample size (approximately 150 respondents in each group).
When the results for college-age individuals (those 18-22) are viewed on their own, across the three main questions the results are roughly the same as they are for participants overall. The only differences were that focusing on health may have been slightly more effective than focusing on cruelty (1.1 animals spared vs. .77 animals spared); and that the difference between focusing on the how and focusing on the why was even wider (1.95 animals spared for how-focused booklets vs. -.59 animals spared for why-focused booklets). Focusing on all animals continued to be much more effective than focusing on just chickens (2.25 animals spared vs. -.89 animals spared). However, the number of participants aged 18-22 who completed the three month follow-up survey is too small for any of these college-specific figures to be very reliable.
High School Students
High school students (defined in this study as those age 13-17) reported a much higher intent to change after viewing the booklet than those of college age. They also reported much higher actual change in the three month follow-up survey compared to those of college age. However, we cannot be sure that the booklet actually caused more diet change in high school students than in college-age individuals. For many high school students in this study there was a confounding factor in that many of them received a second vegetarian advocacy booklet later in the day, several hours after completing the first two parts of the study. (This was also a confounding factor for some college-age students, but for a smaller percentage of them than for high school students). Further, the sample size of high school students who completed the three month follow-up survey is too small to be reliable.
Nevertheless, the fact that high school students reported a much higher intent to change than those of college age (4.78 animals spared vs. 1.92 animals spared) indicates that they may be significantly more impacted by pro-vegetarian booklets.
The results of this study provide strong evidence that pro-vegetarian booklets aimed at young people should focus on all main farm animals, not just chickens. The study also provides weak evidence that booklets should focus more on the cruelty done to animals than on the health benefits of going vegetarian. Once the above two elements have been put into place, it is unclear whether how- or why-focused booklets will be more effective at persuading people to change their diet. The single most effective booklet in the study discussed all farm animals, was cruelty-focused, and was how-focused. However the sample sizes for the individual booklets were not large enough for us to know whether this was actually the most effective booklet.
It is possible that taking a similar approach (focusing on all animals and on the cruelty done to farm animals) will also make other youth-oriented vegetarian advocacy materials like videos more effective at inspiring diet change, but we cannot know without direct testing.
Postscript: Intent To Change vs. Actual Change
Intent to change (as measured by the survey that participants filled out immediately after seeing the booklet) more or less corresponded with actual diet change at the level of individual booklets. Three of the four booklets that performed best in intent to change remained among the top four booklets for actual change. And three of the four booklets that performed worst in intent to change remained among the four worst booklets for actual change.
Interestingly though, when collapsed into categories (cruelty vs. health, all animals vs. chickens, why vs. how), intent to change was a terrible predictor of actual change. For example, those who received a booklet with only chickens reported just as much intent to change as those who received a booklet with all animals, but the latter group ended up sparing more than twice as many animals with their diet changes. Similarly, those who received a booklet focused primarily on why to go vegetarian reported they had dramatically more of an intent to change than those who received a booklet focused primarily on how to go vegetarian; yet three months later the latter group ended up sparing 50% more animals. And those who got a booklet focused on cruelty to animals reported intended changes that would have spared 3.5 times more animals than those who got a booklet focused on the health benefits of eating vegetarian. However three month later, these groups had changed their diets in equal amounts.
It’s unclear why intent to change and actual change corresponded fairly well on the booklet level, but did not correspond at all on the group level. One important takeaway is that intent to change does not necessarily line up with actual change. Therefore follow-up surveys should be done whenever possible, as they were in this study.
One other anomaly found in the study was that, in the three month follow-up study, those in the control group (those who never received a booklet ) reported more of a reduction in animal product consumption than those who received any of the other booklets. We do not believe that booklets promoting vegetarian eating actually decrease people’s likelihood of changing their diet. Other studies which intentionally examined this question have found that people who receive a pro-vegetarian booklets are more likely to reduce their meat consumption than those in a control group. We believe the change reported by the control group in our study is unreliable due to the very small sample size of the control group; only 45 members of the control group filled out the three month follow-up survey, leaving a huge margin of error. Furthermore, because surveys were often filled out by participants who were sitting next to one another and who were friends, and because participants were usually given the booklet to keep, those in the control group probably looked at one of their friends’ booklets after the group of friends had individually completed their initial two surveys.
In any case, this statistical anomaly does not impact the main questions investigated by the study.
|Document (with link)||Description|
|Data set||All respondents|
|Booklet G||(why, all animals, cruelty)|
|Booklet H||(why, chickens, cruelty)|
|Booklet I||(how, all animals, cruelty)|
|Booklet J||(how, chickens, cruelty)|
|Booklet K||(why, all animals, health)|
|Booklet L||(why, chicken, health)|
|Booklet M||(how, all animals, health)|
|Booklet N||(how, chicken, health)|
|Survey||Pre- and Post-Exposure|
|Three-month follow-up survey||Email version|
All data is free to use and share.