Survey Says……The Lake Research Partners Survey on Horse Slaughter is Righteous!

Standard

Written by Heather Clemenceau

The now infamous Lake Research Partners survey on horse slaughter in the United States, conducted on behalf of the ASPCA, shows that voters oppose the slaughter of American horses for human consumption overwhelmingly. The opposition is strong among those who own horses and those who do not, as well as across every key demographic and geographic group, and across political party lines.

Check the survey itself to see how it is stratified across the various demographics.  What does it really mean?  Well,  to start off,  it should be known that surveys conducted by professional research firms are designed and interpreted by statisticians and social scientists,  and unless we’re talking about a complete census,  they all use some form of random sampling.

1)  What makes a survey such as this more valuable than one by Survey Monkey installed on a website?

The fact that it is randomized and not placed on a website where only people visiting the website can see it and vote.  The ability to vote multiple types is eliminated because only one phone call is made and only one vote can be cast.  Survey Monkey is simply a “quick and dirty” way for marketers to ask questions,  often in direct mail campaigns.  They are only as accurate as their distribution methods.  On the other hand,  surveys conducted by professional research firms are used to:

  • Help TV networks decide what people are watching particular shows,  and therefore where they can charge the most advertising
  • Help Statistics Canada study educational and other needs
  • Assist Auto manufacturers in finding out out how satisfied people are with their cars
  • Aid Census groups to obtain information on employment and unemployment
  • Provide information for new businesses who purchase aggregate data,  to decide where to locate their business based on where their markets lie.

2)  Why do research firms limit themselves to samples of around 1,000 people?

They don’t – they will conduct as elaborate and expensive a survey as their client wishes to pay for.  So unless that client is the government,  which has very deep pockets,  there is going to be a budgetted expenditure.  Included in the survey cost will be such things as the labour and material costs for designing,  testing,  meeting with the client to define needs,  printing the questionnaire,  providing pre-stamped return envelopes or budget for phone costs (in the case of this survey,  which was a phone survey of registered voters).

Once the information is returned or collected if a phone survey,  it must be recorded and analyzed by qualified individuals.  A survey of around 1,000 people is both quick and economical – the well-known national polls frequently use samples of about 1,000 persons to get reasonable information about national attitudes and opinions.

3)  How can a survey of 1,000 people provide a true representation of an entire country’s view?

No survey,  unless it is  a complete census (a complete survey of 100% of the population – EXPENSIVE and TIME CONSUMING)  can depict the true population sentiment.  Analysts find that a properly randomized survey with  a moderate sample size is statistically significant. The sample size required for a survey partly depends on the statistical quality needed for survey findings; this, in turn, relates to how the results will be used.

4)  How can I be sure the question isn’t leading people to the answer?

In this case,  the question is straight-forward – “Would you say you approve or disapprove of ALLOWING American horses to be slaughtered for human consumption? [IF APPROVE/DISAPPROVE]: Do you feel that way strongly or just somewhat strongly?[IF UNDECIDED]: Well, which way do you lean?”

The question doesn’t ask you if you “…..approve or disapprove of allowing American horses to be shipped long distances and slaughtered inhumanely….,”  or “wouldn’t you say it’s about time we ended slaughter?” which would also be a leading question.  Phrasing an opinion question this way leads the respondent to a “yes” answer and a distorted or biased perspective of the public’s views on the issue.  Also note that the question is fairly short and uncomplicated.   People are more likely to cooperate if the questions are simple, clear, easy to answer, and personally relevant to them.  Most surveys are written at a grade-school level as well, for simplicity’s sake.

5)  How are the survey respondents selected?

In a bona fide survey (which is not a Survey Monkey offering), the sample is not selected haphazardly or only from persons who volunteer to participate. It is scientifically chosen so that each person in the population will have a measurable chance of selection.   This way, the results can be reliably projected from the sample to the larger population.  This survey started with registered voters as its base,  so if a lot of either pro or anti-slaughter advocates are unregistered,  they would never be contacted.  Quite possibly a random dialer was used to reach the respondents.

6)  What errors can confound (produce false results) in a survey?

This survey had a 3.09% margin of error (MOE). That means that if you asked a question from this poll 100 times, 95 (or whatever the degree of confidence calculated for a survey,  usually 95-97%) of those times the percentage of people giving a particular answer would be within 3.09 points of the percentage who gave that same answer in this poll – so the poll is likely to only vary either upwards or downwards of the 80% by 3.09 percentage points.   If you conduct the same poll 100 times you are not going to see a complete reversal in the trend either.   In other words,   there would be relatively little variation no matter how many times you conducted the same survey,  because it has been effectively randomized to include people from various socio-economic climates,  political affiliations, etc. etc.  Why 95 times out of 100? In reality, the margin of error is what statisticians referred to as a confidence interval. The math behind it is much like the math behind the standard deviation. So you can think of the margin of error at the 95 percent confidence interval as being equal to two standard deviations in your polling sample.

7)  What effects the Margin of Error?

  • Sample size
  • Type of Sampling
  • Size of the Sample Population

Sample size is important to avoid negatively impacting the margin of error. In sampling,  a sample of 100 will produce a margin of error of around 10 percent, a sample of error of 500 will produce a margin of error of no more than about 4.5 percent, and a sample of size 1,000 will produce a margin of error of no more than about 3.2 percent.  Of course, this observation is consistent with the MOE we see in the Lake Research sample – 3.09%.  This illustrates that there are diminishing returns when trying to reduce the margin of error by increasing the sample size.  In order to  reduce the margin of error to 1.5%,  the research firm would require a sample size of well over 4,000,  which of course increases time and cost to the client.

The type of sampling affects the MOE because the survey designer can control the design or the survey.  For example,  if the phone respondents were not adequately randomized,  or the survey designer elected to call people only from his or her regional phone directory,  that would be consistent with a poorly designed survey because it wouldn’t be random, and it would exclude people with unpublished or unlisted phone numbers.  By randomizing the phone numbers,  you can still contact people with unlisted phone numbers,  but not people without phones of course!

The sample population is merely the total number of individuals from whom to choose for your survey or poll.  Proper sampling designs involve defining groups, or strata, based on characteristics known for everyone in the population, and then taking independent samples within each stratum.

Another type of statistical error that can confound a survey is “non-sampling error.”  Not everyone will respond to a survey,  or sometimes they won’t tell the truth.  But the estimate will probably still differ from the true value to some degree,  even if all non-sampling errors could be omitted. Unless a large number of respondents decided to lie,  this would not substantially affect the survey.

Lake Research Partners Poll for ASPCA - Page 1

Lake Research Partners Poll for ASPCA – Page 1

Lake Research Partners Poll for ASPCA - Page 2

Lake Research Partners Poll for ASPCA – Page 2

Lake Research Partners Poll for ASPCA - Page 3

Lake Research Partners Poll for ASPCA – Page 3

About these ads

About heatherclemenceau

Hopefully as I've grown older I've also grown wiser, but one thing I've definitely become cognizant of is the difference between making a living and making a life. Frequently outraged by some of life's cruelties, and respect diversity. But.....I don't suffer fools gladly, and occasionally, this does get me into some trouble! I have the distinction of being the world's worst golfer - no wait, I do believe that there is a gypsy in Moldavia who is a worse golfer than I. Nor am I much of a dancer - you won't see a booty-shakin' flygirl routine from me! I'm also not the kind of cook who can whip up a five-course meal on a radiator either! And I've never figured out how to get an orchid to bloom a second time. I love to discuss literature, science, philosophy, and sci-fi , or even why Seinfeld is funny on so many levels. Words move me. I'm very soft-hearted about most things, especially animals, but I have a stoicism about me that is sometimes interpreted incorrectly. I do have a definite edge and an often "retro-adolescent" sense of humour at times. I'm a big advocate of distributed computing projects to advance science. Check out http://boinc.berkeley.edu/ if you want to find out more. I'm an eclectic (but not crazy) vegetarian, and as such, it's a personal practice of mine to seduce innocent meat-eaters into cruising the (salad) bars at every opportunity. You would be powerless to resist. I was recently surprised to find that a computer algorithm concluded that I write like Dan Brown, which is funny because I didn't think Dan Brown could actually write. Check out your own style - http://iwl.me/ Oh, and I love impractical shoes and funky hats.

17 responses »

  1. Thank you Heather, I’m sure they still will not understand. One has to have a modicum of intelligence to comprehend the basic principles of how surveys work, something many of them are sorely lacking.

  2. Pingback: Survey Says…… « heatherclemenceau | Stop Horse Abuse

  3. Great information although I”m sure they will find a way to discredit it. They will probably say the wording is slanted because they didn’t call it `harvesting’ or `humane euthanasia in a facility’ like I’ve seen them use.

    • Well, they are free to pay for their own survey. They won’t though, because they know the answer. Oh, and they have no money, what with Duquette refunding Scummit registrants from his own pocket!

  4. You did make sure Shannon sees this, right? At first, I thought she had some intelligence but when they started talking about the pros needing to prove that bute is not a carcinogen despite the current “evidence.” (It gets better) Then Shannon says, “I’m surprised that the antis haven’t jumped on this – you know, prove that bute is so dangerous – or something to that effect. We laypersons no matter which side we’re on are going to prove bute is/is not carcinogenic! I honestly could NOT believe what I was reading! I told them they were making themselves look like terminal ignoramuses. How is it POSSIBLE for adults to be so dumb?.

    • Shannon acknowledged it on the “On the fence” page. But I wrote it so I could “trot” it out whenever I needed to! Most of the stuff I’ve written I just now copy/paste it directly into Facebook instead of re-writing an entire answer. I figure that after a few more blogs I can just cruise through FB and news sites without busting any braincells, just cutting and pasting! Are you not on that page anymore? You miss a lot by going to bed and sleeping eh?

      • Apparently I’m not on the page anymore, since I can’t even bring it up. Probably miss a few things, but I also miss a LOT of aggravation too. What I posted was only a pale shadow of what I was THINKING. I dunno. I’m at the point that I just cannot suffer fools gladly anymore. Too many other things to do.

  5. Well, at least take heart in the fact that they are being ignoramuses in private now. At the rate they are banning an moderating over there, there will be about three of them sitting around agreeing with each other and solving all the world’s problems…

    • That’s for sure! They don’t tolerate any disagreement. If you dare to disagree, it’s automatically counted as rude and argumentative. LOL!

  6. Pingback: If You Were Waiting for a Sign – This Is it! « heatherclemenceau

  7. Pingback: Calgary Anti-Slaughter Billboard – If You Were Waiting for a Sign, This Is it! « heatherclemenceau

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s