This is one of the most common questions we get from clients.
The answer depends on something called the margin of error. This is a statistic that expresses how confidently you can trust your survey’s sample to reflect the opinions and behaviors of your target population as a whole. Also known as Confidence Interval, it’s a statistical validation of your survey experiment. The higher your margin of error, the less faith you can have about your findings.
Simply put, you want the margin of error to be as small as possible. And a common goal for market research surveys is to get the margin of error below 5%.
But what does this mean as far as your sample size?
Well, a good rule of thumb is 400 responses.
This means that 400 people will answer your survey in its entirety. And that’s 400 people left in your dataset post-data cleaning.
For most surveys with a general population or broad consumer-base audience, 400 responses puts your margin of error at just under 5%. See chart below:
This is why, at PeopleFish, we typically suggest 400 as the starting point for clients to decide on their sample size. Go up from there (or, when budgets are limited, go down from there, but not too far down!). More than 80% of our surveys have sample sizes of at least 400 people.
Here (below) are a few tools we use to help us calculate sample size and margin of error. We’re working on a sample size calculator to embed on our website — one designed specifically for startups and entrepreneurs.
But it’s important to remember, too, that your survey’s sample size is ultimately up to you. More responses is always better, but the natural cap on your sample size is your budget. Is it worth your while to spend an extra $500 to survey 800 instead of 600 people?
That depends on what you’re trying to do. For some clients, 600 is nowhere near enough — in fact, we’ve fielded many studies to samples of more than 4,000 people. But for others, 600 is more than sufficient (especially when our client has a lot of secondary research already supporting one or another answer to their research hypothesis).
Another thing to consider when deciding your sample size is how you plan to cut and segment your data.
Imagine that, from the outset, you’re very interested in seeing the differences between, say, males’ and females’ answers to your survey questions. And further, that you plan to analyze these two groups independently. Or that you’re mostly interested in analyzing the responses of just the males or just the females.
In that case, you’ll want to be sure that each of the two groups has enough responses to draw conclusive findings. Think about the potential size of your smallest segment, and decide on your full sample size based on the absolute minimum you want in that segment.
For example, imagine you’re running a general population survey in order to compare answers between four generations. You’d like to present a comparison, but you’d also like to present findings from within each of the four group. If you’re surveying 200 people, you can expect about 50 per group — not very many. In this case, we probably suggest a sample size of 600 to 800, in order to be sure each group has enough responses for your analysis.
As you can see, there are lots of things to consider when deciding your sample size. There’s no such thing as the “right” sample size — there is just higher and lower margin of error. But again, to make this easy:
Start with 400, and more is always better.
To learn more about launching your own market research survey, leave your email below.