Philosophies and approaches may vary, but effective market research hinges on two important considerations: reliability and responsibility. Reputations live and die on these pillars of integrity, especially in this era of misinformation sourced from the internet. It seems everyone claims to be an expert online. But when opinion is confused with fact and ranting is given the weight of truth, things get a little muddy. In fact, they can even become dangerous. We get not only “fake news,” but also “fake data” — or at least misleading data.
So the question becomes: What is the best process for the evaluation of secondary data and sources?
Dedicated to the reliability and validity of secondary sources
As a leader in the market research field, Packaged Facts’ reputation has been built from decades of quality, trustworthy analysis. While our report methodology relies heavily on Packaged Facts’ annual proprietary national consumer surveys, we also utilize secondary resources. These secondary external resources include government agencies, annual reports and presentations, trade associations, advocacy groups, third-party credentialing organizations, trade/business/consumer publications, company profiles, retail sales databases, and more.
Criteria for evaluating secondary data sources
The aforementioned list consists of resources that are almost universally trusted and vetted. But at times we are required to go beyond our comfort zone. In such instances, we must reach deeper into the internet and filter through a medley of the misleading and the misinformed. During this precarious undertaking, Packaged Facts’ analysts rely on a “5 Ws and an H” approach for their resource evaluation process. Consider borrowing these tips for your own research and data-related business decisions:
- Who collected the data
- What is the data provider’s purpose or goal
- When was the data collected
- How the data was collected
- What type of data was collected
- Whether the data is consistent with data from other sources
Important questions to ask
Who collected the data: There’s comfort in knowing who originated and collected the data we find. The “who” factor impacts the data’s reliability and whether or not we ultimately opt to utilize or trust it. Data from professional organizations, known brands, or government agencies will always have a trustworthiness not commonly associated with information gathered from less credible resources such as personal blogs or websites.
What is the data provider’s purpose or goal: It’s important to gauge objectivity and intent. Could the data or research be potentially biased? Is there an agenda that’s being pushed? Consider these things, especially when examining data from commercial businesses and even political parties posting information online that might favor them in some way or represent their own interests. The integrity of such research might therefore be compromised.
When was the data collected: Remember the adage about the right time and the right place. If you’re seeking data about the latest trends in a particular industry, it likely wouldn’t make sense to trust research that’s half a decade old. This is true even if the information comes from the most reputable resources. So it’s important to always check the date of your data for the most accurate and relevant information available.
How was the data collected: Be aware of the methodology. To quote Packaged Facts analyst David Morris, “A healthy skepticism of data and research is important. Don’t be afraid to compare and contrast with other research. Look for inconsistencies, and see if there are explanations into how the data collection process explains or damages a source’s credibility.” Further, this question is important when using data directly related to consumer’s opinions and needs. If you know how the information was collected, then you will be able to tell which methods were used. This can also inform you if the data is related to the population you need information about.
What type of data was collected: Stop and consider which types of secondary data will be the most appropriate for your research goals. Not all forms of secondary data are applicable in every instance.
Whether the data is consistent with data from other sources: This goes back to the quote by Packaged Facts analyst David Morris. It’s important to compare and contrast your findings with other research and look for inconsistencies. Sometimes inconsistencies are a matter of differing methodologies, but other times there’s a more distressing, nefarious reason why things aren’t matching up. External data is useless to your organization of it isn’t correct. Take the time to check that your source is reliable so you can trust the data they are providing you.
Asking these six questions (or rather these 5 Ws and an H questions) will safeguard the accuracy of your data and research. More importantly, taking these steps will protect your business reputation with clients, colleagues, the media, and others.
Interested to know more about our research?
Check out our free downloads.