Research is a fundamental and crucial aspect of properly executing product development and management. From understanding demographics to discovering the needs of the marketplace, product development professionals need solid data to back up their choices. However, research can be flawed. In some cases, product managers conduct their own investigation, which can involve deficiencies in research methodologies and the quality and validity of the conclusions derived from the study. Often key, relevant data are either not available or overlooked. If primary research, such as focus groups, is used for a small sample size, it can be incorrectly extrapolated to the larger population.
On the other side, when product managers rely on a third party or in-house group to do the heavy lifting, they can take the research at face value and assume it is correct , which can be a very problematic and risky assumption. Some very large companies such as Iridium (Motorola), Microsoft (Spot watches), Coke (New Coke) and Pfizer (Celebrex) have all conducted market research or product studies that have been inherently flawed. In these cited cases, failure experienced by these companies can be categorized into one or more of the following problem areas:
The reliance on what I call “the purchase intent fallacy,” in which individuals think they can assess true market viability by asking people with no skin in the game (e.g., their own money) how likely is it that they would buy this or that product, often without the benefit of having using it. This common but flawed practice occurred to both Microsoft and Iridium with their hundred million and billion dollar flops.
In other cases, the research failed to include key pieces of information (the omission sin) or was done under unrealistic conditions. For example, in the case of Iridium, it did not share key facts with those being surveyed, such as the fact that the phone would cost $3,000. In the case of New Coke, the results were based on “blind taste tests,” which favored Pepsi but when the sample taste tests were done openly, they overwhelmingly favored Classic Coke over Pepsi.
Cherry picked data is more problematic and occurs when a company or managers want their “baby” to make it through, ignoring any data that contradicts their strongly held convictions (confirmation bias). This occurred to Pfizer with its Celebrex drug; any data that showed no significant efficacy with Celebrex were left out of the studies it submitted to the FDA, which, when uncovered, led to multi-million dollar fines and lawsuits.
In conclusion, if this can happen to these sophisticated companies, it can happen anywhere. “Breaking Failure” is a resource to combat these issues that outlines many other research problems as well as ways to prevent them. Stay tuned for more PDMA blog posts on this topic. This graph for example shows some of these other areas of faulty research.
More about the Author:
Alexander Edsel is the Director of the M.S. in Marketing the Jindal School of Management at the University of Texas at Dallas, where he has been a faculty member for more than 12 years. In addition, he has more than 20 years of product and marketing management experience in both B2B and B2C companies. Edsel holds both MBA and JD degrees. In his spare time, he loves to read nonfiction books, spend time with his family, and explore additional failure-mitigation techniques at his blog at www.breakingfailure.com.
Posted: December 13, 2017, 1:14 am
On the other side, when product managers rely on a third party or in-house group to do the heavy lifting, they can take the research at face value and assume it is correct , which can be a very problematic and risky assumption. fast soundcloud to mp3 conversion and download, read Kissmanga chapters for free