Tom McGoldrick, VP of Research –

Discover the blind spots inherent in AI-generated information and why direct customer research (surveys, interviews, and focus groups) are essential to predict future needs and stay ahead of the market.

Currently, we are inundated with a wealth of unstructured data flowing in from social media, call centers, tech support, online reviews, and even synthetic data. AI systems can ingest this information and quickly produce abundant and seemingly comprehensive information. There is the temptation to mine this information and believe we have learned all we need to know about our customers and market. It is readily available, fast, and seems to cover every topic imaginable. Unfortunately, all this information also has some significant blind spots.

The blind spots of unstructured data:

• It is entirely historical. The basis of AI is a search through content and conversations that already exist online. If you are testing something entirely new or very specific, there will not be any information for it to draw from. It is essentially a reaction to how things are, not what they could be.
• It often lacks information about who is commenting. Not all customers have an equally large impact on your business. Do you want to learn from your most important customers or just your loudest?
• It is vulnerable to algorithmic amplification. We have all experienced how the echo chamber of social media can prioritize and normalize extreme views.
• It has been known to hallucinate. On occasion, generative AI systems have produced completely false information that can only be identified by digging into references it provides.
• It can be too eager to please. AI systems try and emulate the tone you use when asking questions and provide information consistent with what it thinks you want to receive. It is very easy to “lead the witness” when dealing with generative AI.
• It entirely misses one of your largest segments: those customers who consume your products and services, pay their bills, and never contact you for support or feel the need to comment online. Think about all the businesses you interact with each week. How many of those do you post about or call for support? If you are like most people, not many.

With primary research you can answer the questions that are the most important to your business and target the people who are going to have the greatest impact. It is the best source of specific, forward-looking, and targeted research.

• Test new ideas before they hit the market. Survey customers and prospects to gauge interest in a new product or services. AI systems are backward looking. They pull information from the past to form an answer. If you are testing a brand-new concept, there is limited basis for the AI to draw from. In contrast, sharing a new concept with people you know to be your target market gives you precise feedback on how they will respond and if the new concept is something they would pay for.
• Understand trade-offs with conjoint analysis. Run a conjoint study to build models demonstrating the impact of various price and feature options on consumption. When customers are evaluating a new product, it is never in a vacuum. There are always tradeoffs. For example, in the high-stakes environment of health insurance plan design, when asked, everyone likes a low monthly payment and is likely to complain online about deductibles. Unfortunately, that is not how plans are built. A lower monthly payment often means a higher deductible or fewer supplemental benefits. With a conjoint study, we can learn what types of tradeoffs consumers will make (e.g., a higher monthly payment for a lower deductible). These studies also feature a simulation tool to allow the product development teams to run a variety of different scenarios to predict customer behavior and look at differences in how segments view tradeoffs.
• Co-create with your most profitable customers. Talk to your most profitable customer segments to help build products and services for their unique needs. Talk with those people who know the most about your products and who have a vested interest in improving the products they consume. Don’t just look backward and ask how things are going — take this opportunity to present new ideas, test new concepts, and ask probing questions about their needs and desires.

Customers want to feel they are special and part of the process. Asking for their opinions and then acting on that information is a cornerstone of good customer relationship management.

Will you build your business by reflecting on the past, or will you test new ideas and look for novel solutions to truly understand how to succeed in your market? After all, the pace of change will never again be as slow as it is today. How are you preparing for the future?