Qualitative Data: Building A Measurement Machine
The best way to settle debates about the roadmap and the prioritization of product investments is with data. As I’ve outlined in my previous articles, your data is a product too. Specifically, it serves the needs of your company making the right investments in your customer-facing product. But, quantitative data is just one part of the picture – you have to look at qualitative data too. If products only looked at one data set, then we might as well pick a different career.
Qualitative data is insights from your users, organized in an actionable form. These insights can come in many forms – including one-on-one interviews and focus groups. For example, you will be watching a cold user play with your product for the first time. Understanding what your users want and how they feel can be difficult to collect, parse, and find truth in. But, these insights hold the secret to building a great (or greater) product.
I like to build internal infrastructure to not only collect and measure these insights – but to deeply immerse myself and my team in it. Every product has different touchpoints, but in general, these insights come from either inbound or outbound communications. Inbound communication is when a user is actively providing you feedback, wherever they are in the customer journey. Conversely, outbound communication is when you and your product team proactively reach out to users or would-be users to gather targeted insights about your product and brand.
Inbound communication hooks can be easy to set up, but exactly where you place them is critical.
Some examples of inbound communication include:
- App reviews and ratings.
- Feedback and support emails.
- Posts on Twitter feed or Facebook page.
- Responses in community forums.
- Surveys in email drip campaigns or in-product.
Collecting data at the appropriate spot for your product is important. However, collecting it in the wrong place will give you useless, biased, or vanity data. With this, the ability to efficiently gather all of these insights into one place is critical.
I’ve found Airtable to be indispensable for this work. For those not indoctrinated yet, Airtable allows you to organize just about anything and visualize your data in a variety of different ways. It has a powerful formula feature that helps you process heavily tagged data to slice it any way you like to find the answers you need.
The other great thing about Airtable is how open it is with other tools.
The combination of a tool like Zapier with Airtable means the farthest reaches of your consumer touchpoints can be listening for trigger keywords, parse the inbound communication, drop it into Airtable, and tag it any number of ways.
Zapier can pick up qualitative feedback from email, social feeds, and community forums. Moreover, the Zapier <> Slack integration lets you drop this feedback directly into Slack for easy viewing. This doesn’t take the hard work out of organizing your customer insights, but it sure makes it a lot easier. Plus, you can place those insights front and center for your team.
NPS surveys are a great tool for understanding the progress you’re making in your product on a human level. Furthermore, the open-ended responses can be tagged and grouped to help you understand trends. Automating the request to your users to respond to an NPS survey on a regular basis will give you regular insights that can be organized in Airtable alongside feedback from other touchpoints.
When organizing these insights, it’s important to manage your tags carefully.
Over time, your tags will evolve as you begin to get different versions of the same feedback from users. But, it’s also important to tag the source of the feedback. For example, a user who would never be a paying customer has a different value than a user who is already a paying customer. The more you can segment the source of the insights, the better your decisions can be when taking action later on.
Depending on the stage of the product, outbound communication with users takes different forms. At Homer, we had a specific need to regularly test all-new content with kids, since kids have less mature motor skills and their cognitive development is very different than adult users.
If I’m doing a one-on-one interview with a customer or hosting a customer meetup, I document and process the data with the same methodology. For example, if I’m running user testing with an excellent tool like UserBob – I’m clearly tagging both the source and the output of the insights in Airtable. With this, all qualitative data can be organized in one place. At a moment’s notice, the master database of customer insights can be searched and filtered. For example, if you tag and organize your data properly, you can quickly show data from paying customers about a new feature introduced two months ago.
In summary, there’s a lot to gain from collecting data at the right time in the right places. Combined with automation and tagging, you can identify low effort / high return initiatives for optimal engineering resource investment.