Data and the skills around its management are must-know criteria for product managers. We have multiple points to track and multiple skills to refine. What data do I need to manage and why? When and what data do I need to use in my product development? How can I build data stories for my product? Join RBC PM Juan Martin to dive deep into the what, why, when, and how of data analytics.
Don’t forget to join us for new conversations with leading product executives every week. Roll through the highlights of this week’s event below, then head on over to our Events page to see which product leaders will be joining us next week.
Join us at our weekly Speaker Series events to engage with product leaders in your own community and gain insights on how to accelerate digital transformation.
On why we need to use data analytics to make a great product
Data analytics can help in many aspects of building a great product. Juan focuses on four key areas where product managers can utilize this data within their own teams and processes.
“There are many things that you can use data for in many areas, but I can highlight the four ones that are really important when it comes to product. Obviously, the first one is it allows you to create better practice strategies, it really helps you to tell the story about why you are making this product, what you are making about it. That’s definitely one area that you can use data. … It’s also critical for the prioritization process, that is one of the responsibilities of project management. It really helps you to be more informed, to be more fact-based as a product manager when it comes to what features go in, what features go out. …
The third one, and probably the most obvious one, is around the impact of understanding the performance of your product. That’s something always as product we are always wondering about. Is my product performing? As for my hypothesis, are things working out or not working? Why is it not working? Why are they working? This is one of the first areas when entering product management that you will be using the data.
The fourth one … is one of the things that is probably overlooked in some cases is operations. Having a good understanding of the data around how, especially for those broad teams that are managing business-critical, like finance, financial applications, where they’re business is around operations using the data to make decisions about to pull the feature, to not to pull the feature, is critical because it can save time, it can save money, can save energy, and it can save a good sleep. Sometimes product issues are interesting.”
On picking the right data analytics, and how much to share with your audience
Product managers can be handed a lot of data about their products, but it is vitally important that they understand what are the best numbers to share, and how they should share them with different audiences, from stakeholders to managers to the general consumer. Juan lays out important details about vanity metrics versus digging deeper.
“The challenge as product managers here is about using the right data. … One of the things when dealing with some product managers is the use of vanity metrics. … Sometimes people use metrics that actually don’t mean much, such as when they say the metric, you are like ‘so what?’ Your classical examples are they are up or down in the app world, app downloads, app Store ratings, impressions. Those metrics might be very flashy because sometimes the numbers are big. Then you are like, for example, in downloads, how many people are active, how many people are using your app, and how many people are really engaged? So those things very quickly fall apart in the moment when you start doing a bit deeper questions.
However, it is a good thing that you use data and metrics to tell your stories. Data visualization is something to be encouraged, especially in teams, when they are building the product story. It is a key element of this story building because it can make the difference between getting your audience with you or losing your audience. …
The other thing that is interesting is this thing called … data sprinkle. Sometimes we are faced with the situation where product managers come with a lot of data, and they are proud and they are ‘Oh look at this data,’ but then there is no stitching statistic. That’s something that you need to be very clear about is … every data point that you use should be the answer to a very specific question that should be tied to your strategic objectives. That’s key in order not to do this, such as if you are interacting with a stakeholder and then you throw a bunch of data, but then you don’t have a good articulation for why are you using that data. It’s going to look like you are not really grabbing the concept of what you are trying to achieve in your product.”
On the cause-effect relationships as it relates to data analytics
Juan shares that someone needs to ask the right questions before utilizing the data, and this creates a framework to understand the metrics as it relates to your product. One of the key areas of this framework includes the cause-effect relationship of using data analytics to make decisions about your product.
“One role that is really important for you as a product manager is to understand this cause-effect relationship and understand that you have seasonality in mind. So for example, there are certain periods of time that users behave in a different way. So you don’t make attributions to their behaviors. You forget that or there’s some external factor like a campaign that you are not aware of, and all of a sudden, you’re seeing some impact on your product, and you are not factoring that in your cost relationship, and you might be making a misleading product decision. That’s really critical when it comes to AB testing and in the world of experimenting with your product. …
So, we have the process where there was an account open, and we have this compression rate. Then we look at the possibility to introduce product info, meaning what was the product about before the opening of the account. Then we look at the numbers, and we run the test with all the good stuff, the control groups, and all that math coming with that. If you look at the numbers differently, …set A is the one, the winner, therefore I go for it. However, as a product manager, here’s what you play, and if look at the next level, that is the amount of accounts funded, you see that actually what was happening is that, yes, we were getting more drop-offs, but the people that they were going through, they were understanding way more what was the product about and we were getting actually better rates in terms of the funding of the accounts that were actually what we were interested in. So this is where your role working with your data analyst and your teams to understand your product and to really understand the end-to-end.
It’s often that you see especially in the digital marketing space, it’s very easy to get very narrow, start optimizing, optimizing, and then you ended up losing sight of what is the real objective of your product. And then you understand that you are optimizing, but you are losing, you are missing the point completely. So I think that was an interesting point. I always push my teams to think about it when they are doing these experiments, what is the real objective that you are trying to solve? ….
Data is not product positioning, it’s just the input. Sometimes you might feel that even if you have a clear winner in your AB testing, it doesn’t mean that that’s the decision that you need to go. So there is the significance from a statistical point of view, but then there is the business or the product significance.”
About the speaker
About the host
Arun heads Product Management for Royal Bank of Canada’s Solution Acceleration and Innovation group where he leads a team of PMs that manage a portfolio of B2C and B2B products. He also heads RBC Launch, RBC’s innovation hub, where the focus is on rapidly experimenting and developing new products. In prior roles, Arun developed products for RBC's commercial lending business, building large-scale credit structuring applications. Arun has deep domain expertise in financial services gained working in corporate strategy and various business lines. Prior to banking, he has worked as a Management Consultant and as a Physician at different points in his career.