Medium Product VP on Skills For Product Managers (Part 2)
Think about it – everything we work on involves some aspect of the scientific method. In other words, product managers are constantly testing hypotheses. From launching new products and features to starting a company, these tests all follow the same protocols that we learned in 8th grade! Today, product management teams apply best practices from “test and learn” approaches in order to validate new product designs.
While product managers should embrace opportunities to test new designs, there are limitations to its effectiveness. For example, Doug Bowman (formerly of Google) sent out a now-famous “Goodbye Google” post when he left the company. In the post, Doug highlighted that Google was conducting A/B testing on 51 shades of the color blue for a new product. Clearly, the test for optimal color on a new product should not require A/B testing on 50+ variants.
Instead of being handcuffed by over-testing, product managers should be bold in developing hypotheses for new designs. In other words, every test associated with the new product should maximize your opportunity to learn from it. Conversely, you shouldn’t waste time on small experiments (like testing 51 shades of blue) that aren’t going to teach you anything. Most importantly, bold hypotheses force your team to think big and get creative with building amazing products.
That said, product managers need to be smart about how these hypotheses are created. For example, during my time at Twitter, we worked on a new photo solution for the feed. If you recall, photos in Twitter were accessible only by clicking a separate link within a post that took you to another view.
At the time, Instagram was taking off and we saw an opportunity to add photos to the feed as an easy win.
During early testing, we saw that photo views and photo sharing went up. However, many of our most important metrics (favorites, retweets, replies, impressions) went down. By making actual photos visible in the feed, we found that users were not as likely to share it because they’d already seen the content. In addition, photos take up more space than a standard text-only tweet. As a result, scrolling time was impacted and users could not see as many tweets when browsing their feed.
On the monetization side, we had to address these issues for advertisers. Simply put, they aren’t going to invest in paid content if it’s not driving engagement. As a result, we reduced the width of photos to reduce the space taken up by a photo post. In addition, we added favorite/retweet/reply buttons underneath photos to increase engagement opportunities.
Ultimately, we learned that our basic hypothesis of “we need photos because users will like it” was not strong enough. While adding an interactive feature is usually a good idea, product managers need to think about the downstream impact to the rest of the product. As I learned at Twitter, the solution must complement the overall product experience.
About the speaker
Michael Sippey is VP Product at Medium, leading a team that helps more than 80 million readers a month discovery smart thinking on things that matter. Prior to joining Medium in 2017, Sippey was CEO and co-founder of Talkshow Industries, a startup focused on making great conversations easy to discover. He’s also held executive and product leadership roles at Twitter, SAY Media and Six Apart. He has a BA in English Literature from Colgate University, and an MBA from the Haas School of Business at UC Berkeley. He lives in Berkeley with his wife, his two daughters and his dog Scout.