What is the art and science of data-driven decision-making? It’s not just a matter of plugging in Optimizely and running a few tests, nor is experimentation a “checkbox” exercise. It’s a mindset, and one that can have a dramatically positive impact on your products. In this webinar, LexisNexis Product Leader James Rubinstein talks about how experimentation can make your products better.

Join us for new conversations with leading product executives every week. Roll through the highlights of this week’s event below, then head on over to our Events page to see which product leaders will be joining us next week.


Webinar Notes:

  1. James is a Senior Director of Data and Analytics at LexisNexis.
  2. James highlights the importance of data-driven decision making as a product manager.
  3. A/B testing is valuable for validating ideas, challenging assumptions, and improving the user experience.
  4. An example from Amazon demonstrates the power of data in successful feature implementations.
  5. A/B testing involves formulating hypotheses, defining success criteria, and tracking metrics.
  6. A/B testing is iterative and crucial for continuous product improvement.
  7. Avoid complexity and overthinking in statistical analysis and experimentation.
  8. Focusing on a specific goal metric is essential, and statistics help understand user behavior.
  9. The time factor in experiments is important, as is accurate analysis based on the affected user population.
  10. Experimentation mitigates risks and enables continuous improvement.
  11. Examples from eBay, Apple, and Pinterest illustrate the benefits of experimentation.
  12. Start small with third-party services and using tools like Google Analytics and Google Optimize.
  13. Product managers should explore experimentation to address product issues and challenges.
  14. Building an A/B testing platform drives cultural change and improves product decision-making.
  15. Embracing uncertainty and learning from failure are crucial.
  16. Metrics play a significant role in measuring success and driving outcomes.
  17. Running A/B tests when making difficult or unethical changes requires alternative approaches.
  18. Estimation protocols and canary or dark launches can be used in such cases.
  19. Measuring non-normally distributed data can be done through median, nonparametric tests, permutations tests, and linearization techniques.
About the speaker
James Rubinstein LexisNexis, Senior Director of Data and Analytics Member

James Rubinstein is an advocate for data-driven decision making as a way to make better products. He has a Master's degree in Human Factors Psychology from Clemson University. He has been a product manager, statistician, and data scientist (sometimes all at once!) at eBay, Apple, and Pinterest. Currently, he is the Senior Director of Data & Analytics at LexisNexis Legal & Professional, where he tries to make decisions more data-driven. He lives in Raleigh, NC, with his family, a couple dogs, and a fleet of hoopties.

About the host
Rishikesh Yardi Instacart, Head of Chapter Seattle

Rishikesh is a Sr. Product Leader at Instacart. He has 10+ years of field knowledge at some of the most prestigious product companies in the world. He enjoys working on product development from the bottom-up and seeing products come to fruition.

Provide your rating for this post
If you liked this post, please use the buttons to the left to share it with a friend or post it on social media. Thank you!

Leave a Reply

Read more

HomeAway fmr Engineering VP on Data-Driven Product Management

HomeAway fmr VP of Engineering, Jon Loyens, joins Products That Count to explain the ins and outs of data-driven product management.

Indicative CEO on Building Product from a Data-Driven Perspective

Indicative CEO Jeremy Levy speaks on trusting your gut and how decision-making rooted in data analytics leads to a strong product lifecycle.

MongoDB Product Lead on a Data-Driven Approach to Product Management

MongoDB Product Lead Rachelle Palmer shares insights on a data-driven approach to product management to drive success.

Sign-in / Register for Free

Don’t be left behind in your career. Join a growing community of over 500K Product professionals committed to building great products. Register for FREE today and get access to :

  • All eBooks
  • All Infographics
  • Product Award resources
  • Search for other members

Coming soon for members only: personalized content, engagement, and networking.