Duration
2023 - Ongoing
Role
Initiative Owner
Success Metric
• % of UI-based features measured
• % of roadmap items influenced by
behavioral analytics
I have been advocating for smarter use of user data. My role has been threefold - 1) cleaning up the data & naming conventions, 2) empowering our product team to take ownership in measuring feature builds, 3) influence more data-driven decisions across the product team.
I’ve noticed that our product team was heavily focused on filling our roadmaps with new features, and not necessarily looking back at old ones to renew, revamp, or sunset. This often left to feature overload, missed opportunities for optimization, and unsure how to celebrate wins properly.
Main Challenges
Clean up the data & infrastructure so they are easy to digest and use.
Some product managers were uncomfortable setting success metrics on their own.
Who does what at what time? Where do we create what report? How do we share findings?
Product leadership valued the revenue impact of a feature build more than behavioral change from users.
Storyboard
I noticed a trend in new product managers who have joined our team as we grow and evolve.
Process
How I tackled the problems
Step 1: Infrastructure clean up
Step 2: Stakeholder alignment
Step 3: Framework Development
Step 4: Team enablement
Step 5: Consultations & iteration planning
Leadership helped me prioritize which projects needed success metrics the most. I consulted the product owners and the UX lead to make sure the appropriate metrics were set and perspective items were tagged correctly. Seeing which parts of the playbook were difficult to follow helped me to update the playbook and plan the next set of training materials.
Outcomes
Measure the success on measuing success
to prioritize features
have been created
tagged & cleaned up
Key learnings
There is no correct answer (and not just one answer) to the question "what should I measure" and "when should I measure?" I wish I could give each product manager a clear cut answer so they can follow the recipe. But I found myself asking back, what's most important to you?
I had to have teammates accept that Pendo is not a stand alone tool that will get answers to their questions. Most often, Pendo needed to be augmented with another set of data - qualitative or quantitative - to conclude more specific findings.
User adoption is much slower in a B2B SaaS model where usage is not only dependent on good design, but a whole slew of other factors like employee onboarding plans. Often times after the contract negotiations, the users of the platform (not the decision makers of the platform) onboard at different times. This forced our product managers to wait quite some time (sometimes over a year) to see enough adoption to have some statistical confidence in the numbers.