Persona-based personalization
As the lead designer embedded within a multi-disciplinary team for the qualitative feedback tools at Hotjar, a large part of my work is balancing our team’s ability to deliver quality product improvements to customers, while allowing the Engineers to work on technical debt that is blocking the toolset from meeting the current market needs. Because of this constraint, much of the work is focused on a close loop between delivery and discovery so we can better learn the levers that are negatively and positively impacting the tools usage. Therefore our team has worked on many small tests in a deeply agile way to better understand our user’s behaviors and incrementally build more engaging and useful tools for them. Please note that some key metrics have been removed due to intellectual property of Hotjar.
Problem
Polls has a very low adoption usage compared to other Hotjar tools. When we asked users why they weren’t using Polls, the number one reason across all personas was, not knowing which questions to ask, and the second reason was that they didn’t believe Polls would help them with their site goals.
Our current setup experience has a lot of information on a conversion step, lacking a clear action for our users to take, and once users start to set up a Poll, they’re dropped into a blank state.
Hypothesis — Based on customer feedback, a large portion of Hotjar users do not know which questions to ask their customers. We believe that building ready-made templates for our target persona will lead to more Polls being created.
Discovery Insights
In the MVP version of this test we aimed to validate the idea with as minimal time to implement as possible (risk mitigation). We scoped the project down to its minimum: redesign the messaging in the “empty state” of the Polls page. Our test was built around targeted messaging for each of our personas and their jobs to be done. With this targeted messaging, we saw a 30% increase in users who previously never set up a Poll, to initiate the creation process.
USER FEEDBACK + SURVEYS
When we asked users why they didn’t currently set up Polls (via a Survey), the top reason respondents cited was that they didn’t know which questions they should ask.
All personas also cited that they didn’t believe Polls could help them with their site improvement goals.
Polls relating to improving the site experience have higher engagement than others.
Early Design explorations
On the left are the blank state persona messages we had in the MVP version. On the right are various ways we approached the template selection process.
Final designs
In the end, the user is exposed to the empty state test (by persona), then they are able to pick a template (or no template) regardless of their persona. If they pick a template, they are taken to the “questions” section of the Poll creation process and have some pre-selected questions related to the template they selected.
Results
Ultimately, by seeing an increase of 33% in Polls initiated and then ultimately created, we can assume that we in fact, were able to help users ask the right questions at the right time - proving our hypothesis to be true.
+40%
Increase of users setting up a Poll and then viewing responses.
+33%
Increase of users who initiate Poll creation and ultimately create a Poll.
Learnings
Around half the users preferred to create a Poll without a template. Which suggests we need more templates or we aren’t selling the value of each template.
Across all personas, “increase conversion rates” was the most used template.
NPS was the second most popular template across all users.
Process
Although no design process is as perfect as we’d like. We can roughly define the process in the following stages. In the sections below, we’ll go into more detail about each step and how we came to the final solution.
Discovery Deep Dive
Below you can see a bit more of the process our team took in identifying what the problem was, and how we might want to find different solutions for it.
Sources
We used a variety of sources to inform this project. From desk research like looking at analytics (which at the time were very scarce) to surveys, user interviews, and customer support reports.
Current User Journey
In both areas of the creation process for Polls there are significant pain points. One in the conversion area, and one in the creation area.
Customer Feedback
Old Experience
In the old experience there was a lot of content before moving someone into creating a Poll. Then when they entered the creation process it was a big blank sheet experience.
Future user journey
We add in an interstitial step to suggest a template and help users ask the right questions. We also adjust the content on the empty Polls page to make it align with Jobs to be done of each persona.
