Measure Intensity, Frequency, and Recency
As a product team, how do you know which problem to solve? This question might be hard to answer, especially if you don’t understand your customers behaviors, needs and goals. Thankfully, a series of pertinent questions during a user interview can help you generate a map of the most frequent and intense problems in your user’s life.
To classify problems, you simply want to measure these 3 dimensions:
- Intensity: How and how much they suffer?
- Frequency: How often and why it happens?
- Recency: When was the last time it happened?
This framework provides discipline in analyzing problem patterns. It helps prioritize what to focus your product on and ultimately build the right thing. Let’s dive into each dimension to better understand how.
Intensity
To understand a problem, you want to know how it makes a user suffer and to what extent. After hearing a user qualify something as a problem, you can ask several clarifying questions such as:
“Why was this a problem for you?”
“In what way does this negatively impact your company?”
Users feel pain in several ways such as time lost, emotional frustration or physical injury. Finding the type of pain is the first thing you should focus on as you start uncovering negative topics in your conversations with users. If you can qualify the pain, you will then be able to measure how painful it is to your user.
“How much money did you lose?”
“How much time did you spend recovering from that injury?”
Finding a quantitative question is sometimes difficult. It will be easier if you already have identified in what way the user is having a negative experience. The first thing you hear will usually be a negative feeling like sadness, confusion, anger. This is your cue to ask a series of questions to determine if the pain is only emotional or if it has other implications.
“It sounds like this made you “really mad” to use your own words. Besides this emotion, in what other ways (if any) was this detrimental to you?”
Most of the time, your users will have a hard time explaining the pain they experience in general terms. This is especially true when the problem is unpredictable in terms of intensity and frequency. You can ask for extreme examples with questions such as:
“What is the worst instance of this problem you have personally experienced?”.
You can then ask more questions to clarify whether this is a frequent experience. Users might tell you it is a recurring occurrence or that the common case is far easier to handle. You can use a question to get a better idea of what the routine really looks like:
“How often is it that bad?”
Asking a user to rank how important a problem is can be a meaningful way to understand if users would value a solution. The higher a problem is ranked in a hierarchy of needs, the more likely it is that your user will give you a chance. In some cases, although the problem may be significant, there is a worse problem that makes the problem you want to solve a lower priority to your user.
“In the different forms of stress that pertain to your role, where do you rank the problem that we just talked about?”
Here you have already identified that stress is the classification of pain and you are trying to compare a particular situation with other stressful situations in order to determine whether this is really top of mind for the interviewee. If your users constantly rank a specific problem high in a specific context, it might be a sign of an interesting opportunity.
“How did you feel about this problem?”
Sometimes a pain is hard to quantify. This is particularly common when the cost is mostly emotional. You can use vocabulary as a scale to solve for that problem. If the majority of your users use similar words to describe how the problem makes them feel, it might be a good indication of how good your product could ultimately make them feel. If a problem is “infuriating” the appeal of a product that solves for it might be stronger than one that deals with a “merely annoying” one.
Frequency
A big problem that happens once a decade might be less of an opportunity than a lesser one that happens several times a day. Establishing how frequency contributes to a particular pain is another great way to start quantifying a problem. Understanding how frequency and intensity vary will help you get a more intimate understanding of the user.
If you have managed to understand the type of pain, you can proceed to ask about frequency. For example, if you heard a problem is related to losing time, you can ask a clarifying question to understand recurrence:
“How much time do you lose with this per week?”
In the question above, you already know the user perceives the pain as time lost and you are making an assumption that it happens weekly. Sometimes you will get the time scale wrong in the question and the user will explain that it is more of a monthly or yearly phenomenon. Furthermore, it will be interesting to observe if the different users you talk to use a similar unit.
Asking for several example of a problem occurrence will help you support previous answers with evidence the perceived frequency is somewhat consistent with reality. Users might be tempted to draw general rules in a misguided attempt to help you. However, humans are generally bad at estimating while they are pretty good at recalling a specific story they experienced. You can use the following to ensure answers end up consistent by the end of the interview.
“You said it happens every Thursday but you also said last time was Tuesday 2 weeks ago. Why is that?”
Understanding what affects frequency is a nice way to understand better what the root causes are and how user behaviors evolve over time and exposure.
“It seems you had to do less effort last time versus the previous three times. Why is that?”
You might realize through this type of question that a problem disappears over time and only really needs solving in the first month of exposure. This could be key in refining the persona you want to focus on first.
Most problems are multi-dimensional so don’t restrict yourself to a single measurement. You can segment users in interesting ways by aggregating several intensity and frequency factors. Some users might be very aware of an issue and externalize this awareness in their feedbacks while others don’t even recognize it as a problem while they are still experiencing it on some levels.
Recency
“When was the last time this happened?”
“What happened exactly that day?”
These questions will offer an interesting proxy to validate previous and subsequent findings. Is the problem as intense as previously described by the interviewee? Is the frequency the same as advertised? Asking about the last time is great to observe a problem with the highest resolution possible because in most cases the user will remember that example best.
It is a great way to weed out initial blocking answers like “it depends.” You do care that it depends but you need to understand as quickly as possible what are the parameters that induce variability in the problem. You also want to make sure the user is giving you a first hand account of the anecdote and not inferring from situation happening to others. This level of detail is what makes this line of questioning so effective early on in an interview. Perhaps you want to start with those as an opener.
“We heard many Product Managers were having trouble aligning with their stakeholders. When was the last time you encountered that situation?”
Conclusion
Asking about problem intensity, frequency and recency with discipline across multiple users will reveal a clear picture of what kind of people experience what facet of a particular problem. Intensity and frequency are also great to prioritize the most painful problems. Recency will help you gather as much insight as possible on the latest version of a problem as well as confirm general trends in your discovery.
Change is the only constant, so individuals, institutions, and businesses must be Built to Adapt. At Pivotal, we believe change should be expected, embraced, and incorporated continuously through development and innovation, because good software is never finished.
How and Why to Classify Problems in User Interviews was originally published in Built to Adapt on Medium, where people are continuing the conversation by highlighting and responding to this story.
