The difference between simply running tests and continuous optimization lies in the process and the organizational readiness. Before you can optimize, you need a map for success. You and your team have to understand where you are going and how you are going to get there. The dial tone of data-driven UX and product management is a framework we’ve developed called Problem Solution Mapping (PSM). The concept is as simple as it sounds, but executing requires a diligent and deliberate approach.
What is Problem Solution Mapping (PSM)?
It is a regularly revisited and optimized framework with three steps:
- Transparently agree upon the business goals you will use to measure success.
- Determine the biggest problems that stand in the way of achieving those goals.
- Develop hypotheses for solving those problems.
This is an always-on process that takes time, energy and commitment. But the good news is it isn’t rocket science. And every business we talk to has the raw materials to do it.
Start With Goals
PSM begins with goals. It is critical that the people responsible for the health of the digital business define and articulate measurable goals. These goals also need to be clearly communicated to the team, ensuring everyone is on the same page and understands the criteria for success.
The truth is not all goals are created equal, and too many goals will cause a lack of focus. So while there might be a lot of things that you want to achieve, those smaller goals should be grouped into parent goals, which lead you to the most critical goals by which to measure success.
These critical goals are going to vary based on your business. There are usually broad financial targets. Some of your goals might be related to customer satisfaction or customer retention. Others might be related to innovation or the development of new technologies and experiences. And you may also want to include goals on the organization, development and improvement of the digital team.
But no matter what you land on, all your goals must be measurable and have set targets for success. It is not enough to simply improve conversions or increase revenue — rather, they should be improved or increased at a certain rate or by a certain measure. Even if the goals don’t appear to be qualitative, they can judged by sentiment or whether something was completed. All goals can, and must, be measured.
Proceed to Problems
Now that your organization’s goals have been determined, you have the necessary foundation to articulate problems. This is not a forum where every single issue and gripe is documented. Rather it is an opportunity for the team to identify problems that are oriented to your defined goals. The group may come across some problems in the process that don’t easily or immediately map to your current goals, and those can be recorded, but the expectation should be set that they probably won’t be prioritized.
To keep this process productive, it is important to properly articulate your problems. We use the following guidelines during this process:
- The problem correlates to a goal.
- It can be identified as an external customer problem or an internal business problem.
- Decide whether it is a root problem or whether it is actually a symptom of a larger issue, in which case you should try to get to the root problem.
- Do not include a solution in your problem — the problem itself is its own critical unit.
You may come out of this initial brainstorm with dozens if not hundreds of problems. So once you have them logged, start clustering them into problem spaces. You will find that many of these problems will start organizing themselves into themes. For example, you might find that mobility, product content and searchability come up across multiple problems. And those problems spaces represent the most succinct articulation of the challenges you need to solve.
Once all of the problems are captured and clustered into spaces, you need to agree on a final list of problems. With finite resources available to build, test and implement, it is unlikely the team will be able to tackle everything. Elevate problems that you believe will have the greatest impact. This is where the real magic of optimization comes in — performance will improve when you are able to remove a friction, eliminate an issue or amplify a benefit in a way that solves your problems.
Now Propose Solutions
This is the step everyone anticipates most. Without this framework, we’ve consistently seen that people try to start with solutions. While that may seem like a more positive or productive approach, the results are often disappointing. When your solutions are actually oriented around big problems that are tied to big goals, you know you will be making progress in the right direction.
This also happens to be the hardest step in our process. Rather than simply throwing out solutions, like “build a responsive site” or “move email signup,” you should state a full solution hypothesis. While most people know how to communicate goals and problems, hypotheses are a little trickier.
Our format for articulating hypotheses has two parts to it:.
- Start with “I believe that…” and the specific change you are proposing.
- Follow that with “If I am right, then…” and the benefit or impact of the change.
For example, “I believe that if we change the mobile homepage hero image for email subscribers to mirror the promotions featured in our weekly email, users will respond to the first offers they see in their mobile web experience. If I am right, I expect users to click on promotions at a higher rate and bounce will be reduced by 10% for that segment.” This hypothesis is clear about what they want to test against the control as well as the outcome they expect and how they would measure success. A designer, developer & analyst could reasonably begin working on the experiment without many open questions.
Once you have these solution hypotheses, they need to be prioritized. To decide what order to address them in, they need to be scored and ranked accordingly. We assign a value for the following criteria to develop this score:
- How much effort is required to to test this hypothesis?
- Based on data that you have, how likely is this hypothesis to be true?
- If it is true, what is the potential benefit to the business?
- If it is true, is the business able to take advantage of that benefit?
That fourth point can save your team a lot of time and agony. While a solution might seem great on paper, it might be technically too difficult to put into product. Or maybe it is creatively out of sync with your brand, so it would never be implemented.
Those four criteria will help you generate a score for your hypotheses. With that score, you can create a sequenced product optimization roadmap. This roadmap will have prioritized hypotheses mapped to important problems mapped to critical goals. And that is the holy grail of optimization.
Tips & Tricks
As we have worked through PSM with our clients, there are a couple things we’ve learned about this process. Here are some tips to keep in mind as you implement your own program.
Not all problems are valid. One risk is that in a free-for-all problem brainstorm, you may capture and prioritize problems that the end customer doesn’t actually have. So when you’re doing problem definition, before you get into solution development, you need to decide if you want to invest in confirming that the problems are legitimate. To do that, you would look at the data you already have available, and possibly run usability tests if that data is unclear. By spending time and money to confirm problems, you can remove some of that internal bias.
Sometimes you just do it. Once hypotheses are prioritized, you may realize that some solutions are so obvious that they don’t need a full cycle of tests. We call that a “just do it.” These should be reserved for when the pressures of time or the strength of the evidence is so overwhelming that the opportunity cost is too high to split test that change.
Every change is an experiment. That being said, we believe that most solution hypotheses should be tested. Even if it’s something that you have significant evidence to support, or it’s something that you need to get to market quickly, we recommend running an experiment with a small control just to get high fidelity data. The alternative, where you quickly build and launch it at 100%, means you may need to do pre-post analysis or use anecdotal evidence to prove success. Anything that solves a big problems or maps to a critical goal, especially if there is high potential upside that you want to document, deserves an actual test.
Keep the cycle going. While there is a significant initial investment in Problem Solution Mapping, sustaining and iterating on this process is much easier. Goals should remain static for a significant period of time, ideally a full year so that the team has a strong foundation on which to execute. But we recommend coming back to your problems and solution hypotheses on a quarterly basis. By then, you will hopefully have a couple wins and be on the path to solving other important problems. You will also have new data, which may lead to new hypotheses. Iteration allows you to take advantage of what you’ve learned along the way, and that data-fueled discovery process is one of the key benefits of optimization.
Looking for help embarking on your own PSM journey? We’re here for you.
Want to know about our latest content? Subscribe to our blog.