User Testing Belongs in the UX Process: Here’s Why

Originally published on 20 Nov 2018 on CMSWire.com.

People who are new to the idea of user experience or the user-centered design process are often surprised when UX specialists want to test designs and prototypes before engineering writes code. They might assume UX handles A/B testing after release. Misunderstandings like this cause companies to exclude UX testing as a waste of time and budget.

An increasing number of case studies online show teams who adhere to the UX process and include rounds of user testing before delivering to engineering work more efficiently and get better products to market more quickly.

“Lean” doesn’t have to mean stripping down processes to exclude important research and testing. Lean can mean you are always improving ways to get feedback quickly, iterate and release something better for the customer.

UX experts who are proficient in prototyping can create a medium fidelity or (with their UI teammates) fully visually-designed prototypes to test on real or archetypal customers. If testing finds flaws, UX can quickly iterate on designs, executions and the prototype, and then test again.

This approach offers a number of benefits over having engineering “just build it” and “just ship it,” only to learn later it’s a disaster for customers.

1. Testing Means You Don’t Waste Engineering’s Time and Resources

If you want testing participants or real customers to see a finished product created by engineers, you have to build and test it for bugs. If UX testing brings needed changes to light, developers would have to rebuild and QA would have to retest. If UX testing showed a larger failure of the concept, this might mean engineering’s time was wasted as this is not code that will end up anywhere. The concept must be rethought, redesigned and freshly tested.

Some managers love to think that a UX prototype will become real code, ready to be deployed. However, this doesn’t take into account the strong possibility that the design will need iterations, requiring engineering to burn more time on the prototype. Given the number of scheduled and prioritized features engineering has on their plate, prototyping UX designs that might never see the light of day is unlikely to be the best use of engineering resources.

2. Hide How the Sausage Is Made: Iterate Behind the Scenes

When companies just build, ship, iterate, and build and ship again, customers end up seeing a variety of versions. They see the work in progress. They’re watching the proverbial sausage being made. This often results in frustration and confusion as customers need to keep relearning a quickly-evolving system.

In a worst case scenario, users might see a new version only to have it rolled back for improvements, and then see yet another, newer version.

It’s better to iterate behind the scenes in the UX process and to be clear with testers (who can and should be real users) that it’s a prototype or demonstration version, and not yet replacing the current release.

3. Answer the Why: Monitor and Measure

If a new concept is released live, UX researchers don’t have a good way to monitor how it’s going. With user testing, they can watch people use it, ask them questions, and get the type of feedback they need to determine if something is ready or needs another iteration.

When UX has designed the software, app, website or system, how do you know if it’s poised for success? You want and love data. Your organization already uses KPIs and metrics everywhere possible. You measure engineering team velocity as well as website conversions and customer spending. There might be hundreds of data points or more that you measure.

Companies that love quantitative data assume those KPIs tell the complete story. However, UX always wants to know the why, the qualitative, and not just the what or how many. How are users spending, converting, engaging, etc? If a measurement shows 21 percent of people are moving more slowly through the new feature, it doesn’t tell anybody why or what’s going wrong. Avoiding proper UX testing makes it harder to diagnose and fix issues or customer pain points.

4. UX Testing Pays for Itself

UX testing is not a huge expense. Some third-party testing tools want under $100 per testing participant, some require a minimum annual commitment of thousands of dollars. Either way, these are not huge expenses given the company’s overall budget for the software development process and the importance of early testing feedback.

For the first round of testing, the rule of thumb is to use five people. It’s a small sample, but it’s a quick way to find the most obvious flaws early. For future rounds, you can use five to 15 people, especially when testing would benefit from more participants across more personas (target audience archetypes).

You also have the time the UX specialist will need to plan the test, set it up, watch all the videos, read the answers and write a report. Rounds of user testing nearly always cost less and move faster than making programmers build something they might have to undo or build again.

5. User Testing Solves Arguments

When UX is in conflict with product, engineering or a stakeholder, who is right? Which idea should be built and released to the customer? With multiple people saying, “I like this one better, let’s go with it,” user testing can offer a solution.

UX can prototype one, two or three concepts. It’s best to boil the competition down to the top two or three designs, especially if you can already find compromises across ideas and team members.

Without customer satisfaction, you have nothing. Building something because the head of engineering “likes it this way” doesn’t mean customers will love it. User testing gives the customers a voice and helps you find the right direction for the features or product. It resolves arguments by providing teams with hard quantitative and qualitative data that makes clear which idea is likely to bring the most customer satisfaction.

User Test Early, Experiment After Release

Some companies use the experimentation model where the final concept is released to X percent of people, metrics are watched, and then they decide if they will remove the feature or scale the experiment to more users. But dropping an update on real — and possibly paying — customers shouldn’t be the first time we get feedback. It’s too risky.

Whether or not you are experimenting with releases, user testing will give you confidence the product you are deploying is a match to users’ real needs, habits and motivations. It was researched with them, designed for them, and tested with them.

It’s not user-centric design unless users are included at every step of the process.