February 2025
Service Design
In Progress! Check back soon!
View Post
I read Steve Krug’s ‘Don’t Make Me Think’ a year prior as part of my assigned reading for Georgia Tech’s UX program; it was a fantastic read and a great intro to UX but it definitely left me wondering more about the realities of integrating UX into a team with the last few chapters. After getting some experience and finding myself in the position of having to advocate for testing, I decided to give ‘Rocket Surgery Made Easy’ a read.
While Krug doesn’t outright mention the term ‘UX Maturity’ as defined by the Nielson Norman Group (‘UX Maturity’ measures an organization’s desire and ability to successfully deliver user-centered design), he implies that different organizations will have difference approaches to UX Research based on where the organization is in their maturation.
One of the most impactful ways to get a lot of feedback very quickly is by conducting usability testing which Krug defines as:
“watching people try to use what you’re creating/designing/building (or something you’ve already created/designed/built) with the intention of (a) making it easier for people to use or (b) proving it is easy to use.”
As also stated in ‘Don’t Make Me Think,’ Krug shares the differences in qualitative/quantitative testing:
1. quantitative testing - interested in proving something (highly methodical)
2. qualitative testing - interested in improving something (can be less structured)
Quantitative testing and analytics tells us WHAT people are doing but qualitative testing tells us WHY people are doing things.
Krug outlines how to convince leaders to allow the implementation of usability testing by being reasonable and being consistent:
“a morning a sprint, that’s all we ask”
He also suggests that team members (anyone) should watch at least one usability test be conducted; the results will speak for themselves. Treat testing as a ‘spectator sport;’ make it easy to attend and make it enticing.
How do you even start?
Practice! Before starting usability testing on a native product, test someone else’s website/product. Once this has been done, there are a few pieces of advice Krug offers to ensure success:
1. start testing as early as possible: it’s easier and less costly to fix usability issues early
2. start with about 3 people: testing with a few makes it easier to do more rounds; there are diminishing returns after more than a few users
3. recruit loosely: testing users who are representative of your target audience isn’t as important or as simple as it may seem
When coming up with scenarios/tasks, be sure to be as specific as possible. Krug also notes very specifically not to let users use the ‘search’ function or deviate from the product being tested!
There are two. You must be both:
1. tour guide - telling users what to do, keeping them moving and making them happy
2. therapist - getting users to verbalize their thoughts
Being cognizant of the task in hand is vital for the facilitator: use standard equipment to keep focus on the product being tested and make sure the user/participant is comfortable so they can provide as much product-specific feedback as possible.
Specific things to keep in mind:
1. Write a welcome/intro script: try not to deviate from it
2. Have a background section to provide some context of the product being tested: a tour
3. THE Tasks/Scenarios being completed
4. Probe a little bit: follow up with ‘why’ questions or ‘tell me more’ (do NOT influence them)
Note: Remote testing provides about 80% of the benefits with 70% of the effort. It provides for: easier recruiting, no traveling required (saved cost), and easier scheduling.
As Krug says in ‘Don’t Make Me Think:’
“Focus ruthlessly on only the most serious problems”
It’s best to rank usability issues and walk away with two separate things:
1. a list of the most serious usability concerns
2. a list of concerns that ought to be fixed before the next round of testing
“When fixing problems, try to do the least you can do”
Doing less to fix problems has huge benefits for all stakeholders.
1. Tweak elements, don’t redesign
a) costs less, requires less work, happens quicker, break other elements less
2. Take something away
a) there is too much there already; make it simple
1. a morning a month, that’s all we ask
2. start earlier than you think makes sense
3. recruit loosely and grade on a curve
4. make it a spectator sport
5. focus ruthlessly on a small number of the most important problems
6. when fixing problems, always do the least you can do
Wow! At time this book feels a little old-fashioned, especially when Krug is talking about testing equipment and providing examples but the principles discussed are anything but out-of-date. As someone who has had to serve as designer/researcher/strategist before, it was nice being given some very specific and practical tips about how to go about testing. It was also validating reading about just how important it is to communicate to stakeholders how important testing is. Just like Krug’s other book ‘Don’t Make Me Think,’ this guide helped to zoom out and see the bigger picture when it comes to usability - something that, as a designer, I need to be reminded of.
One of the most important takeaways from this book was simple: effective UXR takes preparatory work. Krug jokes about having countless lists around him when conducting testing, but in all seriousness, it is vital to gathering relevant insights. It’s vital to have a script to keep things as consistent as possible but it’s also just as important to follow up with specific users/participants when something unexpected happens to try and get as much context/data as possible. This is why I prefer semi-structured usability testing over other testing strategies (benchmarking is also important).
Overall, I felt validated throughout reading this book; it felt like Krug is an advocate for isolating variables but that in the real world, this is no simple tasks. I took extensive notes while reading that I’ve already referenced back to countless times.
10/10 - essential for my role.
