The verdict’s in: When designing for users, involve users
Posted in User Experience by Bridgeline Digital on December 2nd, 2008
Stories go a long way when trying to communicate the value of User Centered Design (UCD). I have a great story to share. It starts with a grand jury inquiry into the usability of a dispatch and mobile response system used by the San Jose police department. The grand jury ruled that users (here, police officers) MUST be involved in the design of any future system to be used by officers (see finding and recommendation #4 from this Civil Grand Jury Report).
The grand jury ruled this because police officers weren’t involved and didn’t participate in the design of a system whose primary users were police officers. Contextual inquiry would have been invaluable: Simply have the system’s designers ride along with the officers to observe them in action. Usability would have been just as invaluable: test not merely whether the system works, but whether a given user can use the system to accomplish a specific task in a given situation or scenario.
Why weren’t users consulted up front? As with any project, there are lots of reasons why best practices weren’t followed. One reason is one we hear frequently: Many people assume that users or customers don’t need to be interviewed; their managers will do just fine, or customer support will substitute nicely for customers. In this case, instead of consulting users, management was consulted. The assumption is that managers know what workers need, so managers can field any questions about what users need in order to complete their work. A similar line of thinking is to talk to customer support instead of talking to customers, since customer support deals with customers on regular basis. But there are multiple problems with these lines of thinking. For example, both managers and customer support will introduce a bias. Moreover, customer support tends to be privy to only one aspect of the customer’s experience: namely, the “I’ve-got-a-problem-and-I’m-really-frustrated-and-I’ve-been-on-hold-for-10-minutes-listening-to-Muzak!!!” user.
The result of not following a user centered design process in this case?
Worst. ROI. Ever. Oodles of money, time, and resources spent on a system that in the end wasn’t usable—and thus, wasn’t used. And that’s just one consequence. A more dire consequence of the unusable system was endangering lives—not just the officers’ but those who looked to the officers for safety.
To learn more about the problems the officers faced with the usability of the system and how a UCD approach saved the day, check out the case study, “Almost Dead on Arrival” and also this NY Times article, “Wanted by the Police: A Good Interface.” I personally find this a great story that really drives home the importance of user centered design.
Written by Ashley Annis
Textbook example for user-centric design principles. Not sure if the photo represents part of the design flaw, but shouldn’t the physical interface be on the dashboard so the officer does not have to turn around to the rear seat to use the system? Seems like they would have picked that up pretty early in a usability test.
Yes. Keeping the users in mind is key, though sometimes heartbreaking for the designer pursuing an artistic vision.
A consultation with management is often all that I am initially allowed. Often, when trying to get user feedback, I have to first have management use the interface, collect their feedback, and then beg and plead to get some time from actual users. Management will usually agree to commit user time after they themselves have been through a usability exercise, and can see what efficiencies may be gained form donating (read: sacrificing) some employee time.
It’s so simple in theory. But so different in practice. While management can provide some helpful initial guidance–ultimate goals, strategic/operational vision etc—they normally have little understanding of the actual day-to-day details (they’re management, they don’t need to know these things). The “doers” in my experience can’t wait to be involved, so long as it is a structured, good use of their time, with the goal of improving their experience or making them more efficient. A question, though: do you ever get conflicting user feedback, and if so, how do you handle such a situation? Great article, thanks!
We do sometimes get conflicting user feedback, especially when (A) interviewing a large number of users and (B) triangulating our questions – that is, asking a question one way at the start of the session and a different way later in the same session.
When findings conflict, we typically pursue one or more of several options: Often we “pow-wow” as a team to bring our UX expertise to the issue at hand, then test our solution for usability with real customers. We also collaborate with our own customers to ensure their goals are being met. Another tactic we sometimes take is to review any existing research on the topic to identify trends and previous solutions. And if it’s a really thorny issue that can’t be solved by any of these methods, will return to the users and conduct more interviews.
But the best practice that we’ve found for resolving conflict in user input is to craft personas. For example, we may interview 50 prospective and current customers, but craft only six personas. We scour the data–the notes for every person interviewed–to identify similarities and differences among the 50 customers. Inevitably, patterns emerge. Even though some data points may conflict, by following a very methodical process for identifying emerging patterns, we are able to resolve and account for those conflicts, ultimately presenting a unified design solution while representing the different contexts users will bring to their experiences.