ASPECTS OF UX THE KEY TO SUCCESSFUL DIGITAL PROJECTS You’re beginning a user research project You want it to go well You want it to be interesting, to learn something new and to generate ideas Most importantly, you want your project to make a difference This book will show you how to achieve all of that YOUR AUTHOR James has worked in research for 20 years, for organizations including Google, British Airways, the BBC, eBay and the Alzheimer’s Society He is currently Head of Research at cxpartners EMMA HOWELL • Discover how to choose the right research methodology for your project Emma Howell is a User Experience Consultant at cxpartners She has been a research specialist for 10 years, beginning her career in academia before moving into UX Emma loves designing products and services that are intuitive and enjoyable to use • Get smart interviewing tips to help you obtain the data you need • Learn how to analyze your data and present results SITEPOINT BOOKS WEB DESIGN PRINT ISBN: 978-0-9953826-3-3 Advocate best practice techniques Lead you through practical examples Provide working code for your website Make learning easy and fun USD $34.95 LANG & HOWELL Visit us on the Web at sitepoint.com or for sales and support email books@sitepoint.com CAD $46.95 RESEARCHING UX: USER RESEACH Design research can sometimes seem a daunting, confusing world unto itself With its own codes and jargon, it can feel like the domain of specialists, forbidden to outsiders who haven’t been trained in the rules If you’ve ever felt out of your depth on a research project, been unsure what to next, or wondered whether you’re “doing it wrong”, then this book is for you UX JAMES LANG This book is about the craft, the technique and the processes involved in running a design research project Research is at the heart of user-centered design Because it’s so central, we believe that research should be an activity that the whole team gets to participate in and feel ownership of • Recruit people to take part in your research RESEARCHING UX USER RESEARCH BY JAMES LANG & EMMA HOWELL GREAT DESIGN BEGINS WITH DEEP EMPATHY Safety Area: All Text, Logos & Barcode should remain inside the Pink Dotted Lines Bleed Area: All Backgrounds should extend to, but not past, the Blue Dotted Lines i Researching UX: User Research by James Lang and Emma Howell Copyright © 2017 SitePoint Pty Ltd Product Manager: Simon Mackie Series Editor: Joe Leech English Editor: Katie Monk Technical Editor: Kate Towsey Cover Designer: Alex Walker Illustrator: Natalia Balska Notice of Rights All rights reserved No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embodied in critical articles or reviews Notice of Liability The author and publisher have made every effort to ensure the accuracy of the information herein However, the information contained in this book is sold without warranty, either express or implied Neither the authors and SitePoint Pty Ltd., nor its dealers or distributors will be held liable for any damages to be caused either directly or indirectly by the instructions contained in this book, or by the software or hardware products described herein Trademark Notice Rather than indicating every occurrence of a trademarked name as such, this book uses the names only in an editorial fashion and to the benefit of the trademark owner with no intention of infringement of the trademark Published by SitePoint Pty Ltd 48 Cambridge Street Collingwood VIC Australia 3066 Web: www.sitepoint.com Email: books@sitepoint.com ISBN 978-0-9953826-3-3 (print) ISBN 978-0-9953827-9-4 (ebook) Printed and bound in the United States of America ii Researching UX: User Research About James Lang James has worked in research for 20 years, for organizations including Google, British Airways, the BBC, eBay and the Alzheimer’s Society He is currently Head of Research at cxpartners About Emma Howell Emma Howell is a User Experience Consultant at cxpartners She has been a research specialist for 10 years, beginning her career in academia before moving into UX Emma loves designing products and services that are intuitive and enjoyable to use About SitePoint SitePoint specializes in publishing fun, practical, and easy-to-understand content for web professionals Visit http://www.sitepoint.com/ to access our blogs, books, newsletters, articles, and community forums You’ll find a stack of information on JavaScript, PHP, Ruby, mobile development, design, and more iii James: To my mum and dad, Sally and Joseph, for giving me their curiosity, patience and kindness Emma: For raising an analytical and inquisitive creature, my fabulous parents Pauline and Leigh My gyaldem: twerking and tequila makes deadlines feel less painful Laura-Lea: you’ve been amazingly encouraging and patient Thank you iv Researching UX: User Research Table of Contents Preface xiii If You Feel Unsure, Then Read On xiii What We’ll Cover in this Book xiv The Gist of the Book xv Acknowledgments xviii Conventions Used xviii Supplementary Materials xix Chapter Begin the Project .1 Get Started Define Your Research Define Hypotheses 3 Timeline & Resources 4 Engage Stakeholders Identify Risks Tools You Can Use Use Desk Research to Generate Hypotheses Hold a Kickoff Workshop Table of Contents v Talk to Your Stakeholders Create a Research Canvas .10 Project Plan .12 Make a Plan to Engage People 13 Decide How You Will Communicate With Your Team 15 Work Through Risks to the Project .16 Summary .18 Chapter Choose an Approach 20 The Core Concepts 21 What is Data? 21 Qualitative vs Quantitative 22 Discovery vs Validation 23 Insight, Evidence and Ideas 23 Validity .24 Scaling Your Investment .26 Multi-Method Approaches 26 In-The-Moment Research 27 Taking Care .28 Research as a Team Sport .29 Research Methods .29 vi Researching UX: User Research How to Choose Research Methods 34 How Many People? 35 Summary .36 Chapter Finding Participants 37 Who To Recruit 38 Defining Your Sample .39 Creating a Recruitment Brief 41 Getting The Right People 44 Creating a Screener 45 Methods of Accessing People 49 Recruitment Agencies .49 Guerrilla Recruitment 51 Finding Participants for Guerrilla Testing 51 DIY Recruitment 52 Gatekeepers 53 Persuading People to Take Part 54 Why Offer an Incentive? 54 Offering an Appropriate Incentive .55 Arranging Sessions 56 Running Order 56 Table of Contents vii Avoiding No-shows 57 Data Protection 58 Informed Consent 58 Summary .60 Chapter Writing a Discussion Guide 61 What’s the Discussion Guide For? 62 What’s a Discussion Guide Not For? .62 Elements of a Discussion Guide 63 Different Kinds of Session 71 Questions, Tasks, and Activities 71 Types of Content 73 How to Write a Discussion Guide .77 Use Your Imagination .78 Summary .79 Chapter Set Up Your Sessions 80 Getting Prepared for Research in the Lab 81 Monday: Three Days Before Your Session 82 Tuesday: Two Days before your session .84 Wednesday: The Day Before Your Session 87 viii Researching UX: User Research Thursday: The Day of Your Session .90 Getting Prepared for Research in the Field 91 Tuesday: Two Days Before Your Session 84 Wednesday: The Day Before Your Session 87 Thursday: The Day of Your Session .90 Getting Prepared For Remote Research 94 Monday: Three Days Before Your Session 82 Tuesday: Two Days Before Your Session 84 Wednesday: The Day Before Your Session 87 Thursday: The Day of Your Session .97 Summary .97 Chapter Running Your Sessions 98 Roles 99 The Interviewer 99 The Host 99 Note-taker 102 Taking Notes 103 Post-It Notes 103 Taking Notes in a Spreadsheet or Trello 105 Taking Typed or Written Notes 106 Table of Contents ix Taking Notes in Guerrilla/Field Research 108 Taking Notes after the Interview 109 Tools For Recording 109 Top Tips For Recording Your Sessions 111 The Practicalities of Using Different Devices .113 Remote Testing When Using Different Devices 114 Guerrilla Testing With Different Devices 114 Summary 114 Chapter Interviewing 115 Show Participants How to Help You 116 Observing vs Asking 118 Paying Deep Attention .118 Starting the Session 119 Make Sure You Get Consent 120 Starting Guerrilla Testing Sessions 121 How to Ask a Question 121 How To Get Out Of Trouble 127 Summary 129 Chapter Analyzing Your Data 130 169 Researching UX: User Research have your story, so just insert it into the structure above To add visual interest, it’s a good idea to include images in this section too Photographs from your project are a good choice, but you can also use screengrabs of the product, diagrams or stock images For the detailed findings, we like to use annotated screengrabs like the ones shown below Finally, you should ensure that you’ve provided recommendations These might be entirely your own work, but it’s better if you’ve come up with them together with the rest of the team Either way, you have the choice of including them on each page, or listing them in their own slides at the end 9-3 An annotated screengrab You may want to produce different versions of the report for different purposes For example: One for presentations that includes at least one very short video clip This version might just include the summary and main story sections Another as a PDF, with links to video in a shared folder This version would be the entire report, and include all the detail slides A blank report template is available at https://www.cxpartners.co.uk/uxresources/, along with many other free deliverable formats Turning Research Into Action 170 List Reporting In contrast to report decks, list reporting dispenses with the frills and gets straight to work listing the problems and recommendations The upside is that the output is action-focused The downside is that it’s not so persuasive or engaging, so don’t use this approach unless your team has already bought into the project and its findings To produce a list report, follow these steps: Choose your format We prefer a spreadsheet, but you can also use project management tools like Trello, or even a wall with Post-Its Agree a system for rating your issues A popular way to this is to create column headings labelled Issue, Severity, Prevalence, Impact and Recommendation In each row, add an issue raised by the research (for example, “Users can’t find the continue button”) Then, rate the issue for severity (ie, how much of a barrier was it for users in completing their task?) and prevalence (ie, what proportion of users did it affect?) We this by giving a score from to in both columns A rating of for severity means it caused minimal obstruction, while a means it completely broke the experience for users A rating of for prevalence means it only affected a couple of users, while a means it affected everyone Calculate the impact by multiplying severity by prevalence You can then sort the rows in order of impact to show the biggest issues at the top Finally, you can add recommendations for each of the issues Again, it works best to this as an activity with the whole team, rather than on your own Alongside this form of documentation, it’s helpful to maintain a project area with photos, example screengrabs and sketches (see Chapter 8), and to use regular show and tells as your method of sharing back with the team Journey Map Journey maps show the steps taken by a user to complete a specific task They represent a sequence of steps or touchpoints, which might involve multiple devices or channels, and might take place over a few minutes or several days 171 Researching UX: User Research Journey maps are an excellent way to record users goals and expectations, and the problems that they encounter in completing them On the downside, they can become overly-complex if the exact sequence of steps needs to be recorded 9-4 An example journey map To create a journey map, follow these steps: Define the user’s goal, and state this at the top of the map You may also want to associate the journey with a particular persona Working from left to right, map out the sequence of steps that the user takes, starting with the trigger that causes them to begin the task, and finishing with the goal For each step, make a note of the channel and device that’s being used You may also want to add details like an emotion line showing the highs and lows of the journey, quotes or opportunities Turning Research Into Action 172 Experience Map An experience map is like a journey map on a grander scale Instead of focusing on a specific journey, it takes in the entire customer experience for a particular product or service Think of it as a planning tool rather than a design tool, enabling your team to identify where changes are needed rather than how to implement them Because they’re high-level, holistic documents, experience maps have a long shelf life They’re often printed out on a wall-size piece of paper, and put in a prominent place They’re a great reference point for potential projects, because they can be related back to the most pressing customer needs within the overall experience, and used to track progress The downside of experience maps is that they’re large, unwieldy and difficult to scan They also don’t give much direction on how to solve a problem, so you’ll probably need to conduct further research to understand it better before you’re ready to implement a solution Compared with some of the other methods described in this chapter, experience maps are a pro-level skill, and you should expect to put in a lot of time and effort to create them 173 Researching UX: User Research 9-5 Example experience map To create an experience map, follow these steps: Stick a piece of brown paper along the largest wall you can find Use a Post-It note to record each need, activity, pain point and delights you’ve heard about from users Generally it’s a good idea to phrase your Post-It notes in the first person – for example: Needs are statements about what the user requires at this stage in the process: “I need to be certain that the shoes will arrive in time.” Activities are actions that users may take within this phase: “Check the confirmation email.” Pain points are problems, barriers or frustrations that arise, such as: “I can’t find the delivery time guarantee anywhere.” Delights are the positives that users experience, such as: “Customer service gave me a $20 voucher to apologize for the delay.” Once you’ve generated your Post-It notes, group them into columns to represent broad phases of the experience Don’t worry about the precise Turning Research Into Action 174 sequence of actions: that’s a job for a journey map Use one row for the names of phases, the next for tasks, then additional rows for actions, needs and pain points If you like, you can keep adding rows for further information In the past, we’ve added layers to record relevant product attributes, competitor offerings, KPIs and solutions – it’s actually a very flexible format You should involve your team in all of this activity, if possible, but if you can’t, then you should now hold a show and tell of the experience map Walk your colleagues through it, and ask them to call out any surprises or missing information Once you’ve sense-checked the rough experience map, you may want to hand it over to a visual designer for a makeover before you print it and put it on the wall in a prominent position Personas A persona is an abstracted set of user needs, normally presented in the form of an imaginary person 175 Researching UX: User Research 9-6 Example persona Personas can be a great way to build empathy, especially if your team live very different lives from your users They’re also a powerful tool for making decisions, because in the process of creating them you deal with a lot of the difficult discussions about prioritizing one set of user needs above another That’s helpful later on, because you may not have time for further research, and you don’t want to keep derailing your product development process with repetitive arguments about priorities Personas have several downsides, too: They polarize Some of your colleagues may be skeptical about using them based on bad experiences of personas they’ve had in the past If they use invented biographical detail, they run the risk of triggering people’s prejudices (positive or negative), which can get in the way It can be hard to get the level of fake biography right Personas that talk about how many cats someone owns aren’t helpful for design decisions, but a pure list of user needs can feel a little dry It’s easy to confuse them with marketing segments, which look similar, but Turning Research Into Action 176 a very different job The trick to producing effective personas is in their creation If the people who are going to be using them aren’t involved in generating them, they’ll flop But if they are involved, personas can be really effective Here’s how to create them: Start by reviewing any existing reports, and conducting qualitative research to profile users In the analysis process, focus on producing conceptual diagrams, particularly spectrums These allow you to categorize participants by their similarities or differences in terms of user need, which you can then use as the basis for personas Give each of these a name (a real name, like Adil, or a descriptive name, like ‘cyclist’) Using this approach, you can create a longlist of personas (anywhere from five to as many as 20) Now you need to combine and prioritize your longlist of personas to arrive at a smaller final set Two or three primary personas is a great target, but you can include some secondary personas too, if you like The way to this is by debate with your project team, not on your own, because the discussion about which to prioritize is right at the heart of the process Here’s an example: let’s say we wanted to design an underground train station In the process of reducing the longlist to a shortlist, we identify shoppers and parents of young children as two separate groups But in design terms, their needs are identical They both struggle to get up escalators, whether it’s with shopping bags or buggies So we’ll condense them into a single persona, we don’t need both We also agree that we’re going to deliberately prioritize the needs of commuters over tourists So we’ll have a commuter persona, but not a tourist persona Later on, when we have to make tradeoffs, that’s our decision Finally, create the personas you’ve chosen as A3 posters, and add as much biographical color (photos, names, background) as you feel is appropriate More biography helps with creating empathy, whereas if your objective is primarily about guiding decision-making then it’s best to keep it light You may also want to consider other formats to get your personas across, such as mood boards, video reels, stories, role-playing or even mannequins! Like a house plant, you should keep your personas alive with regular attention, updating them based on any new research you As you find out more about your users, stick Post-Its expressing new learnings about that 177 Researching UX: User Research persona onto your poster Personas discriminate In fact, they’re a tool for making difficult prioritization decisions They deliberately don’t cover everybody The point of the exercise is the prioritization and debate By getting it out of the way early, you can avoid continually revisiting arguments about who takes priority That means your team can make better decisions, move faster and have fewer arguments along the way You should make sure you update personas when you additional research, too Showreel Showreels are compilations of video clips They’re great for providing your audience with direct exposure to your users, and it highlights their experiences like no other format can The downside is that they’re time-consuming to produce Also, on their own they can give a skewed view of the findings, so they’re best used alongside another form of documentation like a report, experience map or personas Although showreels are the most common format for sharing the evidence directly, there are others For instance, you might choose to create a scrapbook or website of photographs How to make a showreel: Ensure your note-taker includes time stamps for key examples and quotes: this will make it easier for you to find the right clips later on Ask participants to give their final feedback direct to the camera This is often the best footage to use for a showreel, as it’s more likely to be concise, articulate and focused on the main points Without context, it’s sometimes hard for your audience to spot the point the clip is trying to demonstrate, so it’s a good idea to add an interstitial or title that summarizes it for them For example, “Joey struggles to find the correct shoe size on the product page” Add subtitles if there’s time This helps to make up for poor audio quality If it’s really important to get good quality video, bring a tripod and microphone, and work with a colleague or professional videographer to set up and monitor the recording Turning Research Into Action 178 Although there are no definite rules for duration, it’s a good idea to keep it under five minutes in total, and keep individual clips to less than a minute Several short clips that reinforce the same point are more powerful than a single longer clip It’s worth mentioning that this is a very short summary of these deliverables: there’s a lot more information out there in books and blogs on each one, so keep exploring if you want to know more Debrief Session In most research projects, you’ll want to mark the end of the cycle with a face-toface debrief session There are a number of benefits to doing this: You can ensure everyone’s heard the message It gives everyone in the team an opportunity to ask and answer questions You can collectively agree actions and next steps A face-to-face session is more interactive and engaging, not a one-way communication that’s easy to switch off from Finally, it provides a sense of closure for the project team, and an opportunity to review what worked and what didn’t If you created a project area, then that’s the best place to hold the session Allow at least an hour, to enable you to cover a mix of activities For a larger scale discovery project, you might need as many as three hours for a workshop The first part of the session is a playback of findings: this could be a show and tell, or a more formal presentation The second part of the session is about agreeing actions Presenting Findings Firstly, present back the findings If everyone in the room was present throughout the project, and took part in the analysis, then this could be very brief: just a quick recap to confirm you’re all agreed on the same interpretation Usually, though, you’ll need to spend the first part of the session playing back the findings to the team To this, use a combination of the pyramid story you generated during the analysis process (see Chapter 8,), documentation such as a report deck 179 Researching UX: User Research or experience map, and any additional pieces of evidence like photos or videos Tailor your presentation to your audience It’s normal to encounter a lack of knowledge, as well as resistance and blinkered expectations when you feed back, so you need to think of the best way to counteract this As you can see from the diagram below, you can deploy the different elements of your research project to increase buy-in Encourage questions and discussion through the presentation too 9-7 Feedback model Agreeing Actions The whole purpose of your project is to generate action As we’ve mentioned already, the best way to this is by continually engaging and making the research relevant to your stakeholders Now, though, you can use collaborative activities to shape and assign actions There are many techniques on offer, but these are some of the most useful: Turning Research Into Action 180 Empathy Mapping Complete a different empathy map for each set of users Ask your audience to work in groups to a worksheet like the one below, reflecting what they’ve seen and heard about user needs 9-8 Empathy map example Prioritization Grid Create a 2x2 framework like the one below One axis should be labelled ‘effort’, while the other should be called ‘impact’ Map possible solutions onto the framework When you’ve finished, focus your efforts on the solutions that are in the low effort / high impact quadrant 181 Researching UX: User Research 9-9 A prioritization grid Agree Backlog If you’re already managing a backlog of product changes, you can add the actions from the research onto your existing list If you’re starting from scratch, then work as a group to create recommendations in a list report (see above) Roadmapping This is a particularly good activity if you’ve created an experience map Looking at the experience map, identify user problems you could address with new initiatives and add them as Post-It notes in a different color Or if you’ve got an existing roadmap, add the activities to the experience map to double-check you’re targeting relevant areas Turning Research Into Action 182 Top Priorities Ask each participant to complete Post-It notes capturing recommendations for action during your presentation, or generate them in small groups afterwards Group these together on a wall to remove duplicates, then get the whole team to vote with sticky dots on the ones that should be top priority Summary The more engaged you get your team, the more likely your research will be acted on Make plans to involve them throughout the project Choose the right format(s) to document your research, but don’t overdo it Speed is normally more important than detail Use a pyramid story structure, allowing you to get your message across both as a 60-second summary and as an in-depth account Tailor your delivery to your audience, using the right mix of insight, evidence and ideas to engage and persuade them Host an interactive debrief workshop Use interactive exercises to get the team to generate and own an action plan so your research has an impact 183 Researching UX: User Research ... you’re being a good designer too We’ll walk you through the steps you should take to engage with participants and their data with respect and care xviii Researching UX: User Research Enough... or users, or customers, and engaging with them deeply as real, rounded people with needs, motivations, emotions and desires Think of your research as a collaboration between your team, as designers,... participate in and feel ownership of xiv Researching UX: User Research At the same time, design research can sometimes seem a daunting, confusing world unto itself With its own codes and jargon, it