Remote research: the positives and the caveats
Apr 22, 2020
Remote research encompasses a range of methods and practices, and, whatever the project, we can almost always find a remote method which gathers enough insight to inform decisions and drive a project forward. While COVID-19 is currently putting an unprecedented demand upon remote communication, remote research has been successfully applied since the technology for remote communication have been available to us.
In this article, I explore how remote methods help you to build rapport with participants, identify needs, pain points and delights, and gain an intimate understanding of people’s experience. However, I would not be a thorough researcher if I didn’t also identify the limitations to only conducting research remotely. I also highlight how, in particular circumstances, observational and contextual research have no real substitute.
Remote research: The positives and possibilities
Usability testing and interviews are two core methods employed by user researchers and are regularly conducted using phone and screensharing technology. As the Service Design and User Research Lead on my current client project, I have been conducting remote interviews and usability tests with users of the existing product we’re seeking to improve
This research didn’t come without challenges: communicating in the inconvenient circumstances of coronavirus lockdown from our respective homes, we experienced the additional disturbance of loud DIY from my neighbour above, and some participants’ inability to screenshare. Despite such disruptions, however, these sessions have allowed our team to confidently identify user needs and pain points, gather valuable feedback on our prototype, challenge some assumptions, have in-depth discussions about the findings, and populate our backlog with tasks for future sprints.
This is just one of many examples where I have used remote usability testing or interviews to gather valuable information. Others include: telephone interviews with customers of the Financial Service Compensation Scheme for their Service Design Project; usability testing with customers of a government financial compliance service, preparing the service for its Live Beta GDS assessment; mapping the diagnostic and management pathway for patients with Lewy Body Dementia for Newcastle University through telephone interviews with clinicians of different services.
It is often stated in-person is better than remote, but I have found video conferencing and the telephone both generate an immediacy that can in some ways be considered better, rather than worse, for establishing a rapport with participants. This is not always the case but in my 10 years as a researcher I have been surprised to learn people are often more open and candid during telephone interviews than in-person, perhaps because they feel less self-conscious and more in control. (They can leave whenever they like!).
Some research methods are remote-by-default rather than an alternative for when in-person or contextual research isn’t an option. These include diary studies and surveys, both of which are purposely designed to gather information without a researcher needing to be present. A diary study is a great way to access information about the ‘day-to-day’ of people’s lives.
Inspired by the uniqueness of the times we’re living in during the coronavirus pandemic, I decided to conduct a diary study with people across the world living in lockdown. Over three consecutive days people replied to a set of question over email, telling me what they had done that day, who they spoke to, their highlights and low points. People were candid and reflective, telling me about their worries, the importance of social connection, the things they missed, the new things they were appreciating. I then had telephone interviews with participants, asking questions framed around their diary entries. I’m excited to now be making a podcast from this material!
Remote research: the caveats
Remote research doesn’t come without its downsides and limitations. For one thing, the reliance on technology for a lot of remote research means your sessions are put at risk by technological failure. With remote you can easily lose internet connection or experience some other technological failure that interrupts the session completely.
Nevertheless, these tend to take the form of technical hitches that prevent the research process from feeling seamless rather than preventing research happening at all. Having a ‘Plan B’ can ensure you still get value from the session: I prepare an interview script in case a participant can’t access a prototype for usability testing and ring people if the internet goes down.
In many cases, remote research does enough to gather the insight you need to make informed decisions about the direction of your product or service. However, particularly at the discovery stage, or for a service design project where you want to fully understand a context and its people, remote methods cannot substitute observational and contextual methods for their capacity to generate a lot of in-depth information. These methods include participant observation, shadowing, contextual usability testing, contextual interviewing. Observational methods allow the researcher to notice things their participants would not directly share: things that are tacit, assumed and go unspoken. Behaviour, values, culture, context, process and relationships are all difficult for people to articulate.
Another important limitation is that in some instances certain kinds of remote research are not suitable for accessing certain users. Some people may not have the technological confidence or ability to access your prototype and screenshare, they may not have a good/any Wi-Fi connection or have access to a home computer or smartphone. To make sure you design for all users, and not just the tech savvy, we need to reserve time to reach users in a way that works for them. A semi-structured telephone interview is often a valid option.
Finally, many of us by now will have experienced the difficulty of talking with friends, family and colleagues in group meetings on apps like Zoom, Houseparty, and WhatsApp video. In such communication platforms people accidentally talk over each other, the sound or image quality is often poor, and they tend to favour extroverted people whose voice can be heard above the echoes and distortions caused by a poor connection. This illustrates how difficult it can be to conduct a remote focus group or the remote equivalent of a workshop which involves group practical activity. At the moment we are reliant on the tools and technology improving before remote group research becomes a strong option.
Choosing your methods should always be a response to the research questions you’re trying to answer and the conditions you are attempting to answer them in, with COVID-19 being a uniquely relevant current set of conditions. This article has explored how remote research can be powerful, useful and more than sufficient for generating the insights a team needs to move a project forward. However, for an in-depth and more subtle understanding, observing behaviour in its natural context cannot be replaced.