We asked respondents in our survey how they know when a design project is successful. User research (not just quantitative, but qualitative too) can help us determine whether our designs work as we want them to, and whether we’re meeting our goals. We don't collect any usage data, so sales and revenue exclusively drive product decisions.” Success Criteria “They're the only metrics we have access to. “My manager has access to the analytics and measures success on traffic within our products/sites rather than interpreting the numbers to extract meaningful insights.” “My company started as an A/B testing and CRO company and hasn't evolved their thinking beyond conversion rates.” “We don’t have funding for more advanced quantitative research methods (beyond click tests, surveys, etc.)” This possibility is supported by some of the open text-field comments from our respondents. However, it’s also possible that those simply reporting that their teams were just collecting analytics data in every project, not that it had any real significance for their work. It’s possible that many of the analytics-heavy respondents, who reported using quantitative research at least once per project or sometimes, used that analytics data in a meaningful way to guide their design projects - for example, to help them identify problem areas in the product. Unfortunately, in this survey, we don’t have an indication of exactly howrespondents were using analytics in their projects. 74% of respondents who reported using quantitative research at least once per project also reported using analytics “often” (86 out of 117).Īnalytics data can play a significant role in UX design. These results give some context to the surprisingly high frequency of quantitative studies in design projects. Predictably, the relatively lower-cost methodologies rank near the top (analytics, qualitative usability testing, interviews) while more expensive methodologies are towards the bottom (quantitative usability testing and eyetracking, which can be both prohibitively costly). The most frequently used methodologies, as reported by our respondents, were, in order: Analytics, qualitative usability testing, and interviews were tied for the most popular methodology, each with 67% of respondents reporting that they use it “Often” or “Sometimes.” These three were the only methods used by a majority of respondents, with everything else clocking in at 48% at the most. This stacked bar chart shows the proportion of respondents who reported using each methodology often or sometimes. For each methodology, respondents told us if they were using it “often,” “sometimes,” “rarely,” or “never.” We asked respondents to tell us how frequently their teams were using 11 popular UX research methodologies - 7 quantitative and 4 qualitative. We were somewhat surprised to hear that the majority of our respondents (71%) said they performed quantitative research either “sometimes” or “at least one study per project.” This result makes sense when we look at the methodologies that respondents report using. (Although, since you are reading this article, that likely means the results of this survey are probably fairly representative for the kinds of projects you work on, even though they may not be representative for all design projects in the world.) I’d bet that sample (and probably you, since you’re reading this) likely performs more quant research on average than the wider UX community as a whole. As a result, our sample consisted of NN/g-fan UX practitioners who potentially were interested or had heard of quantitative research methods. We also mentioned that the survey was about quant practices. We recruited our respondents from Twitter and LinkedIn outreach, and offered a chance at a free report or online seminar as an incentive. When interpreting these results, bear in mind that there may be some sample bias at play. The majority of respondents said their teams perform quant research at least occasionally: 44 % report doing it “sometimes” and 27% say they perform one or more quantitative studies per design project.
0 Comments
Leave a Reply. |