Usability Assessments of Mobile Applications as a Function of Geographic Location

Introduction

Recently, researchers tested the usability of 11 products and services with users across the United States to determine if participants from different locations rated the subjective usability of common products differently (Kortum & Acemyan, 2018). The products and services evaluated by these researchers included Microsoft Word, Microsoft Excel, the iPhone, Gmail, the Wii, Amazon.com, Photoshop, the United States tax form W-4, Facebook, Google search, and the Free Application for Federal Student Aid (FAFSA). The researchers found that the usability assessments of these common products and services did not vary with location. Further, this study illustrated that users in rural and urban areas did not rate the usability of these products differently.

These findings are significant because they show that regardless of where a user lives in the United States, the usability of a product, service, or system is judged to be the same. This means that practitioners are free to recruit usability test participants from anywhere in the country to evaluate their products, provided that their subject pool meets other critical demographic criteria identified to be important for that particular product (e.g., experience, technical ability, and/or prior knowledge of a particular topic).

Despite the importance of this study and its potential impact on usability assessment practices, it was limited in its scope. This scope limitation meant that it did not directly address mobile systems and applications, which comprise some of the most widely used and geographically broad systems in use within the United States today. Accordingly, this study aims to replicate the prior research by using the same methods and expanding upon it, by examining whether or not the usability assessments of mobile applications vary across 1) US geographic regions and 2) urban versus rural areas.

If we confirm again that the usability of systems does not vary with location, usability research scientists would have additional evidence that dispels the belief that users from a particular location—and their usability assessments—are unique. This belief often arises from two different sources (Kortum & Acemyan, 2018). First, usability professionals and marketing researchers often use the same demographic data to identify their user base. In the case of marketing, those demographic variables allow marketing researchers to selectively and effectively target those users who are most likely to use their products and/or encourage potential users with specific characteristics to switch to their product (Beane & Ennis, 1987; Kahle, 1986; Thomas, 1980; Weinstein, 2013). For example, marketing space heaters in Death Valley, California (the hottest location in the United States) is likely to be less productive than marketing that product in Fairbanks, Alaska (the coldest place in the United States).

In addition to geographic location, marketing also makes the distinction between urban and rural areas in their marketing efforts (Lumpkin, Hawes, & Darden, 1986; Velayudhan, 2007; Xu & Paulins, 2005). Consumers in high density urban areas are presumed to have different purchasing characteristics than consumers in low-density rural area, based on a number of criteria, including such traits as education level, socioeconomic status, exposure to technology, and mobility, just to name a few.

These differences in consumers highlight how geographic location can both play a role in marketing efficacy and aid with targeted advertisements. If there are circumstances in which user characteristics are critical, as is the case with marketing methods for products, it is understandable that human factors and usability researchers might question if location might be important when selecting users to assess product usability.

The second source of information causing people to think that subjective usability might be different in various geographic locations comes from anecdotal evidence shared by usability professionals. It is not uncommon for usability professionals to hear from their marketing and management peers that a product must be tested in location X rather than location Y; or that a product judged to be of poor usability in one location will have better ratings in another location, because people in that location are “different.” These impressions may be falsely driven by common stereotypes, prejudices, or personal experiences.