Measurements related to user expectations, behaviors and experiences can provide useful data to many roles and teams in a company. Each role provides different views to the question “what should be measured, and why?” We conducted an empirical study on user experience (UX) measurement needs at different units and levels in one corporate (Nokia) and asked which kinds of UX measurements would be useful for different functions. We identified common UX measurement needs on 8 different themes.
TopIntroduction
In all phases of product development, measurements play an important role as they enable systematic improvement of products. Often measurements are designed and conducted in ways that serve only limited audience and purposes. By studying corporate organizations as systems and expecting organizational units to share common interests, it is possible find better ways to conduct measurements and also to apply the measurement data more efficiently.
We can identify measurements that are made before, during or after the development, and measurements that are needed across disciplines and functions. In the development of consumer products the data from users and customers is needed when creating the products, and also to provide actionable feedback about the developed products.
Usability measurements investigate whether the designs are easy and efficient to use for the target user group. As usability has evolved to user experience (UX), we are also interested to know how users feel about using the product. This feeling is affected not only by usability (how I get tasks done), but also by the beauty of the product, brand image, and support services, for example. Therefore, while the customer for usability measurements is typically the R&D department, user experience information interests also people at, for example, sales point, support service, and customer care.
We wanted to understand how UX measurements would help the extended group of stakeholders and whether new measurement needs and requirements could be identified. In specific, our research questions were:
We conducted an email survey with the following company activities: research, concepting, product development, sales, customer care, and quality management. Based on the findings, we developed guidelines to help designing a UX measurement system that serves many organization units in a corporation.
In this article, we first describe earlier work on UX and related measurements. We then describe the study and main findings. We conclude our article with discussion and implications, including a set of guidelines and a process proposal.
TopA large company should not be looked at as a single community of practice but as a federated set of multiple (and often competing) communities of practise and knowing (Klein & Hirschheim, 2008). Each community consists of the knowledge, concepts, observations, values, meanings, assumptions, beliefs, and so on that comprises a community’s shared ‘‘thought-world”. According to Klein & Hirschheim, the more communication and shared knowledge there is across communities, the better an organization can achieve its objectives and improve. They propose a new research priority to create better ‘‘understanding of organizational stakeholders” and to develop social boundary spanning ‘‘knowledge creation and transformation networks”.
It is important for companies to keep old customers and recruit new ones, that explains why customer satisfaction has been measured for long with different kinds of measures and in different kinds of teams, communities and activities. From company viewpoint, user experience measurements are close to customer satisfaction and usability measurements, so we will shortly discuss what metrics are in use in these different fields.
Customer satisfaction models come from the field of commerce and many of them reach beyond product usage to all customer touch points. The touch points are interfaces where a customer encounters a company: products, advertisements, sales, care centers, Web sites, etc. (Peppard, 2000). A Net Promoter Score (Reichheld, 2003) is seen as an important measure for a company to find new customers, as it measures how many people would recommend the product to others. In the U.S., customer satisfaction is measured annually with American Customer Satisfaction Index, ACSI (Fornell et al. 1996). The metrics used in ACSI are overall satisfaction, expectancy disconfirmation, and perceived performance compared to ideal performance.