Play or Vote: Matching Games as New Approach for Design Evaluation in Innovation Contests

Introduction

Pushed through concepts like crowdsourcing (Howe, 2008; Kozinets et al., 2008), co-creation (Winsor, 2005), and open innovation (Chesbrough, 2003), firms increasingly use the creativity, skills, and intelligence of millions of individuals encountered on the Internet as source for innovative ideas. Building on the means of competition, innovation contests are one particular method to do so. Within the course of such contests, companies invite interested users to deal with a certain topic or product range, to show their talent by uploading their creative content, and further discuss and share their insights with like-minded people. Users can vote which idea or design they like best, discuss various topics by leaving comments at other users’ pin board, and compete for prizes (e.g. Morgan & Wang, 2010).

While such contests ensure a large variety of submissions, the identification of the best and most promising ones often causes large efforts. On the one hand, the mere magnitude of ideas generated can be overwhelming, on the other hand most approaches do not increase the chance of really selecting the best submissions nor reduce the risk of relying on the wrong ones (Bjelland & Wood, 2008; Riedl et al., 2010). The existence of social media and new information and communication technologies (ICT) offers new opportunities to encounter these challenges. In contrast to traditional competitions, many current innovation contests are based on IT-platforms. Participants can share their ideas, communicate with each other, establish relationships and even comment and evaluate others ideas. The latter is also referred to as open evaluation (Haller, 2011). Recent research indicates that open evaluation bears plenty of potential to support the selection of relevant submissions (Blohm et al., 2009; Möslein et al., 2010). Still, it is also recognized that many methods are prone to fraud. By using multiple accounts, participants can vote for themselves to increase their chance of winning or by voting down competitors, respectively. Hence, effective open evaluation has to avoid these pitfalls, while still tapping into the wisdom of the crowd.

Online games seem to be a promising approach. First experiments show the suitability of online games to elicit user preferences, while making it harder to cheat (Hacker & von Ahn, 2009). By predefining the ideas, designs, concepts or solutions to be judged, participants cannot vote for the submission they want to support. Further, games are perceived as an enjoyable and fun activity, leveraging the willingness of participants to share their knowledge. However, so far only a few companies use online games for this purpose. This paper introduces and discusses the use of games with a purpose for the evaluation and identification of most promising design ideas. Further, we investigate whether online evaluation games attract the attention of participants and if so, how their results correlate with other approaches focusing on the elicitation of participants’ preferences.

The remainder of this chapter is structured as follows. First, we provide an overview of innovation contests, followed by an overview on evaluation in innovation contests in general and of games with a purpose (GWAPs) as means of interest. Then we outline the methodical approach, before presenting the results of two studies and closing the paper with the discussion, future research directions and conclusion.