An Integrated Approach to Interaction Design and Log Analysis

An Integrated Approach to Interaction Design and Log Analysis

Gheorghe Muresan (Microsoft Corporation, USA)
Copyright: © 2009 |Pages: 29
DOI: 10.4018/978-1-59904-974-8.ch012
OnDemand PDF Download:


In this chapter, we describe and discuss a methodological framework that integrates analysis of interaction logs with the conceptual design of the user interaction. It is based on (i) formalizing the functionality that is supported by an interactive system and the valid interactions that can take place; (ii) deriving schemas for capturing the interactions in activity logs; (iii) deriving log parsers that reveal the system states and the state transitions that took place during the interaction; and (iv) analyzing the user activities and the system’s state transitions in order to describe the user interaction or to test some research hypotheses. This approach is particularly useful for studying user behavior when using highly interactive systems. We present the details of the methodology, and exemplify its use in a mediated retrieval experiment, in which the focus of the study is on studying the information-seeking process and on finding interaction patterns.
Chapter Preview

Logging The User Interaction: An Introduction

A good understanding of people – what they are like, why they use a certain piece of software, and how they might interact with it – is essential for successful design of interactive systems, which help people achieve their goals. While each user is unique, and may have a particular background, context, interest and motivation to use a system, it is necessary to learn what is generally true about the users of a system and what behavioral patterns are common. Specifically, the designer should learn (1) the users’ goals in using a system; (2) the specific tasks undertaken in order to achieve some goals; (3) the language or terminology used by users to describe what they are doing; and (4) the users’ experience and skills at using a certain kind of system (Tidwell, 2006). Some common methods and techniques used before and during system design in order to understand the users’ needs and to establish system requirements, as well as during the implementation and testing in order to evaluate the usability and effectiveness of a system, are direct observation, interviews, surveys, personas, focus groups.

While these methods are excellent tools for evaluating the quality of the interaction between human and system, the quality of the system in supporting the users to achieve their goals and the user satisfaction, they have a number of drawbacks. First, people are often incapable of accurately assessing their own behaviors, especially when removed from the context of their activities (Pinker, 1999) and therefore interviews and surveys may not provide true answers. Second, direct observation may be obtrusive – the users may be distracted, or they may not behave naturally. Third, they are expensive to run, and therefore provide information from a rather limited sample of users, so the results are often informative, but may lack statistical significance, may miss unusual cases, and may not capture behavioral patterns or trends.

Logging the user interaction with the system provides a complementary tool for analyzing the interaction and evaluating a system. It provides the means to acquiring large quantities of data about patterns of interface usage, speed of user performance, rate of errors, or frequency of requests for online assistance (Shneiderman & Plaisant, 2005). An important ethical issue, which indirectly affects user behavior and therefore the validity of the results, is whether users are told and know that their activity is logged. However, when logging is done in order to evaluate a system rather than user preferences or private activities, and when no personal information is captured, this problem is minimal.

An interesting set of constraints on what data can practically be logged, and on designing a logging system, is dictated by the software architecture of the system being investigated. The simpler situation is that of a standalone system, when the entire user activity runs on the same machine, and where all the data resides. In such situations, if the logging module is designed and built as part of the system, then all user actions, all user events and all data being generated or manipulated can potentially be logged. Logging the interaction with third-party software is more challenging: while operating system-level actions such as keystroke or mouse events, or opening/closing a file, or starting/stopping a certain application can be captured and logged, semantic events specific to a certain application are usually impossible to capture. For example, while it is possible to capture the text typed by a user, it is not easy or even possible to determine if the text was typed as a query for a search engine, or for filling in a form. This problem can be addressed by video-recording the interaction or by using screen-capturing software (e.g., Morae:, so that the researchers can subsequently examine the interaction, interpret what is happening, insert annotations or mark significant events. While these tools can be helpful in analyzing the captured data, they rely on the manual-intellectual annotation done by the researcher, and are therefore very labor intensive and error-prone. Moreover, the format used for the logs is usually proprietary, which forces the researchers to buy proprietary analysis software that is not customizable. So, in order to fully benefit from the power of user activity logging, it is preferable that the designer of the logging module has access to the source code of the system being evaluated.

Key Terms in this Chapter

Mediated Information Retrieval: A model of IR interaction in which the systems supports the user’s exploration of the information space and the formulation of queries.

Interaction Design: Designing interactive systems that support certain functionality and a range a user behaviors.

Logging Module/System: Component of an interactive system that logs/records relevant interaction between the user and the system (events, user actions, system responses).

User Behavior: The set of actions taken by a user interacting with the system in order to reach a goal or complete a task.

State Diagram (Statecharts): Model of an interactive system that describes (i) a finite number of existence conditions, called states; (ii) the events accepted by the system in each state; (iii) the transitions from one state to another, triggered by an event; (iv) the actions associated with an event and/or state transition.

Interaction Schema/Model: A formalized description of interaction rules and actions allowed in specific contexts.

Log Analysis: The analysis of user behavior based on the actions recorded during interaction.

Complete Chapter List

Search this Book:
Table of Contents
Bernard J. Jansen, Amanda Spink, Isak Taksa
Chapter 1
Bernard J. Jansen, Isak Taksa, Amanda Spink
This chapter outlines and discusses theoretical and methodological foundations for transaction log analysis. We first address the fundamentals of... Sample PDF
Research and Methodological Foundations of Transaction Log Analysis
Chapter 2
W. David Penniman
This historical review of the birth and evolution of transaction log analysis applied to information retrieval systems provides two perspectives.... Sample PDF
Historic Perspective of Log Analysis
Chapter 3
Lee Rainie, Bernard J. Jansen
Every research methodology for data collection has both strengths and limitations, and this is certainly true for transaction log analysis.... Sample PDF
Surveys as a Complementary Method for Web Log Analysis
Chapter 4
Sam Ladner
This chapter aims to improve the rigor and legitimacy of Web-traffic measurement as a social research method. I compare two dominant forms of... Sample PDF
Watching the Web: An Ontological and Epistemological Critique of Web-Traffic Measurement
Chapter 5
Kirstie Hawkey
This chapter examines two aspects of privacy concerns that must be considered when conducting studies that include the collection of Web logging... Sample PDF
Privacy Concerns for Web Logging Data
Chapter 6
Bernard J. Jansen
Exploiting the data stored in search logs of Web search engines, Intranets, and Websites can provide important insights into understanding the... Sample PDF
The Methodology of Search Log Analysis
Chapter 7
Anthony Ferrini, Jakki J. Mohr
As the Web’s popularity continues to grow and as new uses of the Web are developed, the importance of measuring the performance of a given Website... Sample PDF
Uses, Limitations, and Trends in Web Analytics
Chapter 8
Danielle Booth
This chapter is an overview of the process of Web analytics for Websites. It outlines how visitor information such as number of visitors and visit... Sample PDF
A Review of Methodologies for Analyzing Websites
Chapter 9
Gi Woong Yun
This chapter discusses validity of units of analysis of Web log data. First, Web log units are compared to the unit of analysis of television to... Sample PDF
The Unit of Analysis and the Validity of Web Log Data
Chapter 10
Kirstie Hawkey, Melanie Kellar
This chapter presents recommendations for reporting context in studies of Web usage including Web browsing behavior. These recommendations consist... Sample PDF
Recommendations for Reporting Web Usage Studies
Chapter 11
Seda Ozmutlu, Huseyin C. Ozmutlu, Amanda Spink
This chapter summarizes the progress of search engine user behavior analysis from search engine transaction log analysis to estimation of user... Sample PDF
From Analysis to Estimation of User Behavior
Chapter 12
Gheorghe Muresan
In this chapter, we describe and discuss a methodological framework that integrates analysis of interaction logs with the conceptual design of the... Sample PDF
An Integrated Approach to Interaction Design and Log Analysis
Chapter 13
Brian Detlor, Maureen Hupfer, Umar Ruhi
This chapter provides various tips for practitioners and researchers who wish to track end-user Web information seeking behavior. These tips are... Sample PDF
Tips for Tracking Web Information Seeking Behavior
Chapter 14
Sandro José Rigo
Adaptive Hypermedia is an effective approach to automatic personalization that overcomes the difficulties and deficiencies of traditional Web... Sample PDF
Identifying Users Stereotypes for Dynamic Web Pages Customization
Chapter 15
Brian K. Smith, Priya Sharma, Kyu Yon Lim, Goknur Kaplan Akilli, KyoungNa Kim, Toru Fujimoto
Computers and networking technologies have led to increases in the development and sustenance of online communities, and much research has focused... Sample PDF
Finding Meaning in Online, Very-Large Scale Conversations
Chapter 16
Isak Taksa, Sarah Zelikovitz, Amanda Spink
Search query classification is a necessary step for a number of information retrieval tasks. This chapter presents an approach to non-hierarchical... Sample PDF
Machine Learning Approach to Search Query Classification
Chapter 17
Seda Ozmutlu, Huseyin C. Ozmutlu, Amanda Spink
This chapter emphasizes topic analysis and identification of search engine user queries. Topic analysis and identification of queries is an... Sample PDF
Topic Analysis and Identification of Queries
Chapter 18
Elmer V. Bernstam, Jorge R. Herskovic, William R. Hersh
Clinicians, researchers and members of the general public are increasingly using information technology to cope with the explosion in biomedical... Sample PDF
Query Log Analysis in Biomedicine
Chapter 19
Michael Chau, Yan Lu, Xiao Fang, Christopher C. Yang
More non-English contents are now available on the World Wide Web and the number of non-English users on the Web is increasing. While it is... Sample PDF
Processing and Analysis of Search Query Logs in Chinese
Chapter 20
Udo Kruschwitz, Nick Webb, Richard Sutcliffe
The theme of this chapter is the improvement of Information Retrieval and Question Answering systems by the analysis of query logs. Two case studies... Sample PDF
Query Log Analysis for Adaptive Dialogue-Driven Search
Chapter 21
Mimi Zhang
In this chapter, we present the action-object pair approach as a conceptual framework for conducting transaction log analysis. We argue that there... Sample PDF
Using Action-Object Pairs as a Conceptual Framework for Transaction Log Analysis
Chapter 22
Paul DiPerna
This chapter proposes a new theoretical construct for evaluating Websites that facilitate online social networks. The suggested model considers... Sample PDF
Analysis and Evaluation of the Connector Website
Chapter 23
Marie-Francine Moens
This chapter introduces information extraction from blog texts. It argues that the classical techniques for information extraction that are commonly... Sample PDF
Information Extraction from Blogs
Chapter 24
Adriana Andrade Braga
This chapter explores the possibilities and limitations of nethnography, an ethnographic approach applied to the study of online interactions... Sample PDF
Nethnography: A Naturalistic Approach Towards Online Interaction
Chapter 25
Isak Taksa, Amanda Spink, Bernard J. Jansen
Web log analysis is an innovative and unique field constantly formed and changed by the convergence of various emerging Web technologies. Due to its... Sample PDF
Web Log Analysis: Diversity of Research Methodologies
About the Contributors