Cognitively Informed Intelligent Interfaces: Systems Design and Development

Cognitively Informed Intelligent Interfaces: Systems Design and Development

Eshaa M. Alkhalifa (Royal University for Women, Bahrain) and Khulood Gaid (Royal University for Women, Bahrain)
Release Date: May, 2012|Copyright: © 2012 |Pages: 383
ISBN13: 9781466616288|ISBN10: 1466616288|EISBN13: 9781466616295|DOI: 10.4018/978-1-4666-1628-8


Humans interact with the world through perception, reason about what they see with their front part of their brains, and save what they experience in memory. They also, however, have limitations in their sight, hearing, working memory, and reasoning processes.

Cognitively Informed Intelligent Interfaces: Systems Design and Development analyzes well-grounded findings and recent insights on human perception and cognitive abilities and how these findings can and should impact the development and design of applications through the use of intelligent interfaces. Many software and systems developers currently address these cognitive issues haphazardly, and this reference will bring together clear and concise information to inform and assist all professionals interested in intelligent interfaces from designers to end users.

Topics Covered

The many academic areas covered in this publication include, but are not limited to:

  • Adaptable systems
  • Applications of cognitive science findings
  • Cognitively-informed interface design
  • Decision support technologies
  • Distance learning technologies
  • End user computing
  • Human Computer Interaction
  • Human side of IT
  • Intelligent interface design
  • Web-based Learning and Teaching

Reviews and Testimonials

All the ideas presented in this book, creative and enjoyable as they are when gone through in detail, end up in the very same vision of improving our lifestyle and leading us to a better future, especially in the education and health systems.

– Ahmed Dabbagh, Ankabut Project, UAE

Table of Contents and List of Contributors

Search this Book:


Windows to the World!

A human being can only experience the world through the mind’s eye.  Human senses perceive the inputs received from the environment, and the mind interprets them to produce understanding.  Limitations in the cognitive system directly impacts what we understand. An example that is established exists in visual tricks.  Limitations in the visual system cause interpretations of what is seen to be manipulated.

However, this book does not describe old knowledge. Instead, it covers new emerging concepts.  To understand for example, how our conceptualization of time limits how to we analyze a question ponder this question.

“I do not believe what you believe and you do not believe what I believe,

I will not believe what you will believe and you will not believe what I will believe in the future.

Did we at any time in the past share the same belief?”

Now, if the reader tries to forget that question and think only of the following one:

“I did not believe what you believed and you did not believe what I believed in the past,

I do not believe what you believe and you do not believe what I believe in the now.

Can we at anytime in the future share the same belief?”

The two questions are logically equivalent, with the first question setting the scene for the present and future time slots as having a disagreement while leaving the past without any input about it while the second places disagreement in the past and present time slots while leaving the future without input.  Human inability to change the past caused those tested to mostly answer NO to the first question and YES to the second.  It is not a logical choice as the possibility should be technically possible for both questions.
These limitations that guide the way we analyze the world can be studied and utilized to increase the efficiency and effectiveness of interfaces.  This book focuses on interfaces that are designed for computer based systems, for smart phones, or for intelligent paper books.  Screen set up, attention paid to different color arrangements, and general interface design issues are discussed here.  One critical example is to detect suspicious areas in images of patients in order to attract doctor attention to those areas.  Human perception is limited and confused if the images have uniform shade or color.  Consequently, software is designed to detect suspicious areas and to highlight them with a bold color to ensure that doctors attend to those areas.  The doctors can then dismiss the area or submit it to further testing.  Here, suggested systems assist limits in perception to reduce human error in medicine.

Another emerging direction is to make devices more human like.  An agent can be designed to exhibit human characteristics to resemble a specific teacher that the designer once knew, or to resemble a very creative salesman. Visitors of these sites can remember that agent and describe him as being shy, honest, and funny, for example.  Adding the human traits would make a site more distinguishable from the others.

This book is of interest to a vast spectrum of readers that range from managers who wish to see where interface technology is heading, to average people who are just curious about the inventions that are just around the corner.  The information contained may be crucial to managers of companies involved in interface design or applications that interact with customers.  The reason is that the chapters of this book exhibit advances in various areas that bid attention.

Chapter 1 is entitled “Multitasking Bar: Prototype and Evaluation of Introducing the Task Concept into a Browser” and written by Eric Wang, Huiyon Chang, and Huiyang Liu.  People multitask in two ways whenever using a web browser. They can either open multiple tabs to perform a single task. For example, if one is searching for a hotel to stay in when visiting a foreign country, it is quite typical to browse different hotel pages on several tabs all open simultaneously.  On the other hand, it is also possible that the same person wishes to perform a task during more than one online session. One day, one browses a few hotel sites then one leaves the task at that point. The next day after speaking to a friend, one comes back to browse a different hotel in the same site.  This chapter introduces a Multitasking Bar that can be added to browsers with the ability of classifying historical data of task performance. The goal is offer more support to people who surf the Internet whether it is professional or for leisurely activities.

Chapter 2 is entitled “Soft Keyboard Evaluations: Integrating User’s Background in Predictive Models” and written by Bruno Merlin and Mathieu Roynal.  The authors examine soft keyboard predictive evaluation models. Soft keyboards are currently used in touch screen phones and Ipads.  Increasing input speed of these keyboards has become a very important issue.  Existing keyboards as in Qwerty are based upon very old typewriter technology which have nothing in common with touch screen mobile phones. The authors indicate that predictive models of evaluation of best and worst possible performance do not conform to empirical data.  Additionally, new keyboard designs are possible in the new mediums that now exist.

Chapter 3 is entitled “A Fresh Look at Graphical Web Browser Revisitation Using an Organic Bookmark Management System”  and written by Siu-Tsen Shen, Stephen D. Prior, and Kuen-Meau Chen.  This chapter presents a graphical medium to represent information displayed in the history folder of web browsers.  The graphical representation resembles a pot with a growing plant that relates different sites visited in the history folder.  The goal is to relate webpage to each other to make it easier to find previously visited web paged.  Since existing searches in the History folder is more like searching for a needle in a haystack unless the visit of the site was extremely recent, this is an avenue that bids researcher attention of how to make that representation more meaningful to users of the browsers.

Chapter 4 is entitled “MobiGaze: Gaze Interface for Mobile Devices” and written by Takashi Nagamatsu, Michia Yamamoto, and Hiroshi Sato. This chapter describes a user interface that uses human’s gaze to operate a handheld mobile device.  It is difficult to use a touch screen device with one hand so a touch plus gaze tracking interface was designed.  This interface uses the surface revolution around the optical axis of the eye as a basis. One implementation that the authors may consider in the future is to utilize this interface to serve the physically impaired people to enable them to increase their level of interaction with the world and increase their quality of life.

Chapter 5 is entitled “Evaluating Visualisations and Automatic Warning Cues for Visual Search in Vascular Images” and written by Boris Van Shooten, Betsy Van Dijk, Avan Suinesiaputra, Anton Nijholt, and Johan H. C. Reiber.  This chapter evaluates visualization approaches that assist in visual search in vascular images. A patient wishes that his or her doctor can immediately and correctly diagnose the existence of a condition or the lack of it. They compare four visualization techniques then discuss the professional opinions of users of these systems which indicate a preference to “paranoid” highlighting of areas of suspicion.  In other words, the authors recommend that physician support systems should leave the decision in cases of confusion to the doctor in change rather than try to build systems that may cause misdiagnosis. They also note that if the system does not highlight an area, then the doctor is not likely to examine that area at all even though it may have a small tumor.  Consequently, visual interfaces should take this tendency to ignore areas that are not highlighted into account especially in the field of medicine where human life may be at risk.  

Chapter 6 is entitled “Considering the Influence of Visual Saliency during Interface Searches” and written by Jeremiah D. Still and Christopher M. Masciocchi.  The authors examine the areas on the screen visitors of the web pages attend to. Saliency represents how unique an object is on the screen when compared to its surroundings.  The authors compare the bottom-up approach to the top-down approach. The bottom-up approach investigates how parts of the screen attract attention in the presence of distracters. The top-down approach is goal driven when a person visually searches the screen looking for an object.  The authors highlight the role of bottom-up mental processes that can be easily extended to web pages. Web page designers can use this knowledge to attract the attention of web page visitors to specific areas on the screen.  The authors also examine the existence of a correlation between saliency and eye fixations using an eye tracker.

Chapter 7 is entitled “Design and Evaluation of a Multimodal Representation for Conveying the Vast Range of Imperceptible Smallness” and written by Minyoung Song.  When teaching students about objects that are too small to see, these objects are frequently enlarged and displayed.  The authors of this chapter argue that students fail to grasp the distinct classes of sizes these objects fall into.  Some of these very small objects may be much larger than others.  The chapter describes an education tool that uses time in addition to enlarging the object image.  Objects are also placed next to a reference object with a given visible size like 1 millimeter.  The larger the small object is, the longer the animation takes to place it on the screen when compared to a smaller object that is placed more quickly on the screen.  The authors justify this approach to be based upon cognitive science research that found that the mental representation of space and time are interwoven.  The educational tool is tested with 7th grade students with positive results.

Chapter 8 is entitled “Interactive  Diagram  Layout” and written by Sonja Maier and Mark Minas.  Layouts in general are relevant to any work that involves drawing a structure as in the structure of a text, or representing how different concepts relate to each other.  The authors of this chapter assigns different groups of students the task of developing layout editors.  They found a similarity between the layout editors that different groups of students created.  The chapter also discusses a proposed pattern based layout approach that allows the definition of a combination of different layout algorithms rather than to display each algorithm alone.  This approach produces good results according to the authors when integrated with existing visual language editors.

Chapter 9 is entitled “Optimization in Mental Configuration Space: The Case of Pointing Behavior” and written by Huahai Yang.  This chapter investigates how humans mentally represent points in two dimensional space.  It starts by reviewing a number of existing behavior models.  Configuration space is defined as a space with one axis per variable. Finding points represented in two dimensions will require a representation on two axes (x,y). If there are three dimensions, then the representation will be in the form (x,y,z) and this would point to one point that is described in three dimensions.  The authors gave students a pointing task on the screen and found that their behavior was best explained by the configuration space model.  Other models failed to compete with this predictive ability of the configuration space model.  This may be indicative of how humans conceptualize and reason about physical configuration in real life.

Chapter 10 is entitled “Question-Answer Approach to Human-Computer Interaction in Collaborative Designing” and written by Petr Sosnin. A software intensive system is software in one or more locations that continually adapt to the environment by interacting continually with it.  This chapter presents a question/answer approach as a communication medium between humans and the system, and also between computer processors.  Question/answer pseudo code programs are explained to be a useful means of interaction that adapt to problems’ needs and can be automatically and easily translated to an executable that is run on a compiler.  One outcome of such a theory may be a system that generates software codes upon interacting with a human through the question/answer communication medium provided.

Chapter 11 is entitled “Intelligent Agents with Personality: From Adjectives to Behavioral Schemes” and written by François Bouchet and Jean-Paul Sansonnet.  The authors describe an extensive effort made to offer a tool to scholars who wish to give intelligent agents a personality. They used two references for their work and found a good match between them.  Their first source of information included psychological research that classified human personality traits.  Their second source of information came from a complete list of adjectives that describe people in the English Language.  Perhaps the assumption made here is that the word would not exist unless there is a personality trait it can describe.  Their contribution is a taxonomy available on the Internet that describes all personality traits and can be used to give intelligent agents a personality.

Chapter 12 is entitled “The Activity Circle: A Social Proxy Interface to Display the Perceived Distributed Viscosity about Workflow Technology” and written by Marcello Sarini.  Workflow technologies are designed to allow collaboration between employees and to streamline their output to perform the organization’s objectives.  Simultaneously social software focuses on conversation and interaction between users.  The goal of this chapter is to visually display perceived viscosity to help employees realize more about themselves, the roles they assume, and others in the organization.  Management may also use this information to identify conflicts and issues that may impede work in the organization.

Chapter 13 is entitled “A Global Process for Model-Driven Approaches in User Interface Design” and written by Sybille Caffiau and Patrick Girard. The authors of this chapter present a process for user interface design that can start from a task based model or from a user defined simple prototype.  A task model is based upon specifying the tasks an interface has to perform and generating the design from that point. A hierarchical design on the other hand, starts by defining the main tasks performed and based upon them, the subtasks are designed and so on. User defined prototypes are designed as small working systems that are shown to clients following which they are either replaced or expanded into the full working system.  The chapter shows how iterative modifications using rule additions while checking these rules.  The generative approach generally is associated with errors that are avoided by the authors in this approach because new rules are added without affecting the process.

Chapter 14 is entitled “Programming a User Model with Data Gathered From a User Profile” and written by Danial Schere, Ademar V. Netto, Yuska P. C. Aguiar, and M. F. Q. Vieira.  Interfaces in general need to learn about those who use them in order to reduce the cognitive load on users. This chapter proposes a combined approach of building a user profile and a programmable user model.  The relationship between the two is explored to examine the erroneous assumptions that are sometimes made in user characteristics and correct them.  Future directions of work may incorporate biometrical data in the user model.

Chapter 15 is entitled “The Bootstrap Discovery Behaviour Model: Why Five Users are not Enough to Test User Experience” and written by Simone Borsci, Stefano Federici, Maria Laura Mele, Domenico Polimeno, and Alessandro Londei.  This chapter examines how to estimate the number of users necessary to test user experience.  The authors examine the Return On Investment model that helps predict the number of users necessary to evaluate a system.  They argue that this mode is not suitable if testing has to be performed by more than one category or users.  They proposed that the bootstrap model makes a more accurate estimation of the number of users needed to test any system. They find that in their case, 5 is not enough for testing.

Chapter 16 is entitled “Designing ‘Faster’ Progress Bars: Manipulating Perceived Duration” and written by Chris Harrison, Zhiquan Yeo, Brian Amento, and Scott E. Hudson.  Progress bars are an integral part of the installation process. The main goal of their presence is for the system to inform the person performing the installation that the process is in progress. However, not all progress bars were created equal. Some get stuck at one location for a long while before starting to advance again.  This may happen if the folder being copied is larger than the others.  Many complaints are sent by impatient customers who stopped the installation because they assume the installation was stuck and could not proceed to completion. This chapter is cognitively informed about customers so it uses knowledge about human perception to retain human confidence in the installation process and gives the impression that it is also a fast installation.

Chapter 17 is entitled “Dynamic Generation of Adaptive Tutoring” and written by Khulood Gaid and Eshaa M. Alkhalifa.  Adaptive Educational Systems are able to alter the online course as per the needs of each student.  Such a powerful system has a large overhead in the time and effort required to design and build such courses.  This chapter offers a practical adaptive system that is automatically built as instructors upload their tests to the online site.  In short, the system asks the instructor to associate multiple choice questions that are incorrect with error pattern names and to associate the error patterns with lessons they need to review.  The result is that the adaptable system is dynamically built as the course progresses.  An instructor can use reports to extract common error co-occurrences and infer information about misconceptions at the end of the course.

Chapter 18 is entitled “Interactive Picture Book with Story-Changeable System by Shuffling Pages” and written by Hiroki Yamada and Michitaka Hirose.  The authors of this chapter hope to add digital content to paper books. The idea is not as farfetched as it may seem. They use very thin Integrated Circuit (IC) tags and a Radio Frequency Identification Antenna (RFID). Users can flip through the pages of this book that allows them to change the story line in one of many possible patterns.  A particular feature that may be of interest in this work is that this makes it possible to add some action to “kototama” words.  The authors indicate that in Japan words are believed to have their own spirits and special force since early times so they call them spiritual words.  Words come to life in this chapter as their meanings turn into actions, whether by sound or animation. “Kototama words are projected around the pages of the book as if they were ‘floating’ outside the world of the book” (excerpt from the chapter). Since in many cases fictional movies predict the future path of science, it may not be purely magic that allows words to float on a page, if science catches up with human imagination.  This chapter is a definite ‘must read’.

To sum up, this book presents work that may enter our lives in the not so distant future as devices or upgrades to existing tools.  It is merely a stage, in the explosive reaction of cognitively informed systems. 

Eshaa Mohamed Alkhalifa
Royal University for Woman, Bahrain

Author(s)/Editor(s) Biography

Eshaa Mohamed Alkhalifa has obtained her BSc in Computer Science of the University of Bahrain, her MS in Artificial Intelligence from the University of George Washington, and her PhD in Cognitive Science from the University of Edinburgh in Scotland. She has published a number of books, and a large number of journal and conference papers. She held the role of the Academic Vice President of the Royal University for Women until recently when she took some time off to focus on research. Her research interests include the utilization of Cognitive Science findings in increasing the efficiency of learning in general or elearning in particular.
Khulood Gaid has a Bachelor degree in Computing Science from the Royal University for Women, Bahrain. She is currently working as a System Analyst at She is planning to continue research in the field of Human Computer Interaction and earn her Master’s degree. Khulood had worked on many projects in the field of educational adaptive systems. Her next research projects will concentrate on the usage and implementation of cognitively informed interfaces in educational systems.


Editorial Board

  • Ahmed Dabbagh, Ankabut Project, UAE
  • Mohamed Ahmed Juman, AMJ Projects, Bahrain
  • Ali Athar Khan, University of Bahrain, Bahrain
  • Kinshuk, Athabasca University, Canada