TopPreface
Introduction from the EditorThis book gathers the contributions of authors working in the area of visual languages from countries
as diverse as Spain, Italy, United Kingdom, Australia, Greece, Canada, Brazil, and the USA.
It consists of 22 chapters written by a total of 54 different authors to provide an in-depth investigation
of new approaches and methods for visual language theory, visualisation techniques, aesthetic
computing, and usability.
I hope that academics, researchers, technicians, and students in computer science will find this book
a valuable contribution to their knowledge of this important research area.
In normal life, people organise a complex structure of signs to communicate with the world around
them. The brain acts as an interface between the sign system and the environment, orienting the individual
in space. Visual languages systemise the visual representation of concepts in a formalised language
using visual objects such as images or pictorial objects and possibly formal visual expressions. They
are based on simple visual elements, which construct symbols that can be grouped to form structured
visual sentences.
The Overall Objective of This Book
The main objective and mission of this book is to present problems and methodologies concerning the
syntax, semantics, and ambiguities of visual languages. Various syntactic techniques can be used to define
visual languages, which differ significantly in the way they conceptualise a visual notation and describe
its syntactic structure. From the semantic point of view, the precise meaning of a visual language is required
in order to use it effectively. Finally, ambiguity is one of the main problems of visual languages.
A visual configuration expressing a sentence or a visual action may be interpreted in a number of ways
producing ambiguity. Visual languages offer an intuitive and incremental view of sentences, but may
give different interpretations of the same sentence.
Organisation
The book contains 22 chapters split into five sections. Section I examines the theory underlying visual
languages. Section II discusses different approaches to and methods for visual languages on specific
domains. Section III describes visual languages for the semantic Web. Section IV examines visual
interfaces for standard languages. Finally, Section V considers several topics closely related to visual
languages such as visualisation, aesthetic computing, and usability
Section I. Visual Languages Theory
Section I opens with a chapter by Bottoni, Costabile, Levialdi, and Mussio, introducing the Theory of
Visual Sentences by highlighting their main concepts, presenting some problems central to the definition
of a theory of visual languages, and describing current developments. After an informal overview
of the main components of the theory, the authors explore the characteristics of interactive processes
and the modelling of visual transformations by visual rewriting systems. They go on to examine the
dynamics of visual sentences, focusing on the relationship between constancy and variability in visual
sentences, enabling and disabling mechanisms and coordination of transformations. The chapter ends
with a discussion of how the theory’s components can be integrated to provide a framework for the
design of interactive visual systems before drawing some conclusions.
Chapter II, by Meyer and Bottoni, traces progress in Visual and Diagrammatic Languages: The Logic
Perspective. The chapter starts by outlining the grammatical approach to visual language specification
highlighting its shortcomings as a basis for this research plan. It then revisits the history of logic approaches
to diagrammatic languages and details the comparative advantages and disadvantages of the
different types of logic formalisation. Finally, it develops a new approach based on linear logic, which
avoids most of these shortcomings and completely subsumes the grammatical approach.
Chapter III, by Bottoni, Frediani, Quattrocchi, Rende, Sarajlic, and Ventriglia, proposes a Transformation-
Based Metamodel Approach, which enables an abstract view of the semantic roles that visual
elements can play with respect to the process being described. More specifically, the chapter proposes
an integrated framework and interactive environment based on a collection of metamodels in which it
is possible to express both syntactical characterisations of diagrammatic sentences and their semantic
interpretations.
Chapter IV, by de Lara and Guerra, presents an approach for the definition of Multi-View Visual
Languages (MVVLs). These are made up of a set of different diagram types, which are used to specify
the different aspects of a system. The chapter introduces two techniques to define MVVL environments:
Meta modelling and graph transformation. The former is used to describe the syntax of the language as a
whole. A meta-model for each of the language’s diagram types (viewpoints) is defined as a restriction of
the complete MVVL meta-model. Consistency between views is ensured by translating each one into a
unique repository model, which conforms to the meta-model of the language as a whole. The translation
is performed by automatically generated graph transformation rules.
Chapter V, by Costagliola, Deufemia, and Polese, presents an overview of eXtended positional
grammars (XPG), a grammar formalism for modelling visual notations, which represents an extension
of context-free grammars and describes the XpLR parsing methodology. XPG and XpLR extend positional
grammars (PG) and the associated pLR parsing methodology. These extensions have enabled
the modelling and efficient parsing of a wide class of notations. The associated parsing algorithm is the
XpLR parser based on the well-known LR parsing technique. The benefits of the formalism of a visual
language include easier customisation and modification as well as maintenance and debugging, code
and report generation by definition of suitable semantic productions, and implementation of visual and
textual languages within a common framework.
Chapter VI, by D’Ulizia, Grifoni, and Rafanelli, classifies the Different Kinds of Ambiguities that
can arise in visual sentences, distinguishing between lexical and syntactic ambiguities. When an image
associated with a visual sentence is unable to express exactly (be completely faithful to) the user’s
intentions, the system may produce an erroneous interpretation. Ambiguities are generally produced by
(1) the language, which can produce such one-to-many relationships, (2) imprecision introduced by theinteraction behaviour producing the visual sentence. In the first case, an image can assume more than
one meaning. The second case is connected with incorrect/imprecise information that does not permit
an unequivocal interpretation of the image produced by the drawing actions.
The final chapter in this section, by Caschera, Ferri, and Grifoni, proposes Different Kinds of Solutions
to Ambiguities that can arise in visual sentences. This chapter deals with ambiguities related to the
system’s interpretation function and methods to resolve them. These methods can be grouped in three
classes: prevention of ambiguities, a-posteriori resolution, and approximation resolution methods. Prevention
methods consider only a predefined set of possible system configurations, avoiding ambiguous
configurations. A-posteriori resolution methods are based on mediation techniques, which enable the
user to disambiguate his or her intention by dialogue. In contrast, approximation resolution methods
are based on theories such as Fuzzy Logic, Markov Random Field, and Bayesian Networks and do not
require user disambiguation.
Section II. Approaches and Methods for Specific Domains
Section II opens with the chapter by Caschera, D’Ulizia, and Tininini, which analyses the main characteristics
of VQLs, concentrating on visual languages to Query Conventional Relational Databases
but also examining information systems with a less rigid structure such as Web resources storing XML
documents. It considers two main VQL classifications: the adopted visual representation technique
(e.g., based on forms and tables, diagrams, icons, sketches, or combinations thereof) and the underlying
data model (e.g., visual languages to query relational databases, object-oriented databases, collections
of XML documents, and languages specifically designed for particular data such as geographic and
multidimensional data).
Chapter IX, by Cavalcanti, Schiel, and de Souza Baptista, focuses on the specific category of Visual
Query Languages for Spatio-Temporal Databases, which enable formulation of queries involving both
spatial and temporal dimensions. Current papers treat these dimensions separately with only a few
integrated proposals. This chapter presents a VQS called spatio-temporal visual query environment (STVQE),
which allows the formulation of conventional, spatial, temporal, and spatio-temporal database
queries in an integrated environment. With S-TVQE, the user, instead of querying the database by textual
query languages, interacts with the system by visual operators to state the query conditions.
Chapter X, by Fogli, Marcante, Mussio, Provenza, and Piccinno, considers that the knowledge relevant
to the design of an interactive system is distributed among several stakeholders: domain experts,
software engineers, and human-computer interaction experts, and presents a Multi-facet Design of Interactive
Systems. Each community describes an interactive system through visual sentences of a visual
language (VL). This view results in an approach to VIS design based on the definition and use of three
visual languages. Each VL permits user-system interaction process to be specified from a different point
of view and for a different audience.
The chapter by Deray and Simoff starts from the assumption that although interaction is the core of
interactive computing, its mechanisms remain poorly understood. The tendency has been to examine
interactions in terms of the results they produce rather than to provide mechanisms explaining how
interactions unfold in time. The authors present a framework for creating visual languages to represent
interactions using human movement as a source for the language’s core concepts. The approach is motivated
and supported by evidence from research on kinaesthetic thinking that constructs based on human
movement support higher-level cognitive processes and can be intuitively recognised by humans.
The final chapter of Section II, by Deufemia, considers Sketch Understanding. This is a particularly
difficult task as the symbols of a sketched diagram can be drawn with different stroke orders, numbers, and directions. The recognition process is often made even harder by lack of precision and ambiguities
in messy, hand-drawn sketches. The chapter presents a brief survey of sketch understanding techniques
and tools.
Section III. Visual Languages for the Semantic Web
Chapter XIII, by Catarci, Kimani, and Lodi, examines User-Interface Formalisation in Visual Data
Mining. A formal specification facilitates the description of the system properties without touching on
implementation details, and enables fundamental design issues to be detected before they are manifested
in the implementation. An approach for the formalisation of the visual interface of a core data mining
system is given.
Chapter XIV, by Kontopoulos, Bassiliades, and Antoniou, discusses the Visualisation of Defeasible
Logic Rules in the semantic Web domain. Logic plays an important role in the development of the semantic
Web and defeasible reasoning seems a very suitable tool. However, it is too complex for end users who
often need graphical traces and explanation mechanisms for the derived conclusions. The chapter proposes
an approach that uses directed graphs to assist the user by offering the notion of direction, which appears
highly applicable to the representation of rule attacks and superiorities in defeasible reasoning.
The last chapter in the section, by Vrakas, Hatzi, Bassiliades, Anagnostopoulos, and Vlahavas, is
concerned with Knowledge Representation for AI Planning Problems, especially those related to semantic
Web service composition. It discusses current approaches in encoding planning problems and presents
ViTAPlan, a user-friendly visual tool for planning.
Section IV. Visual Interfaces for Standard Languages
Chapter XVI, by Almendros-Jimenez and Iribarne, shows how to use and specialise UML Diagrams for
Describing the User Interfaces of a software system considering three specialised UML diagrams called
user-interaction, user-interface, and GUI-class diagrams. It also examines code generation to implement
the system’s user interfaces through GUI-class diagrams and user-interaction diagrams. A case study of
an Internet book shopping system is introduced to test and illustrate the proposed user interaction and
interface design technique.
Chapter XVII, by Campi, Martinenghi, and Raffio, describes a visual framework, XQBE that covers
the most important aspects of XML Data Management, spanning the visualisation of XML documents,
query formulation, the representation, and specification of document schemata, the definition of integrity
constraints, the formulation of updates, and the expression of reactive behaviours in response to data
modifications.
Section V. Visualisation Aesthetic Computing, and Usability
Chapter XVIII, by Voudouris and Marsh, discusses how cartography, GIS, and HCI shape aspects of
Geovisualisation can support decision-making. The chapter analyses the relationships among geovisualisation,
human computer interaction (HCI), geographic information systems (GIS), and cartography as
a means of supporting decision-making. It emphasises the importance of data modelling and associated
visualisations in terms of what the user can do by way of analysis and the methods by which he or she
can undertake the analysis.
The following chapter, by Skhiri and Zimanyi, presents a Graph Layout Algorithm for Drawing
Biochemical Networks. Due to the huge amount of information available in biochemical databases, biologists
need sophisticated tools to accurately extract the information from such databases and interpret
it correctly. Those tools must be able to dynamically generate any kind of biochemical subgraph (i.e.,
metabolic pathways, genetic regulation, signal transduction, etc.) in a single graph. The visualisation
tools must be able to cope with such graphs and take account of the particular semantics of all kinds
of biochemical subgraphs. The chapter presents an algorithm designed for the generic representation
of biochemical graphs, in which users can present knowledge about how to draw graphs in accordance
with the biochemical semantics.
Fishwick discusses the Aesthetic Computing Method. The purpose of aesthetic computing is to apply
the theory and practice of art and design to the field of computing. The range of aesthetics within the
arts is broader than in mathematics and computing where aesthetics is often synonymous with optimality
criteria. This chapter introduces aesthetic computing for the multimedia representation of formal
structures.
Chapter 21, by Costa, Grings, and Santos, presents Documentation Methods for Visual Languages.
Visual programming languages (VPL) are generally self-documenting; this chapter analyses their use
for the design and documentation of real world applications. Finally, the chapter proposes new tools and
approaches for the documentation of data flow for VPL.
The book ends with the chapter by Padula and Reggiori, discussing Usability. This does not propose
a specific viewpoint on usability but rather examines numerous concepts, aspects, and potentialities,
which must nowadays be taken into consideration when detailing usability and design suited systems.
This topic is currently of great interest in the field of human-computer interaction as it is highly dependent
on the user interface. Systems must be used by a community of users in their working activity as
a whole to process material or information, modifying not only rough material but also the working
environment and methods. This requires them to be considered as tools in a social context, which expects
ever-greater technological progress.