Interactive Multimedia Music Technologies

Interactive Multimedia Music Technologies

Kia Ng (University of Leeds, UK) and Paolo Nesi (University of Leeds, UK)
Indexed In: SCOPUS View 1 More Indices
Release Date: October, 2007|Copyright: © 2008 |Pages: 410
ISBN13: 9781599041506|ISBN10: 1599041502|EISBN13: 9781599041520|DOI: 10.4018/978-1-59904-150-6

Description

Many multimedia music content owners and distributors are converting their archives of music scores from paper into digital images, and to machine readable symbolic notation in order to survive in the business world. Interactive Multimedia Music Technologies discusses relevant state-of-the-art technologies and consists of analysis, knowledge, and application scenarios as surveyed, analyzed, and evaluated by industry professionals.

Interactive Multimedia Music Technologies exemplifies the newest functionalities of multimedia interactive music to be used for valorizing cultural heritage, content and archives that are not currently distributed due to lack of safety, suitable coding models, and conversion technologies. Interactive Multimedia Music Technologies explains new and innovative methods of promoting music and products for entertainment, distance teaching, valorizing archives, and commercial and non-commercial purposes, and provides new services for those connected via personal computers, mobile and other devices, for both sighted and print-impaired consumers.

Topics Covered

The many academic areas covered in this publication include, but are not limited to:

  • Architecture and algorithms of optical music recognition
  • Audiovisual content protection
  • Design Strategies
  • Digital Rights Management
  • Information technology in music
  • Interactive Multimedia
  • Interactive systems for multimedia opera
  • Markup language design
  • MPEG symbolic music representation
  • Online music distribution
  • Optical music imaging
  • Sound Synthesis
  • Technology support for learning and culture
  • Web-based music intelligent tutoring systems
  • XML music notation modeling

Reviews and Testimonials

This book aims to provide the latest insights and future directions integrating relevant experiences, information, and knowledge in these domains to help bring the music content industries, information technology companies, and research communities closer, and to bring music into the interactive multimedia era.

– Kia NG, University of Leeds, UK

Ng and Nesi bring together 14 chapters that describe interactive multimedia technologies used in enhancing and providing musical content more effectively. The book is aimed at academics, researchers, practitioners, and students in technology, media, and music fields, in addition to musicians, educators, and music and audiovisual technologies.

– Books News Inc . (2008)

This work is a compendium of articles designed to give a state-of-the-art description of music technology at several distinct levels. Summing Up: Recommended. General readers; researchers and faculty; professionals.

– CHOICE, Vol. 45, No. 09 (May 2008)

Table of Contents and List of Contributors

Search this Book:
Reset

Preface

Currently, there is a wide range of interactive multimedia technologies that can help evolve the music market to the next level, to enhance and to provide musical content in a more effective, more readily accessible, faster and more easy-to-use manner.

Content owners, producers and distributors such as publishers, archives, libraries, conservatories, music shops, music information technology industries and educators recognise the usefulness and potential of these developments. There are many opportunities to exploit these novel technologies and a great deal of interest from the relevant parties. However, there are concerns about losing the rights and ownership of content. This obstacle is further increased by the widening gap and the lack of effective knowledge transfer between the industrial and research communities.

Many multimedia music content owners and distributors are converting their archives of music scores from paper into digital formats such as images and machine-readable symbolic notation, in order to excel in the market. Typically problems arise due to the requirements of archives which have been organised differently, where several other related digital objects are collected (e.g., images, documents, lyrics, videos, etc.). Issues such as standards and formats (machine representations), integrated cross-media datasets, digital rights management and tools are important considerations and invaluable knowledge. The new functionalities of multimedia interactive music can be used for:

valorising cultural heritage, content and archives that are not currently distributed due to digital rights issues, suitable coding models, and conversion technologies;

promoting music and products for entertainment, for distance teaching, for archives, for commercial and non-commercial purposes; and

providing new services for consumers connected via personal computers, mobiles and other devices, and widening accessibility for impaired users.

This book aims to provide the latest insights and future directions integrating relevant experiences, information and knowledge in these domains to help bring the music content industries, information technology companies and research communities closer and to bring music into the interactive multimedia era. With a primary focus on the activities and findings of the MUSICNETWORK (see www.interactivemusicnetwork.org) co-supported by the European Commission, this book consists of analyses, knowledge and application scenarios, as surveyed, analysed and tested, including music representations and the developments in the standardization of machine-readable Symbolic Music Notation Representations of MPEG ISO; Music Imaging (e.g. Optical Music Recognition); with informative discussions on efficient mechanisms for distributing and sharing multimedia music content (such as e-commerce, mobile applications, etc.), rights control and management, towards more secure and new formats to exploit new functionalities with interactive multimedia technologies. These subject areas are useful to professionals and researchers working in the field of interactive multimedia-related technologies, computer music, music technology, publishers, librarians, e-business, digital media, digital rights, music representations, cultural, learning and teaching and many other interdisciplinary domains, including students, researchers, businesses and SMEs (Small and Medium Enterprises).

To provide the background context and motivations, this book starts with an introductory chapter, “Interactive Multimedia MUSICNETWORK: An Introduction”, to present the MUSICNETWORK that was co-supported by the European Community under the fifth framework programme. The aim of the MUSICNETWORK is to help bringing the music content providers, cultural institutions, industry, and research institutions to work together, to reduce the barriers between the technology- and content-providers and to improve the exploitation of new interactive multimedia technologies. It brings together research institutions, industries, SMEs, and experts to build the required momentum in order to study and define multimedia music modelling and coding for the digital era and finally to open new markets and possibilities with new technologies and solutions.

With a large number of participants, the MUSICNETWORK addressed several aspects of music within a set of Working-Groups including: •

  • Music Notation: examines all aspect of coding music notation, such as modern music notation, format conversion, fonts, and defining standards for music symbols, and started the work on MPEG Symbolic Music Representation (MPEG SMR). •
  • Music Libraries: This group has a cross-domain perspective including museums, archives, industry catalogues and other collections. •
  • Music Multimedia Standards: studies and analyses of multimedia standards for music coding, including audio and video coding (e.g. MPEG7, MPEG21, etc), portable internet formats, media integration and other standardization aspects, with MPEG and others. •
  • Music Distribution: examines the distribution of coded music including streaming, Internet, distribution models (B2B, B2C, P2P, etc.), mobile systems, WEB-TV, and transaction models (on-line, off-line, kiosks, etc). •
  • Music Protection: focuses on issues related to the protection of coded music, such as encryption, fingerprint, watermark, Digital Rights Management, profiling functionalities, active and passive protection, and other security issues. •
  • Music Accessibility: examines music coding for print impaired people (visually impaired, dyslexic, etc), and studies accessibility issues, user interfaces, computer assisted software and devices and the provision of music in alternative formats. •
  • Music Imaging: concentrates on issues relating to imaging and processing of sheet music, printed music scores and handwritten manuscripts, including music image acquisition, acquisition of music with different types of page support, digitising ancient music, coding for images, optical restoration and preservation, and Optical Music Recognition (OMR, also known as optical character recognition for music) and evaluation of OMR systems. •
  • Music Audio: This working group is focused on audio processing aspects such as: conversion from audio to music notation, query by content, beat tracking, audio shrinking and stretching, audio recognition, etc. •
  • Music Education: This group analyses and works on educational aspects of music with the support of the information technology and pedagogical aspects. In particular it deals with the aspects of: cooperative work on music notation, performances, playing instruments by using internet support, e-learning, distance teaching, courseware tools, assessing music performances, self learning, software tools for music education, etc. •
  • Music Culture: This working group addresses the cultural aspects of music and musicology. It considers historical interpretation, context, etc.
  • The chapters of this book present a selection of the most relevant activities in the above mentioned sectors, highlights key achievements and analyses main results. Most of the results produced by the project have a strong international value and involved many participants globally.

    As one of the main results of the MUSICNETWORK, chapter 2 on “MPEG Symbolic Music Representation: a solution for multimedia music applications” by Pierfrancesco Bellini, Paolo Nesi and Giorgio Zoia presents an overview of the development of the MPEG SMR (Symbolic Music Representation) standard which was started by the MUSICNETWORK and has recently been accepted by the MPEG ISO global multimedia standard organization. The chapter discusses the MPEG SMR design and development including its decoder for the MPEG-4 standard, for interactive TV, PC and mobiles. A large part of the work performed in this area has been coordinated by the MUSICNETWORK. The authors believe that the MPEG SMR development represents a crucial step that will bring the next phase of multimedia music development into all forms of electronic devices that will further support interoperability and widen accessibility to music enjoyment, research, culture, as well as education.

    Music Notation and representation is one of the fundamental aspects of the new knowledge presented in this book. Chapter “XML Music Notation Modelling for Multimedia: MPEG-SMR” by Pierfrancesco Bellini outlines the structure of the forthcoming MPEG SMR ISO standard, and proposes the formalization in terms of XML for music representation and provides several examples. This chapter discusses the continuous growth of the MPEG SMR (XML and Binary formats) which includes modelling of music representation, multilingual lyrics, integration of music representation and multimedia, formalization of a language for the automatic formatting of music representation, music representation streaming, etc.

    In order to take advantage of new interactive multimedia capabilities and possibilities such as those discussed in this book, paper-based music scores have to be transformed into machine-readable representations. This book devotes two chapters to Music Imaging issues and presents two OMR systems.

    The Chapter “Optical Music Imaging: Music Document Digitisation, Recognition, Evaluation and Restoration” by Graham Jones, Bee Ong, Ivan Bruno and Kia Ng, introduces the background of music imaging related issues including digitisation, processing, restoration and automatic transcription of digitised music score to recognise musical features (from the image of the digitised score) and convert them into machine-readable formats (e.g. MPEG SMR). With a general overview, the chapter discusses challenges in this domain, digitisation issues and necessary components including hardware and software. It also presents the design and development of SharpEye which is one of the most popular commercial OMR systems. After that, the chapter presents an approach for the evaluation of OMR systems and evaluates a number of systems (SharpEye, SmartScore and O3MR) using this approach. This chapter also presents automated recognition and graphical restoration issues that are important for the preservation of a vast amount of invaluable paper-based heritage, including printed music scores and handwritten manuscripts that are deteriorating over time due to natural decaying of paper and chemical reaction (e.g. printing ink and paper).

    Following the theme of OMR, chapter “Optical Music Recognition: Architecture and Algorithms” by Pierfrancesco Bellini, Ivan Bruno and Paolo Nesi present the design and development of the O3MR system for optical music recognition. The O3MR solution has been realised with a large set of methods and technologies to analyse and process images of music scores. Experimental results reported demonstrate a high efficiency in the correct location of basic symbols. With a neural network based classifier, a rule-based music grammar and iterative aggregation process, the system supports reconstruction of notation of varying complexity, which is represented with the WEDELMUSIC XML model.

    In chapter "Challenges of Designing a Markup Language for Music", Jacques Steyn analyses the background development of music representations using markup languages. Steyn makes comparisons with other representations (e.g. text and mathematics markup languages) and discusses considerations such as paged or scrolled rendering, writing systems and reusability and scalability issues. This chapter highlights the particular challenges of markup languages for music, such as multiple and simultaneous events that need to be synchronised.

    This is followed by a chapter on "Alternative design goals for a general music markup language" by the same author. In this chapter, Steyn notes that "Design goals determine the particular structure of a markup language, while the philosophy of what markup languages are about determine the framework within which its structure is developed". In this context, this chapter presents brief surveys of related representations including SGML, DSSSL, HyTime and SMDL, together with HTML and XML. With several XML-based music applications examples, the author proposed an ideal structure of XML for music, and discusses design criteria and application issues.

    The chapter on “Interactive Systems for Multimedia Opera” by Michael Oliva, presents current trends towards technology-enhanced performing arts, exploring interactive multimedia technologies for stage performance and how they increase artistic possibilities. It describes the background, the design and the development of several software applications and hardware systems to deliver multimedia content for operas, and discusses gestural control interfaces. It analyses requirements, presents a recent performance and recommends effective and practical approaches to these issues. The author believes that the requirements and integrations of these new elements should be performed centrally from the beginning and not simply applied “atmospherically or decoratively”.

    In chapter "Driving Sound Synthesis with a Live Audio Signal", Cornelius Poepel discusses issues of musical expression, challenges of real-time computer-based systems. The chapter introduces an audio synthesis method called "Audio Signal Driven Sound Synthesis" that uses the raw audio signal of a traditional instrument to drive a synthesis algorithm. This method can be used for the augmentation to various existing musical instruments as well as new instruments.

    The author discusses an application of the method with a bodiless viola that allows the player to explore new sonic dimensions of electronic sounds with an interface (the playing of the instrument) that is already familiar to the player. Since the synthesis engine is driven by the audio signal of the bodiless instrument, nuances of articulation and playing technique can be represented in the synthesized sound even if not tracked as it is necessary in common parameter driven approaches. The chapter also reports user tests and comments from expert players who have tested the system and discusses plausible future directions.

    Chapter “How Technology can Support Culture and Learning” by David Fuschi, Bee Ong and David Crombie discusses the interleaving interrelations of education, technology, content accessibility and intercultural issues. It presents a wide range of related background for contextualisation and uses language learning as an example for the discussions. The authors suggest that technology is both a tool and a factor which can foster cultural and learning development and presents the evolution of educational models and developments with respect to technological innovations. The chapter also considers accessibility issues, together with tools and approaches to e-learning, considerations on the market sectors, standards, regulation and related issues.

    The chapter on “Web-Based Music Intelligent Tutoring Systems” by Somnuk Phon-Amnuaisuk and Chee Keh Siong, presents a web-based intelligent music tutoring system for music theory. It considers the pedagogical aspects of music learning and teaching and discusses the pros and cons of web-based systems for learning and teaching. The chapter discusses the key challenges including the lack of domain specific content authoring tools, intelligent feedback, and presents the design and development of an interactive multimedia environment for music score editing. In order to provide further understand of the performance of individual students and to analyse students’ behaviours and performances, monitoring functionalities and modelling of learning (using a Bayesian network) are explored. With the framework, technical details and results analysis, the chapter also discusses several potential future trends, including multi-modal interface, standardisation requirements of teaching materials for music, and intelligent systems for music theory tutoring.

    DRM systems can be viewed as multimedia information management systems that take into account digital rights and protection functionalities. They enable controlled distribution and usages of the multimedia content through the digital value chain. In chapter "Digital Rights Management Technologies and Standards", Jaime Delgado and Eva Rodríguez discuss the background, state-of-the-art developments and requirements related to Digital Rights Management (DRM), with particular focus on the technological advancements and standardisation activities.

    The chapter discusses a number of selected initiatives (standard and proprietary), that specify a DRM system or the elements that form a DRM system, including MPEG-21, Open Mobile Alliance (OMA) DRM, TV-Anytime DRM, DReamM, etc. It also presents current industry solutions such as Windows Media DRM 10 and Helix DRM and discusses their components and architecture. It further describes the MPEG-21 standard initiative, with particular focus on the different pieces and formats that normatively specify and are needed by a complete DRM system, such as the Digital Item Declaration, Rights Expression Language, Intellectual Property Management and Protection and Event Reporting. The chapter presents the Rights Expression Languages (RELs) in details, as one of the key components of DRM system that define the content usage rules, and addresses interoperability issues between RELs.

    Following the theme on DRM, chapter "Possibilities, Limitations, and the Future of Audiovisual Content Protection" by Martin Schmucker briefly discusses DRM and presents several key developments of technologies for the protection of digital content including the so called passive protection technologies, particularly on watermarking and fingerprinting (perceptual hashing). These passive protection technologies are less interfering with content usage and enable new protection applications. For each method, the general principle, characteristics, requirements, and limitations are described.

    The chapter further considers different application contexts and discusses issues from related parties such as the content owner and consumer and discusses examples of conflicts between security requirements and consumer issues. It analyses current technological possibilities and practical limitations, studies the music industry value chain and the influences of DRM. The final section discusses potential future trends and solutions.

    The Chapter “On-Line Music Distribution” by Francesco Spadoni, presents the development of on-line music distributions and related issues including its challenges, business models, market structure, new technologies, and distribution media, together with DRM issues including rights clearing, Intellectual Property protection, content retrieval and metadata management. It also discusses the legal framework and the developing market of digital music and the quality and accessibility aspects of online music distribution services. In the final section, this chapter summarizes the findings and projects towards plausible future trends.

    This book consists of a collection of selected chapters that encompassing information, experiences, and knowledge to bring the music industries and research communities closer and to bring music into the interactive multimedia era. With a wide range of background research and introductory materials that are supported by relevant references for further reading, this book aims to show the latest possibilities and to ponder potential future directions. Real-life application scenarios are used to explain the concepts and provide further understanding on how to apply the new knowledge provided. This book covers key elements of interactive multimedia music; from data acquisition to processing, representations, preservation, protection, management to digital distributions, together with applications and market considerations. In its different chapters, this book provides comprehensive coverage of the interdisciplinary subject area of interactive multimedia music.

    Author(s)/Editor(s) Biography

    Kia Ng obtained his PhD in computer science from the University of Leeds, where he is director and cofounder of the Interdisciplinary Centre for Scientific Research in Music (ICSRiM), and senior lecturer in computing and music. Ng’s research links together work in the School of Computing and the School of Music on computer vision, computer music, and AI. Currently, he is the president of the International Association of Interactive Multimedia MUSICNETWORK. Ng is involved in several domains and initiatives relating to 2-D and 3-D imaging including document imaging (printed and handwritten music manuscripts, paper watermark, etc.), gestural interfaces, and interactive multimedia systems, in collaboration with many European and international organisations and individuals in the field. His music via motion (MvM) system, which provides interactive gestural control of musical sound, has been widely featured in the media, including the BBC and Sky TV. Ng has served as general chair and programme committees for many national and international conferences including WEDELMUSIC, AXMEDIS, AISB, and so forth. He is a fellow of the Royal Society of Arts, a chartered engineer, and a chartered scientist. For more information, visit http://www.kcng.org
    Paolo Nesiis a full professor at the University of Florence, department of systems and informatics, and head of the distributed systems and Internet technology research group. Nesi received a PhD in electronic and informatics engineering from the University of Padoa. His research interests include object-oriented technology, real-time systems, quality, system assessment, testing, formal languages, physical models, computer music, and parallel and distributed architectures. He has been the general chair of IEEE ICSM, IEEE ICECCS, WEDELMUSIC international conferences, and program chair of several others. He has been the coordinator of several R&D multipartner international R&D projects of the European Commission such as MOODS, WEDELMUSIC, AXMEDIS, VARIAZIONI, and MUSICNETWORK (The Interactive Music Network) and involved in many other projects. Currently, he is co-editor of the MPEG SMR ISO standard. (nesi@dsi.unifi.it)

    Indices