Personalized Real-Time Virtual Tours In Places With Cultural Interest

Introduction

Virtual tourism (Beck J. & Egger R., 2018) reduces time or spatial limitations of real tourism. Services such as 360^o video streaming (Qian, Ji, Han & Gopalakrishnan, 2016), 3D animation (Bustillo, Alaguero, Miguel, Saiz & Iglesias, 2015), Augmented Reality (AR) (Marques, Tenedório, Burns, Romão et al., 2017) and mixed reality (MR) (Debandi, Iacoviello, Messina et al., 2018) are used to construct a virtual world for the user. Drones (Mirk & Hlavacs, 2015) could be used for the 360° video capture of places with cultural interest. The video can be streamed to users in real time, enriched with additional audio descriptions, 3D, AR or MR content. To support such services, fifth generation (5G) (Akpakwu, Silva, Hancke & Abu-Mahfouz, 2018) mobile infrastructures could be used, providing plenty of networking, computational and storage resources. Indicatively, the enriched 360° video is streamed to the user through a 5G Mobile Edge Computing (MEC) or Fog (Roman, Lopez & Mambo, 2018) infrastructure, which assures the satisfaction of its strict constraints in terms of throughput, delay, jitter and packet loss. The 5G infrastructure could support heterogeneous network access technologies, such as the 3GPP Long Term Evolution Advanced (LTE-A) (TS36.300-V13.2.0., 2016), the IEEE 802.11p Wireless Access for Vehicular Environment Road Side Units (WAVE RSUs) (1609.3-2016, 2016) and the IEEE 802.16 WiMAX (802.16q-2015, 2015).

Virtual tours with drones can be used in numerous cases dealing with protection, preservation and enhancement of tangible heritage, as well as servicing special groups of people, i.e. the elderly, children, persons with disabilities that cannot reach inaccessible monuments. Each drone is remotely controlled by the user or it is autonomous navigated (Kan, Okamoto & Lee, 2018). A critical task of the autonomous navigation service is the selection of the most appropriate flying route for the drone, as in heritage sites where multiple monument types exist, the user preference for each type should be considered, in order personalized user experience to be provided.

User preferences could be modeled using the MPEG-21 (Metta,S., Montagnuolo,M. & Messina,A., 2015) standard, which defines a framework for both multimedia resources and user preferences manipulation. MPEG-21 uses the architectural concept of the Digital Item (DI), which is a combination of resources (such as audiovisual content), metadata (such as descriptors) and structures describing the relationships between resources. DIs are declared using the Digital Item Declaration Language (DIDL). The MPEG-21 Digital Item Adaptation (DIA) architecture and the MPEG-7 (Park, Hong, Lee & Lee, 2014) Multimedia Description Schemes (MDS) for content and service personalization provide a Usage Environment that models user preferences. Additionally, description of the multimedia that enriches the 360^o video is required in order the appropriate content to be retrieved and placed in certain places into the video. This description could be performed using the MPEG-7 standard.

Both MPEG-21 and MPEG-7 (MPEG-21/7) metadata are stored in XML format allowing efficient indexing, searching and filtering. The metadata structure could be described using Web Ontology Language (OWL) (Sengupta & Hitzler, 2014). OWL is a knowledge representation language used for composing ontologies. The ontologies are described in OWL documents by defining classes, attributes and individuals. Classes are collection of concepts, attributes are properties of classes and individuals represent the objects of a class. Thus, the system will be intelligent enough to autonomous retrieve information of the metadata files without requiring additional information about the metadata structure from the user. OWL ontologies could be queried using the SPARQL (Peng, Zou, Özsu, Chen & Zhao, 2016) language. SPARQL features include conjunctive or disjunctive patterns as well as value filters. Subsequently, when the system has learned about the metadata structure, it can query them using Mpeg Query Format (MPQF) (Tous & Delgado, 2010) queries. MPQF specifies the message format for querying the metadata repositories.