Design of an Integrated Web Services Brokering System

Introduction

Web Services are becoming the technology used to share data in many domains. Web Services technologies provide access to discoverable, self-describing services that conform to common standards. Thus, this paradigm holds the promise of an automated capability to obtain and integrate data. While desirable, access and retrieval of data from heterogeneous sources in a distributed system such as the Internet pose many difficulties and require efficient means of discovery, mediation and transformation of requests and responses. Differences in schema and terminology prevent simple querying and retrieval of data. These functions require processes that enable identification of appropriate services, selection of a service provider of requested data, transformation of requests/responses, and invocation of the service interface. Service availability must also be resolved. There have been a variety of approaches developed for these functions, but primarily independently of each other and not fully automated, that is, often requiring human intervention.

In this paper we describe the design of an integrated end-to-end brokering system that performs automated discovery, automated mediation and automated transformation of Web Services requests and responses. In contrast to more static approaches that deal with pre-selected data servers, our approach creates a dynamic knowledge base from Web Service interface specifications that are discovered on the fly. The dynamic knowledge base assists with mediating requests to data providers that have ad-hoc interfaces or differing versions of a community accepted interface.

Our design incorporates ontologies into the development of an Integrated Web Services Broker (IWB). This approach contrasts with developments that assume that shared ontologies have been adopted or published in order to support service discovery and integration. In addition to the use of ontologies we have evaluated classifier technology for the subtask of Web Services discovery. It has been noted that classifiers generalize well in sparse data, which is a characteristic of our Web Services application domain. Our use of classifiers in this manner does not require a formal domain definition nor does it require data providers to deploy any additional specialized ontological descriptions of their Web service.

There are general characteristics that should be found in any environment in which an automated system will operate. First, the domain must be one in which human intervention is neither required nor desirable. Since we are considering a Web Services context, data providers must have adopted a text-based, structured Web Services interface. This interface should subscribe to Web Services standards of self-description. While the structural content of each Web Services interface may differ significantly, the domain should be one in which key terminology that may be found in any interface has common conceptual content and is well understood and bounded. In this operating environment, it is desirable to isolate potential data sources in advance as opposed to attempting to discover service availability and capability on demand.

These characteristics are broadly applicable and encompass many typical application areas, and in this paper we describe the design and development of the IWB relative to these characteristics. The steps and issues we will be describing are basically applicable to any Web Services brokering system. Here, we illustrate the design specifically for the application context in which we are developing this system, that is, meteorological and oceanographic (MetOc) forms of data.