Enterprise Architecture and Integration: Methods, Implementation and Technologies

Enterprise Architecture and Integration: Methods, Implementation and Technologies

Wing Lam (U21 Global, Singapore) and Venky Shankararaman (Singapore Management University, Singapore)
Indexed In: SCOPUS View 1 More Indices
Release Date: May, 2007|Copyright: © 2007 |Pages: 364|DOI: 10.4018/978-1-59140-887-1
ISBN13: 9781591408871|ISBN10: 1591408873|EISBN13: 9781591408895|ISBN13 Softcover: 9781616927486
Hardcover:
Available
$132.00
List Price: $165.00
20% Discount:-$33.00
TOTAL SAVINGS: $33.00
E-Book:
Available
$132.00
List Price: $165.00
20% Discount:-$33.00
TOTAL SAVINGS: $33.00
Hardcover +
E-Book:
Available
$156.00
List Price: $195.00
20% Discount:-$39.00
TOTAL SAVINGS: $39.00
OnDemand:
(Individual Chapters)
Available
$37.50
No Current Special Offers
TOTAL SAVINGS: $37.50

Description

Enterprise integration is a broad activity that involves solving a range of issues relating to business process definition, common data standards, architectural compatibility, technical interoperability, and organizational alignment.

Enterprise Architecture and Integration: Methods, Implementation, and Technologies provides a detailed analysis of the important strategies for integrating IT systems into fields such as e-business and customer-relationship management. This Premier Reference Source supplies readers with a comprehensive survey of existing enterprise architecture and integration approaches, and presents case studies that illustrate best practices. It takes a holistic view of enterprise integration, describing innovative methods, tools, and architectures with which organizations can systematically achieve enterprise integration.

Topics Covered

The many academic areas covered in this publication include, but are not limited to:

  • Application Integration
  • Business process integration
  • Business Process Modeling
  • Development of integrative business applications
  • Dissolving organizational and technological silos
  • Enterprise application integration
  • Enterprise integration and process improvement
  • Implementing SOA
  • Information systems integration
  • Mobile working environments
  • Performance Indicators
  • Process integration through hierarchical decomposition
  • Realizing the promise of RFID
  • Social matters
  • Standards-based integration platform
  • Success Factors
  • Supply Chain Integration
  • Visual environment
  • Web services hybrid dynamic composition models

Table of Contents and List of Contributors

Search this Book:
Reset

Preface

It is clear that we now live in a world where one expects everything to be connected, or for want of a better word, integrated. For example, when I get onto the Web and do my Internet banking, I expect to be integrated directly with my bank to see exactly how much money I have. Similarly, when I want to book air tickets online, I expect to be integrated to the airline reservation system to see what flights are available, and whether they are fully booked or not. Businesses too are demanding this kind of close integration. A supermarket, for example, that wants to match demand and supply needs to integrate its own stock control system with that of its suppliers so that goods being taken off the shelf are quickly replenished.

Increasingly then, there is a need for organizations to provide services in an integrated fashion. But this, however, is easier said than done. Integration can not be achieved unless the Information Technology (IT) used by the organization is also integrated. Unfortunately, organizations have had a long history of creating systems which operate in isolation from other systems. From conception, many of these systems would have been designed to address the specific requirements in a particular functional area such as accounting, personnel and stock control. This functional orientation, however, has tended to reinforce departmental silos within the organisation, resulting in an IT architecture characterized by numerous ‘islands of applications’ that remain quite separate from each other (Sawhney 2001).

Of course, integration is not an entirely new problem. Many organizations would have undertaken specific projects in the past where they would have had to integrate a number of different systems. The traditional approach to integration, often referred to as ‘point-to-point’ integration (Linthicum 2001), has tended to involve crafting custom interfaces between two systems, e.g. System A and System B. However, when System A needs to share information with System C, another set of interfaces needs to be created between System A and System C. Similarly, when System B needs to communicate with System C, then another set of interfaces is created. In such an approach to integration, a new set of interfaces is created for each pair of systems that need to communicate. Unfortunately, such a piecemeal approach does not scale well when tens, even hundreds, of individual systems need to be integrated as is often the case in large organisations. Organisations that have attempted to use the point-to-point approach for large-scale integration have typically ended up with is tangled web of integration interfaces in which the high cost of maintaining such a large number of interfaces have become a burden on IT budgets.

In short, the challenge faced by organisations today is to find scalable and economical solutions to the problem of large-scale integration (Sharif et al. 2004). This has given rise to the term ‘enterprise integration’, which denotes the need to integrate large number of systems that may be highly distributed across different parts of the organisation.

Past solutions

This is not to say that solutions to the challenge of large-scale integration have not been proposed in the past. In the early 90’s, distributed objects and component architectures were mooted as the answer to the challenge of integration (Digre 1998). This was typified by the Common Object Request Broker Architecture (CORBA), which provided an open standard for distributed systems to communicate (Vinoski 1997). However, CORBA projects were perceived as technically very complex, requiring significant development effort (Henning 2006). Furthermore, industry support and standardization efforts for CORBA proved problematic, leading to its slow demise. More recently, Enterprise Resource Planning (ERP) solutions, such as SAP and Peoplesoft, which involve replacing existing systems with a suite of interconnected modular systems from a single vendor, was seen as the answer to systems integration (Davenport 1998). However, organisations soon realized that no single ERP system could address all the requirements of an organisation (Hong and Kim 2002). In fact, an ERP system often needed to co-exist with existing systems, so heightening, rather than removing, the need for integration (Themistocleous et al. 2001).

However, it is not just the scale of integration which is challenging. The integration of IT systems is also severely hampered by the technical complexities of integration (Lam 2004). For one, a particular system may have been developed on a technology platform which is, at worse, incompatible with the technology platforms used by other systems. In the 70’s and 80’s for example, many systems were developed on what is now considered legacy mainframe technology, while the 90’s saw the adoption and proliferation of Internet and Web-based technologies. Therefore, most organizations have, over time, ended up with a portfolio of systems based on a diverse mix of technologies. It must also be mentioned that many legacy systems were originally conceived to serve as standalone systems with little intent for integration. Such systems, which represent a huge financial investment to the organization, tend to become operationally embroiled within the organization which makes them difficult to replace with newer, more modern systems (Robertson 1997).

Promising Developments

Recently, however, there have been some promising developments within the IT industry that, going forward, offer the potential for easing the technical challenge of integrating systems in the future. These developments centre on the adoption of Web Services (Cerami 2002) as a standard for open communication over the Internet. In short, Web Services enables systems to communicate to other systems over the Internet or, as the case may be, the Intranet. For this to happen, systems must expose, as services, the functionality they wish to make available to other systems. A stock system, for example, may expose a ‘check current stock’ service to other systems so that they can check, in real-time, the current stock levels of a particular product. So you can imagine how this might help buyers and suppliers co-ordinate the supply chain much more effectively. But there are several, not significant, barriers to the adoption of Web Services. One of these barriers is the immaturity of the standards (Bloomberg and Schmelze 2002). The standards relating to Web Services are relatively new, and continue to evolve at a rapid pace. For obvious reasons, organizations are often loathed to make substantial technology investments relating to standards that may be quickly superseded. Other barriers include concerns over performance and reliability. Because Web Services relies on the Web, performance and reliability are, and can not be, guaranteed. Nor are Web Services generally suitable for high-volume transaction processing. For these reasons alone, Web Services may not be a suitable technical solution to systems integration in mission-critical business processing. In addition, while it might be useful to design new systems with Web Services in mind, existing systems may need to be substantially re-engineered in order to conform to a Web Services model. So while Web Services is certainly a promising technical solution to systems integration, it is by no means a complete one.

Another interesting development within the IT industry is that of the Service Oriented Architecture (SOA). SOA (He 2003) is something that has piggy-backed off the Web Services bandwagon but, in the last 3 years or so, has gained a momentum of its own. In a SOA view of the world, systems expose their functionality as a set of services, typically as a set of Web services. It does not matter which technology platform the system sits on, or what development language the system has been written in, as the services are defined in a way that other systems can readily ‘understand’. In fact, other systems don’t need to worry, nor necessarily care, about how these services are actually implemented. These services can either by public or published in nature. If services are public, they are typically known to a limited set of other systems, such as in the case within a corporate Intranet. If services are published, however, details of the services are registered in a directory from which other systems, including those external to the organization, can discover and use the services. In essence, in a SOA, there is network of loosely coupled systems, where a complex business function may be implemented by calling upon the services offered by other systems.

With SOA, the issue of integration is no longer problematic, so long as every system conforms to the SOA model and is able to offer their functionality through a set of defined services. But that, unfortunately, is the point where SOA suffers the same adoption problems as Web Services with which it is closely aligned. If organizations could start from a clean sheet again, they would probably develop their systems to conform to a SOA model. In reality, however, organizations need to manage, and live with, at least in the immediate and near future, the huge investments they have already made in existing, legacy systems. So SOA and Web Services are not, although some would like to believe, the ‘silver bullets’ for that will resolve all IT woes. They clearly offer a pathway, or at a least a direction, for resolving many integration issues, but is not a solution that can be readily implemented by organizations today.

Enterprise Application Integration

Fortunately, what might serve as a more complete and holistic solution to enterprise integration are the Enterprise Application Integration (EAI) tools being marketed by integration vendors such as Webmethods, TIBCO, IBM, Microsoft, Seebeyond, BEA, Mercator and Vitria (Mckeen and Smith 2002). Such EAI tools did not appear overnight, but evolved from the message-oriented middleware (MOM) tools that became popular as a means of providing high volume, reliable communications between systems. In general, EAI tools have three main components. The first is an integration broker which serves as a hub for inter-system communication. The integration broker performs a number of functions such as multi-format translation, transaction management, monitoring and auditing. The second is a set of adapter which enables different systems to interface with the integration broker. An adapter is essentially a gateway or wrapper that provides the means by which packaged applications (such as SAP), database applications (such as Oracle), legacy systems (such as mainframe) and custom applications (written in Java or other programming language) can connect to the integration broker (Brodie and Stonebraker 1995). The first component is an underlying communications infrastructure, such as a reliable high-speed network, which enables systems to communicate with each other using a variety of different protocols.

Although EAI has, until now, occupied a rather niche market, the growing importance of enterprise integration can only mean that the size of the EAI market will expand. One can also observe a growing sophistication and maturity in EAI tools. One area, for example, is that of business process management (or BPM). The reason for this is because the motivation behind many integration projects is to support business processes that span across different parts of an organisation. An e-business transaction, for example, may begin at the order entry system, but such transactional information may be passed onto an account management, payment, logistics and then eventually a delivery order system as part of broader business process flow. Hence, the integration of these systems is driven by business process needs. Some EAI tools are therefore beginning to include BPM tools that enable the modeling of business processes in a graphical format. These business process models are subsequently linked to calls or operations that initiate processing within a system. As it turns out, few organizations have their business processes so rigorously defined and mapped out. The introduction of BPM tools therefore provides a timely and pertinent reason for organizations to revisit their business processes

Another related area of growing sophistication in EAI tools is that of business activity monitoring (BAM). BAM is about monitoring the health of activities within a business process. If, for example, a business process fails for some reason, this will be picked up by the BAM tool and an appropriate alert can be raised. BAM can also be used to identify bottlenecks within a process by tracking throughput and assessing the rate at which the different activities within a business process are successfully completed. Clearly, BAM tools, and for that matter, BPM tools, are particularly well-suited to organisations that have business processes that are, or inclined to be, heavily automated.

So EAI tools are themselves evolving, and what one is beginning to see is a closer alignment between technical integration, which relates to how systems talk to each other, and business integration, which relates to why systems need to talk to each other. At the same time, business integration is being facilitated by the increasing standardization within the business integration space and convergence by EAI vendors in working towards these standards. Central to this effort is the Business Process Modelling Language (BPML 1.0) standard, developed under the auspices of the Business Process Management Initiative (BPMI.org), which provides a formal model for describing any executable end-to-end business process. In theory, if all organisations described their business processes in BPML, they would find it much easier to collaborate.

Strategy

Aside from technology and process issues, the other important element of enterprise integration is the strategic element. Spending thousands of dollars on EAI tools and teams of individuals modeling business processes does not necessarily mean that organizations will solve their enterprise integration problems. One missing piece from the all this, like any other major endeavor, is the importance of strategic direction and senior management support (Lam 2005). Enterprise integration is something that affects many different parts of the organization; the problem is not confined to one particular part of the organization. As such, success can only be achieved if the various departments ‘buy-in’ to enterprise integration and share the same vision of how to achieve it. This, of course, is easier said than done and there are several challenges here. One of the challenges is the fact that, in a large organization, individual departments may be used to operating autonomously, with minimal interaction and engagement. The notion of working with other departments, or at least co-coordinating their technology strategies, is something that may appear quite novel. At worst, the endeavor becomes a political battle, where certain divisions are seen to be vying for control, encroaching on the space occupied by others. This, of course, is not something which is peculiar to enterprise integration projects, but any large project that involves different divisions within an organisation working in new ways.

Importantly, each department must believe that there is a case for enterprise integration, either financially in terms of reducing the costs of systems integration, or from a business perspective in terms of enabling new business processes or enhancing business performance. Unfortunately, individual departments, by their nature, have a localized view of their integration needs. Getting the bigger picture of an organization’s enterprise integration needs, however, is a message that must be communicate to individual departments so that they understand the rationale for a strategic perspective. Of course, if one left each department to its own devices, to develop its own solution to its own set of integration problems, there would be much ‘reinvention of the wheel’ and wasted time and effort. A single strategic integration solution therefore makes much more sense, where an organization can address the integration requirements in different parts of the organization in a cost-effective and controlled manner. It certainly does not make sense for each department to purchase their own EAI tool!

Another thing to bear in mind is that enterprise integration is not something which can take place overnight. Enterprise integration is a long-term initiative that, in some cases, may take years to complete depending upon the number of systems to be integrated and the complexity of the integration challenge. Organisations therefore need to think carefully about how to plan and rollout the enterprise integration initiative. One approach would be to identify the high priority integration projects within the organization where the urgency or potential business returns from integration, is greater. Such successful projects could then serve to reinforce the case for integration and perhaps even provide inspiration for further integration projects. Another, more risk adverse approach would be to identify pilot projects which could serve as the grounds for organizations to build up expertise and knowledge of enterprise integration before tackling larger and more substantial projects. Such a more cautious approach might suit organizations with little prior experience with enterprise integration. It might also be wise to consider integration projects in parallel with other business improvement projects which, in turn, can help shape the integration project. A good example is business process re-engineering, where it does not make any sense to automate a process that is intrinsically inefficient or problematic, but where an opportunity presents itself to make broader organizational changes. In fact, from an organisational perspective, information systems integration involves changes in business process, and more broadly, realignment of technology goals with business goals (Themistocleous et al. 2001).

To sum up, enterprise integration has become a strategic enabler for many of the business initiatives organizations are, or wish to embark on, whether it is supply chain management, customer relationship management, e-business or simply more efficient ways of business processing. The traditional methods of systems integration have not proved to be scalable, so solutions are needed that can address both the scale and complexity of integration. Enterprise integration, however, is not just about technology integration, it is also process and business integration, and so may involve a re-conceptualization of how organizations work and do business.

Organisation of the book

This book is organized into four main sections, each of which has a number of chapters. The first section is entitled ‘Managing Enterprise Integration’, and contains the following chapters:

    Chapter 1 examines the evolution of enterprise integration and its growing importance amongst organizations. The chapter provides a good overview of the benefits of enterprise integration and some of the challenges associated with its adoption.

    Chapter 2 looks at five critical success factors for enterprise application integration, namely, minimal project expense, optimal reuse of components, complexity reduction, optimal coupling of applications, and minimal cost of EAI infrastructure. The authors conclude that the success factors are closely integrated, and develop from this a number of hypotheses.

    Chapter 3 explores how social and organisational aspects can affect ERP projects and their ability to integrate different parts of the enterprise. The paper draws from a successful case of a multinational ERP implementation in a large international organisation, and concludes with the recommendation that one should examine the roles of all actors with the power to affect not only the implementation project but also the system being implemented

    Chapter 4 discusses experiences in the acquisition of software intensive systems prior to establishing a contract. One significant challenge is the identification of functional requirements, and the authors contend that business process modeling is an effective way to explicitly define requirements while creating visibility and consensus among different stakeholders of major IT projects.

The second section is entitled ‘Business Process Management’, and contains the following chapters:

    Chapter 5 presents an approach to constructing enterprise processes based on the integration of component-processes. In this approach, process representations at the logical as well as physical levels are depicted in a hierarchy, and a graphical tool is used to support enterprise process construction.

    Chapter 6 explores, through a case-study, the use of XML and Web Services in conjunction with an integration broker to provide a B2B solution. The authors argue that existing B2B solutions, largely based around EDI, present a high entry barrier to smaller organizations, and that the use of XML and Web Services provide an alternative lower-cost B2B solution that may be more suitable.

    Chapter 7 provides an overview of business process management. The chapter describes how systems development has changed from being largely implementation-oriented to now being analysis-oriented, and suggests that business process management standards and technologies will drive the transition to more service-oriented IT architectures in the future.

    Chapter 8 describes the importance of meta-models, particularly in relation to knowledge-intensive service industries. The key guiding principle is to define the process model at the logical level, free from any technical implementation. Business process integration is then a matter of achieving the best possible overall physical ‘engine’ to implement that process model – from available legacy applications, applied investment opportunity and expert development resources.

The third section is entitled ‘Integration Methods and Tools’, and contains the following chapters:

    Chapter 9 presents a methodology for the creation of Enterprise Application Integration solutions based on the concepts of Software Product Line Engineering. The overall idea is to view the external applications, with which a given system wants to integrate as a family of systems. In this way the flexibility required by EAI applications can be assured.

    Chapter 10 demonstrates a visual tool for automatic supply chain integration. By clicking in the appropriate tables, functions and fields, users can specify the data needed for integration. Integration code can then be automatically generated using Web Services.

    Chapter 11 presents an integration framework called DAVINCI aimed at providing mobile workers with mobile software applications to query and update information coming from different and heterogeneous databases. The chapter describes the trials developed using the DAVINCI architecture in different European countries.

    Chapter 12 describes the basic notions of intelligent agents and multi-agent systems, and proposes possible types of their application to enterprise integration. The agent-based approaches to enterprise application integration are considered from three points of view: i) it means using an “agent” as a wrapper of application or service execution; ii) it means constructing a multi-agent organization within which agents are interacting and providing emergent solutions to enterprise problems; iii) it means using the agent as an intelligent handler of heterogeneous data resources in an open environment.

    Chapter 13 describes an attempt to introduce semantics to workflow-based composition. A composition framework is presented based on a hybrid solution that merges the benefit of practicality of use and adoption popularity of workflow-based composition, with the advantage of using semantic description to aid both service developers and composers in the composition process and facilitate the dynamic integration of Web services into it.

The fourth and final section is entitled ‘Enterprise Integration Case-Studies’, and contains the following chapters

    Chapter 14 examines common EI challenges and outlines approaches for combining EI and process improvement based on the process improvement experiences of banks in several different countries. Common EI-related process improvement challenges are poor usability within the user desktop environment; a lack of network based services; and data collection and management limitations. How EI affects each of these areas is addressed, highlighting specific examples of how these issues present themselves in system environments.

    Chapter 15 explores the gradual evolution of SOA through various phases and highlights some of the approaches and best practices that have evolved out of real world implementations in regional retail banks. Starting from a step by step approach to embrace SOA, the chapter details some typical challenges that creep up as the usage of SOA platform becomes more and more mature. Also certain tips and techniques that will help the instructions to maximize the benefits of enterprise wide SOA are also discussed.

    Chapter 16 describes a case-study of application integration in a Korean bank. The case examines the integration technology employed, and the adoption of the technology on a pilot project. The chapter highlights some of the managerial implications of application integration and discusses the broader, organizational impact of application integration.

    Chapter17 uses the example of a retail business information system to illustrate the concept of service-oriented development and integration in a number of different scenarios. The paper shows how using a service-oriented architecture allows businesses to build on top of legacy applications or construct new applications in order to take advantage of the power of web services.

    Chapter 18 discusses the adoption and potential usages of RFID (Radio Frequency Identification) in the case of enterprise integration projects such as supply chain management. Through electronic tagging, RFID can enable stocks to be monitored to a level that was previously not practical or cost-effective.

    Author(s)/Editor(s) Biography

    Wing Lam is Associate Professor at Universitas 21 Global, a leading online graduate school, where he is also programme director for the Master in Management in Information Technology. In addition to academic positions in universities in Singapore and the UK, Dr Lam has held consultancy positions with Logica-CMG, ICL (now Fujitsu) and Accenture. His current research interests include enterprise integration, knowledge management and software engineering management. Dr Lam has over 80 publications in peer-reviewed journals and conference proceedings and currently serves on the editorial boards of several journals.
    Venky Shankararaman is a Practice Associate Professor at the School of Information Systems, Singapore Management University, Singapore. His current areas of specialization include enterprise architecture, enterprise integration, service oriented architecture and business process management. He has over 14 years of experience in the IT industry in various capacities as a researcher, academic faculty member and industry consultant. Venky has designed and delivered professional courses for governments and industries in areas such as enterprise architecture, technical architecture, enterprise integration and business process management. Venky also worked as a faculty member at several universities in the UK and Singapore, where he was actively involved in teaching and research in the areas of intelligent systems and distributed systems. He has published over 50 papers in academic journals and conferences.

    Indices