Application Integration – A Short Tutorial
Application integration is a broad topic with various aspects and views. Before presenting different views, let us look at the very basic models that drive application integration. They are:
Level at which applications are integrated -- data, application logic (method), or presentation (user interface)
Loose versus tight coupling (synchronous versus asynchronous) between the players
Point to point versus hub/bus type integrations
Application Connectivity Types
Figure 1 shows the key players in integration: information suppliers that are
custodians of the needed information, information consumers that are the clients
of the needed information, and the mediators that enable the consumer-supplier
interactions . As shown in this figure, you can interconnect applications at
data, method, or presentation (user interface) level by using different types of
middleware services. These middleware services comprise the mediators between
information suppliers and information consumers. Examples of middleware
technologies are:
Remote Data Connectors (RDCs)- access of remote data directly, typically through remote SQL and ODBC/JDBC, without invoking any enterprise functionality (e.g., business logic).
Remote Method Connectors (RMCs)- access of a remote method (e.g., a C++ procedure that handle payments) through APIs by using a variety of technologies such as CORBA or RPC packages.
Remote User Connectors (RUCs) - access of remote information through some presentation services by using Web scrapers or screen scrapers.
Naturally, there are tradeoffs in connecting at different
levels. For example, data level interconnectivity means that the supplier has to
expose the database schema.
Figure 1: Application Connectivity Type
Synchronous versus Asynchronous Communications : Loose Versus Tight Coupling
Another aspect of application connectivity is the Interaction model that is used between suppliers and consumers. For example, a supplier can communicate with the consumer in:
Blocked, also known as synchronous communication, mode (i.e., the client must wait for response from the server)
Unblocked, also known as asynchronous communications, mode (i.e., the client can continue processing while the server is busy developing the response)
Synchronous communications
are interactive in nature and assume a tight coupling between the clients and
servers. The foundation of synchronous communication is the client/server (C/S)
model that allows application processes at different sites to interactively
exchange messages. Most C/S applications utilize the remote procedure call (RPC)
paradigm which extends the scope of a local procedure call. A very common
example of the C/S model is the World Wide Web where the browsers are the
clients that access remote Web servers in real time. Thus the synchronous
communication model is used by the Web browsers and servers -- when you click on
a link, you are blocked and must wait for a response from the server. This
implies that the Web browsers are tightly coupled to the Web servers through
HTTP. The synchronous communication model is also used extensively in the
distributed objects technologies such as CORBA, DCOM, RMI, and SOAP (e.g., SOAP
RPC).
Synchronous communications offer maximum synchronization capabilities between
partners. These are the types of technologies that application vendors such as
SAP use to integrate components of their application suites. The drawback to
this approach is that it hampers the autonomy of collaborating organizations and
typically requires combinations of joint development, shared client/server logic
and a central architecture. These approaches work best for centrally architected
solutions but are inappropriate for large scale enterprise-wide and B2B
solutions.
In asynchronous communications, the clients are not blocked when a message is
sent to the server—the clients can put a message on the queue and then continue
processing. Another segment of client logic can continue to monitor the output
queue for any responses and process the outputs received. Asynchronous
processing has the following benefits:
Failure of server does not stop the clients from continuing to process as much as possible.
Clients can be more efficient and responsive because they do not have to wait for the server to respond.
It is suitable for detachable (e.g., wireless) computers that need to access data asynchronously without having to be in constant session with the hosts.
Clients and servers can use “deferred message” or call back features if a response is not available.
Clients and servers can operate in a publish-subscribe model where events are posted by the publisher on a channel. The subscribers listen to events of their choice and act accordingly.
Which one is better? In most
enterprise level integrations, it is better to use an asynchronous model because
it implies loose coupling. The suppliers and consumers do not have to be blocked
from further processing. This model also supports a publish-subscribe model that
is very suitable for large scale integrations. This is the main reason why most
integration platforms are based on asynchronous processing. The best known
example of asynchronous middleware is MOM (message oriented middleware).
The synchronous/asynchronous communications can be used at the data, method or
presentation levels discussed in the previous section. For example, an order
processing applicant can access the data of an inventory management application
by using synchronous or asynchronous communications.
Point-to-Point Versus Hub/Bus Models
Integrations for a number of years were done on a point to point basis (see
Figure 2).In these cases, the suppliers and consumers directly interfaced with
each other through mediation technologies. As shown in the following figure, the
point to point communication may work well for small number of partners. For
example, you can use point to point communication when integrating two or three
systems. However, this approach does not scale well because you need N(N-1)/2
interfaces between N partners.
Hubs provide a better approach for integrating different applications. Hub-based
application integration is useful because:
It cuts down on the number of interfaces that need to be supported between the applications. Thus instead of point to point interconnections, all applications connect to the hub that organizes and mediates between applications.
New applications can be added to the hub easily and without disrupting other applications. This is especially useful in trading networks where different partners can appear and disappear on a regular basis.
Bus-based integration is a variant of hub-based integration because they can be used to combine the point-to-point with hub-based architectures. The parties can communicate directly with each other over the bus or can use a control engine (a “process engine”) that can provide basis for flow of information between different players. For example, A can directly communicate with B because they are part of the same organization but go through a control engine when talking to C, an external entity. Process engines are typically workflow managers.
Combining the Models -- The Enterprise Service Bus
From the above discussion, it is obvious that asynchronous, hub/bus-oriented
systems that support integration at the data, method as well as presentation
level are needed for large scale enterprise integration. This is what the
Enterprise Service Bus (ESB) provides.
An Enterprise Service Bus (ESB) is a key component of SOA and provides the main
mechanism for integrating the enterprise applications. A SOA ESB provides a
collection of technologies (middleware such as Web Services, adapters/gateways
for protocol conversion, data transformers, transaction managers, and
work/process flow systems) that allow diverse applications to communicate with
each other. Figure 3 shows a conceptual view of an ESB. The ESB platform is not
a new technology – rather, it is a combination of “well-known” technologies that
can integrate multiple applications. All applications (business components)
provide services that are invoked through well defined interfaces. In reality,
ESB is an extension of the well known EAI (Enterprise Application Integration)
platforms that were based on the older CORBA and DCOM technologies.
Figure 3: Enterprise Service Bus (ESB) – Conceptual View
An ESB is
expected to possess the following common set of characteristics:
Communication through a Broker
Although ESB
platforms may use different types of middleware technologies (e.g., Message
Queuing, etc), Web Services are the most popular technologies of ESBs at the
time of this writing. ESBs may also exist as EAI (enterprise application
integration) platforms or message brokers. ESB software is commercially
available from vendors such as IBM (Websphere ESB), Microsoft (Biztalk Server
2010) and others (e.g., Mule, Sonic Software, Systinet, Tibco, Fiorano, IONA).
An ESB basically provides “adapters” that allow different applications to
connect at data, method or presentation level over the bus.
These platforms commonly use the publish/subscribe model for asynchronous
(unblocked) communication for loosely connecting a diverse array of applications
over a bus. In a publish/subscribe model the messages can be
“published” into a “channel”. This is similar to the notion of a television news
channel. Thus a channel publisher simply posts (“pushes”) events of interest to
the channel. The publisher is unaware of the actual recipients (subscribers).
The subscribers retrieve (“pull”) the information from the channel. This
provides a useful mechanism for decoupling applications from one another. A
trading system, for instance, can simply publish a new item to be traded on a
“new items” channel and various traders can respond to this event.
Publish/subscribe is currently available through MOM, JMS, and Web Services. The
ESB in effect is the channel. An ESB can be used to interconnect applications
for internal as well as for B2B trade.
Source: Umar, A., “Enterprise Application Integration Using SOA”, NGE Solutions,
2nd edition, 2013.