1. Introduction to the Enterprise Service Bus – Enterprise Service Bus [Book]

Across all industries, executives are demanding more value from
their strategic business processes. What process is
strategic to a given company may vary dramatically by
industry, but a common theme is that CEOs want their IT
organizations to measurably improve the flow of data
and information driving key business decisions. Whether
it’s a financial services firm seeking a competitive
advantage by guaranteeing a higher volume of faster
foreign exchange trades, a retail chain looking to
accelerate the flow of store data back to brand
managers at corporate headquarters, or a building
materials supplier striving to optimize order flow
through a complex distribution chain, there are common
and significant technical challenges to be overcome.
Information is locked up in applications within
different departments and organizations, and it is both
time-consuming and costly to pry that data loose. In
short, the enterprise is far from integrated.

The past several years have seen some
significant technology trends, such as Service Oriented
Architecture (SOA), Enterprise Application
Integration (EAI), Business-to-Business (B2B),

and web services.
These technologies have attempted to address the
challenges of improving the results and increasing the
value of integrated business processes, and have
garnered the widespread attention of IT leaders,
vendors, and industry analysts. The Enterprise Service
Bus (ESB) draws the best
traits from these and other technology trends.

The ESB concept is a new approach to integration that can
provide the underpinnings for a loosely coupled, highly
distributed integration network that can scale beyond
the limits of a hub-and-spoke EAI broker. An ESB is a
standards-based integration platform that combines
messaging, web services, data transformation, and
intelligent routing to reliably connect and coordinate
the interaction of significant numbers of diverse
applications across extended
enterprises

with
transactional integrity.

An extended enterprise represents an
organization and its business partners, which are
separated by both business boundaries and physical
boundaries. In an extended enterprise, even the
applications that are under the control of a single
corporation may be separated by geographic dispersion,
corporate firewalls, and interdepartmental security
policies.

With its distributed deployment infrastructure, an ESB can efficiently provide central configuration, deployment, and management of services that are distributed across the extended enterprise.

An ESB provides the implementation backbone for an SOA. That is, it provides a loosely coupled, event-driven SOA with a highly distributed universe of named routing destinations across a multiprotocol message bus. Applications (and integration components) in the ESB are abstractly decoupled from each other, and connect together through the bus as logical endpoints that are exposed as event-driven services.

Service components in an SOA expose coarse-grained interfaces with the purpose of sharing data asynchronously between applications. Using an ESB, an integration architect pulls together applications and discrete integration components to create assemblies of services to form composite business processes, which in turn automate business functions in a real-time enterprise.

In the realm of SOA, events are represented in an open XML format and flow through a transparent pipeline that’s open to inspection and subject to intermediation…

In an event-driven enterprise, business events that affect the normal course of a business process can occur in any order and at any time. Applications that exchange data in automated business processes need to communicate with each other using an event-driven SOA to have the agility to react to changing business requirements. An SOA provides a business analyst or integration architect with a broad abstract view of applications and integration components to be dealt with as high-level services. In an ESB, applications and event-driven services are tied together in an SOA in a loosely coupled fashion, which allows them to operate independently from one another while still providing value to a broader business function.

An ESB provides a highly distributed approach to integration, with unique capabilities that allow individual departments or business units to build out their integration projects in incremental, digestible chunks. Using an ESB, departments and business units can continue to maintain their own local control and autonomy in individual integration projects, while still being able to connect each integration project into a larger, more global integration network or grid.

The ESB architecture addresses these needs, and is capable of being adopted for any general-purpose integration project. It is also capable of scaling pervasively across enterprise applications, regardless of physical location and technology platform. Any application can plug into an ESB network using a number of connectivity options, and immediately participate in data sharing with other applications that are exposed across the bus as shared services. This is why the ESB is often referred to as an integration network or integration fabric.

In an ESB, applications and event-driven services are tied together in an SOA in a loosely coupled fashion. This allows them to operate independently from one another while still providing value to a broader business function.

It needs the flexibility and ability to react to and meet the needs of changing business requirements and competitive pressures.

It must provide an SOA across the pervasive integration that enables integration architects to have a broad, abstract view of corporate application assets and automated business processes.

It must have simplicity of design and low barriers to entry, enabling the everyday IT professional to become a self-empowered integration architect.

It must extend beyond the boundaries of a single corporate IT data center and into automating partner relationships such as B2B and supply-chain scenarios.

It must adapt to suit the needs of general-purpose integration projects across a variety of integration situations, large and small. Adaptability includes providing a durable architecture that is capable of withstanding evolutions in protocols, interface technology, and even process modeling trends.

The common goal of applying technologies such as SOA, EAI, B2B, and web services is to create an architecture for integration that can be pervasive across an extended enterprise and beyond. For an integration infrastructure to achieve this pervasiveness, it must have the following characteristics:

The core tenets of SOA are vital to the success of a pervasive integration project, and are already implemented quite thoroughly in the ESB. The web services standards are trending in the right direction, but remain incomplete with respect to the enterprise-grade capabilities such as security, reliability, transaction management, and business process orchestration. The ESB is based on today’s established standards in these areas, and has real implementations that are already being deployed across a number of industries. The ESB is quite capable of keeping in step with the ongoing evolution of the web-services equivalents of these capabilities as they mature. Chapter 12 provides a more detailed discussion on this subject.

With the advent of the ESB there is now a way to incorporate web services and SOA into a meaningful architecture for integrating applications and services into a backbone that spans the extended enterprise in a large-scale fashion. An ESB makes web services, XML, and other integration technologies immediately useful with the mature technology that exists today.

Web services have bestowed newfound importance on service-oriented architectures by providing a standards-based approach to interoperability between applications. The main objective of web services has been to provide a service abstraction that allows interoperability between applications built using disparate platforms and environments. The achievement of this goal will provide an easier path to pervasive integration between applications.

Chapter 6 further describes the contrast between integration using an application server architecture and integration using an ESB. MOM concepts are discussed in Chapter 5 . “The Accidental Architecture” in Chapter 2 continues to discuss the separation of business process routing and business logic.

Finally, in an ESB, services can be configured rather than coded. Process flow and service invocations can transparently span the entire distributed bus. An ESB provides a highly distributed integration environment that spans well beyond the reach of hub-and-spoke architectures, and a clear separation of business logic and integration logic such as routing and data transformation. An ESB architecture forms an interconnected grid of messaging hubs and integration services, with the intelligence and functionality of the integration network distributed throughout.

Message Oriented Middleware (MOM) provides the ability to connect applications in a loosely coupled, asynchronous fashion. However, MOM by itself requires low-level coding in an application. Using a traditional MOM, along with custom coding techniques, can get you a long way toward a distributed integration solution. However, without a higher level of abstraction of the routing logic, this approach also suffers from having integration logic hard-wired and intertwined with the application logic. Depending on the MOM being used, even the distributed characteristic might be limited because some traditional MOM infrastructure is not capable of spanning physical network boundaries very well.

EAI brokers provide increased value by separating the application logic from the integration and process routing logic, yet still suffer from the hub-and-spoke architecture.

Application servers can interoperate through standard protocols, yet they link things together in a tightly coupled fashion, and intertwine the integration logic and application logic together.

Traditional EAI brokers, which include those that are built upon application servers, use a hub-and-spoke architecture. A hub-and-spoke architecture has the benefit of centralized functions, such as management of routing logic and business rules, but does not scale well across departmental or business unit boundaries. Chapter 2 will examine the huge price of early attempts at integration using EAI hubs, as well as their moderate success.

Traditional formalized approaches to integration have their pros and cons. Chapter 1 shows some of the high-level traits of integration approaches, which range from the least desirable on the lower left point of origin, to the most desirable on the upper right quadrant.

An ESB applies web services and other complementary standards by combining them with technology concepts and best practices learned from EAI brokers. However, an ESB is more than simply a web-services veneer on top of the same old EAI hub.

IT professionals had been disappointed by some previous technology trends such as CORBA and EAI. CORBA had the right idea with SOA, but turned out to be too complex to implement and maintain due to its reliance on tightly coupled interfaces between applications and services. EAI also suffered from steep learning curves and expensive barriers to entry on individual projects (more on this in the next chapter). What was really needed was a simple approach to SOA, with an architecture that could be adapted to suit the general needs of any integration effort, large or small. In addition, there needed to be a durable architecture that was capable of withstanding evolutions in protocols, interface technology, and even process modeling trends. The ESB concept was created to address all these needs.

At the same time, the biggest needs that had yet to be addressed included how to effectively provide integration capabilities such as application adapters, data transformation, and intelligent routing in a way that could be used for general-purpose integration projects across a variety of integration situations. Also required was a more universal technology and an architectural approach that could be used to connect applications beyond the needs of individual tactical integration projects.

A key characteristic of an ESB is to provide the underpinnings to support the needs of distributed, loosely coupled business units and business partners automating supply chains. These capabilities in an ESB were born out of necessity, as a result of middleware vendors working with industry professionals who were trying to create an architecture for large-scale integration. These industry professionals included IT architects in large corporations, and innovators in the e-Marketplace trading hub community who needed to build a B2B trading exchange backbone based on shared services, messaging, XML, and numerous connectivity options, while adhering to industry standards for each component. Chapter 3 will discuss the many catalysts that contributed to the creation of the ESB concept.

If nothing else, the completion of Indigo will make applications and services built on the Microsoft platform even more attractive as endpoints to connect into an ESB. The inclusion of Indigo into the Microsoft platform and development environment will facilitate making applications capable of being loosely coupled and messaging-aware.

Indigo is based on messaging and has the notion of combining MSMQ and web services. That could provide the basis for a messaging bus. However, the rest of its integration capabilities are locked into BizTalk, which is a hub-and-spoke integration server. To qualify as an ESB, both a distributed message bus and distributed integration capabilities need to exist.

Roy Schulte, an analyst at Gartner Inc. in Stamford, Conn., noted that Indigo is a superset of Microsoft’s Messaging Queuing (MS MQ) technology, as well as the company’s Component Object Model (COM), COM+, .Net remoting and Web services support. “Think of this as a simplification, a unification of communication middleware on behalf of Microsoft’s plan,” he said, adding that he sees Indigo as a very good enterprise service bus (ESB).

At the time of this writing, Microsoft has not publicly made any statements regarding its Indigo project and the term ESB. However, some journalists and analysts made that connection when Indigo was first announced. A November 30, 2003 ComputerWorld article, “Developer interest piqued by Microsoft technologies,” quoted Roy Schulte of Gartner Inc. regarding Indigo:

A few vendors in this list deserve special mention. Sonic Software pioneered the concept, and shortly thereafter a number of other smaller vendors got on board, saying they also were providing an ESB or were in the process of building one. Once the incumbent integration companies such as webMethods, SeeBeyond, and IBM finally got “on the Bus” and announced their intent to build an ESB, the ESB term really began to get widespread notice as a growing technology category with staying power.

A number of middleware and integration vendors have either built, or are in the process of building, something that matches the description of the ESB. And the list is growing all the time. The Appendix lists all the known vendors. Some vendors say they are providing an ESB already; some have announced plans to create one; some are just using the terminology in marketing materials but don’t really have anything substantial behind it. This technology category is destined to become as hot as application servers were in the late ’90s, when more than 25 vendors were competing for the same technology space.

Those four pillars—MOM, web services, transformation, and routing intelligence—represent the foundation of any good ESB. This book will focus on the role of each piece and many other required components as we explore the ESB. We will examine what the ESB can do for an enterprise, and the role that each basic component plays. We will also explore some advanced topics, including architectural overviews of practical uses across a number of industries.

Analyst firms such as Gartner Inc., IDC, and ZapThink have been tracking and writing about the ESB technology trend since early 2002. In a report issued in 2002 (DF-18-7304), Gartner Inc. analyst Roy Schulte wrote the following:

Since ESB was first introduced in 2002, the ESB approach to integration has been adopted by numerous significant vendors in the middleware, integration, and web-services markets. Its acceptance continues to grow steadily.

Characteristics of an ESB

Due to the fast flurry of vendors trying to gain
attention in the growing ESB product category,
combined with the number of industry analysts and
journalists reporting with their opinions on it,
understandably there is much confusion as to what
an ESB actually is. This section serves to outline
the main characteristics of an ESB.

Pervasiveness

As illustrated in Chapter 1, the ESB can
form the core of a pervasive grid. It is capable
of spanning an extended enterprise and beyond,
having a global reach across departmental
organizations, business units, and trading
partners. An ESB is also well suited for localized
integration projects, and provides flexible
underpinnings enabling it to adapt to any type of
integration environment.

ESB forms a pervasive grid that can span a global enterprise network

Figure 1-2. ESB forms a pervasive grid that can span a
global enterprise network

Applications plug into the bus as needed,
and are capable of having visibility and of
sharing data with any other applications or
services that are plugged into the bus. While
web-services interfaces are an integral part of an
ESB architecture, all applications do not have to
be modified to become true web services to
participate in the ESB. Connectivity is achieved
through multiple protocols, client API
technologies, legacy messaging environments, and
third-party application adapters.

Standards-Based Integration

Standards-based integration is a fundamental
concept of an ESB. For connectivity, an ESB can
utilize
J2EE components such
as the Java Message Service (JMS) for MOM connectivity,
and J2EE Connector Architecture (JCA or J2CA) for
connecting to
application adapters. An ESB also integrates
nicely with applications built with .NET, COM, C#,
and C/C++. In addition, an ESB can easily
integrate with anything that supports SOAP and
web-services
APIs, which includes de facto standard
web-services toolkit implementations such as
Apache Axis. For dealing with data

manipulation,
an ESB can use XML standards such as XSLT, XPath,
and XQuery to provide data transformation,
intelligent routing, and querying of “inflight”
data as it flows through the bus. For dealing with
SOA and business process routing, an ESB can use
the Web
Services Description Language (WSDL) to describe
abstract service interfaces, and Business Process
Execution Language for Web Services (BPEL4WS),

WS-Choreography,
or some other XML-based vocabulary such as ebXML
BPSS, to describe abstract business
processes.

Don’t worry if you don’t know what all these
buzzwords mean. Though this book is not intended
to be an exhaustive reference or tutorial on all
these individual technologies, they will be
explained in sufficient detail in the context of
how they relate to an ESB.

These standards-based interfaces and
components are put together in a meaningful way
that comprises an open-ended pluggable
architecture. An ESB provides an infrastructure
that supports both industry-standard integration
components as well as proprietary elements through
the use of standardized interfaces. Chapter 1 shows a
simplified view of an ESB that is integrating a
J2EE application using JMS and JCA, a third-party
packaged application using a JCA-compliant
application adapter, a .NET application using a C#
client, and two external applications using web
services.

An ESB integrating a variety of disparate technologies

Figure 1-3. An ESB integrating a variety of disparate
technologies

Highly Distributed Integration and
Selective Deployment

The ESB draws from
traditional
EAI broker functionality in that it provides
integration services such as business process
orchestration and routing of data, data
transformation, and adapters to applications.
However, integration brokers are usually highly
centralized and monolithic in nature. The ESB
provides these integration capabilities as
individual services that can work together in a
highly distributed fashion, and can be scaled
independently from one another. In Chapter 6 you will
learn more about the ESB “service container,” a
core concept of the ESB, which allows the
selective deployment of integration
services.

Distributed Data Transformation

A key part of any integration
strategy is the ability to readily convert data
formats between applications. Many applications do
not share the same format for describing similar
data.

Data transformation is inherently part of
the bus in an ESB deployment. Transformation
services that are specialized for the needs of
individual applications plugged into the bus can
be located anywhere and accessible from anywhere
on the bus. Because the data transformation is
such an integral part of the ESB itself, an ESB
can be thought of as solving the
impedance mismatch
between
applications.

Extensibility Through Layered
Services

An ESB gives you all the
required
core capabilities for virtually any integration
project, and can be augmented with layered
technology to handle more specific uses. For
example, specialized capabilities such as Business
Process Management (BPM) software can
process workflow-related business processes, and
collaboration servers can provide specialized
services for managing business partners.
Specialized third-party translators can provide
data conversion from external data formats such as
EDI into a target Enterprise Resource Planning
(ERP) system or onto the general bus as an
internal canonical XML representation.

Event-Driven SOA

In an ESB-enabled, event-driven SOA,

applications and services
are treated as abstract service endpoints, which
can readily respond to asynchronous events. The
SOA provides an abstraction away from the details
of the underlying connectivity and plumbing. The
implementations of the services do not need to
understand protocols. Services do not need to
understand how messages are routed to other
services. They simply receive a message from the
ESB as an event, and process that message. The ESB
gets the message to anywhere else it needs to
go.

In an ESB SOA, custom integration services
may be created, extended, and reused as ESB
functionality. Application endpoints, which are
exposed as services, can be constructed together
with specialized integration enablers to form
composite services and process flows that can be
recombined and reused for various purposes, with
the goal of automating business functions in a
real-time enterprise.

Chapter 7 will
discuss SOA in the ESB in more detail.

Process Flow

An ESB’s process flow capabilities
range from simple
sequences of finite steps to sophisticated
business process orchestration with parallel
process execution paths using conditional splits
and joins. These can be controlled by simple
message metadata or through the use of an
orchestration language such as BPEL4WS.

The process flow capabilities of the ESB
make it possible to define business processes that
are local to an individual department or business
unit, and that can also coexist within a larger
integration network. This is something a
hub-and-spoke integration broker or a BPM tool
can’t do very well on its own. Chapter 7 will examine
the details of a distributed processing capability
that provides highly distributed business process
orchestration without the need for a centralized
processing or rules engine.

Process flow in an ESB can also involve
specialized integration services that perform
intelligent routing of messages based on
content.

Because the process flow is built on top of
the distributed SOA, it is also capable of
spanning highly distributed deployment topologies
(even spanning oceans at times) without the need
to be painfully aware of the physical network
boundaries or multiple protocol hops between
applications and services on the bus (Chapter 1).

Orchestration and process flow spanning highly distributed deployment topologies across physical and logical boundaries

Figure 1-4. Orchestration and process flow spanning
highly distributed deployment topologies across
physical and logical boundaries

Security and Reliability

The connections between nodes
on the
ESB are firewall-capable. The security between
applications and the ESB, and even between the ESB
nodes themselves, is capable of establishing and
maintaining the most stringent authentication,
credential-management, and access control.

Reliability is achieved by having an
enterprise-capable MOM at the
core of the ESB. The
MOM core provides asynchronous communications,
reliable delivery of business data, and
transactional integrity. As you will learn in
Chapter 5, this
is not your traditional MOM technology from a
decade ago. Requirements have evolved and matured
since then, and a MOM core of an ESB must be
capable of meeting today’s requirements.

Autonomous but Federated
Environment

Traditional hub-and-spoke EAI broker
approaches tend to have organizational boundary
problems, which are sometimes caused by the
physical limitations of the EAI broker’s
incapability of easily spanning firewalls and
network domains. More importantly, even if a
hub-and-spoke architecture is capable of being
stretched out across organizational boundaries, it
still does not allow the local autonomy that
individual business units need to operate
semi-independently of one another. One of the
biggest problems associated with extending the
reach of integration beyond the departmental level
is the issue of local autonomy versus centralized
control.

As part of the business culture in most
large corporate environments, each department or
business unit needs to operate independently of
one another. However, they still rely on shared
resources, and reporting and accounting
information that funnels into a common business
function.

In such an environment, it is not reasonable
to impose an integration strategy that requires
all message traffic to flow through a centralized
message broker sitting in headquarters. This is
not simply a technical obstacle; it is a corporate
culture issue as well. In an environment of
loosely coupled business units, it does not make
sense for things such as business process flow
between localized applications, or security
domains, to be managed by a single centralized
corporate IT function. Loosely coupled business
units within an organization need to operate
independently of one other. Each should have its
own IT function and not have to think in terms of
routing all its message traffic, or delegating
control of its business rules and security
domains, through a centralized integration broker
at one location or the other (Chapter 1).

Separate business units lack the necessary autonomy if using a centralized hub-and-spoke integration broker

Figure 1-5. Separate business units lack the necessary
autonomy if using a centralized hub-and-spoke
integration broker

Local business units and departments need to
have control over their own local IT resources,
such as the applications running at their site.
The integration infrastructure should support
deployment topologies to support that business
model with practicality. The ESB provides this
deployment model, allowing for local message
traffic, integration components, and adapters to
be locally installed, configured, secured, and
managed, while still being able to plug together
the local integration domains into a larger
federated integration network with an integrated
security model (Chapter
1).

Autonomous and federated, an ESB allows organizations to cooperatively federate operations across organizational boundaries

Figure 1-6. Autonomous and federated, an ESB allows
organizations to cooperatively federate operations
across organizational boundaries

The distributed characteristics of the ESB
are achieved by abstraction of endpoint
definitions from the physical deployment details
and underlying wire protocols, along with
orchestration and routing of data between those
endpoints. The federated characteristics are
achieved by the ability of the ESB to segregate
and selectively traverse application domains and
security boundaries.

Remote Configuration and Management

In some business models it

doesn’t make
sense to have local IT staff at each remote
location, although there is still a need for a
loosely coupled, autonomous yet federated
integration network. To illustrate this point,
let’s examine a simple case study for an ESB
deployment in the retail industry. A video rental
chain can have hundreds or thousands of remote
locations that all contain the same set of
applications, and all operate in the same fashion
with regard to inventory management, accounting,
and reporting (Chapter
1).

A video retail chain with thousands of remote stores, all containing the same set of applications

Figure 1-7. A video retail chain with thousands of
remote stores, all containing the same set of
applications

Using an ESB, an integration blueprint can
be established to handle the local communication
between the applications at a remote store. This
includes the interface definitions to in-store
applications, the routing of message traffic and
message channel management, and security
permissions. It may also include integration
components such as an application adapter,
protocol adapter, or data transformation. This
integration blueprint, or template, can be
deployed and customized at each site and act
independently of all the other stores (Chapter 1).

ESB configuration blueprint deployed at each remote location and remotely configured and managed

Figure 1-8. ESB configuration blueprint deployed at
each remote location and remotely configured and
managed

This remote deployment blueprint capability
is not unique to retail—it has applications across
other industries as well. The unified remote
management of federated integration realms is a
key contributor to the success of any ESB
deployment in a highly distributed
environment.

Secure, Reliable Messaging Links

In addition to sharing data between
applications locally at each store, these remote
stores need to share information with headquarters
to do accounting and reporting, credit management,
and tracking of employee data. The remote stores
may also need to share information with each
other. For example, a large video chain may want
to offer a service whereby a consumer can rent a
video from a store close to home, and return it to
another store near the office. Therefore, stores
in the same geographic area need to be able to
share information about the rental in a near
real-time fashion. Because of the latency and
resiliency issues of satellite networks between
the remote stores and headquarters, it’s not
feasible to just maintain a constant centralized
access point for all rental information. That
transient information about what you just rented
two hours ago needs to be shared, or be accessible
with an integrated data-sharing link between the
remote stores.

Because the link between the headquarters
and the remote stores is achieved using reliable
messaging, the interrupts in network service due
to unreliable satellite links are compensated for
by the messaging layer. Also note that it is
possible for the stores to link among themselves
using a secure and reliable messaging channel
across an Internet connection.

XML as the “Native” Datatype of the
ESB

XML is an ideal foundation for representing data
as it flows between applications across the ESB.
The data that is produced and consumed by a vast
array of applications can exist in a variety of
formats and packaging schemes. While it is
certainly possible for the ESB to carry data using
any form of packaging or enveloping scheme that
you like, there are tremendous benefits to
representing in-flight data as XML, including the
ability to use specialized ESB services that
combine data from different sources to create new
views of data, and to enrich and retarget messages
for advanced data sharing between applications.
Chapter 4 will
explore a fundamental benefit of XML—the ability
to insulate an organization from the need to
synchronize upgrades between applications—and will
discuss the philosophy behind distributed XML
transformation services in more detail.

Real-Time Throughput of Business
Data

An ESB eliminates latency problems

by providing
real-time throughput into in-flight data as it
travels between applications across the ESB.
Currently, one of the most popular integration
methods is nightly batch processing. However, bulk
batch-processing integration strategies, nightly
or otherwise, are prone to high margins for error
and can cause delays in information retrieval. The
resulting high latency in getting up-to-date
information can be costly. Chapter 9 discusses the
details of this, and explores how an ESB can be
used to refactor your organization from nightly
batch processing toward real-time throughput of
business data.

Operational Awareness

Operational awareness
refers to the ability of a
business analyst to gain insight into the state
and health of business operations. An
infrastructure that allows the timely tracking and
reporting of data as it flows across an
organization in the form of business messages in a
business process is an invaluable tool in helping
to achieve operational awareness. A separate
category of products known as Business
Activity Monitoring (BAM) has emerged to address
the many issues of operational awareness.

One of the benefits of using XML as the
native data format for the ESB is that messages
are not treated as opaque chunks of data. If all
data between applications and services is
formatted as XML documents, underpinnings are
provided by the ESB that allow you to layer
advanced capabilities on top of the ESB to gain
real-time insight into the business data that
flows through your enterprise. These capabilities,
whether they are inherently part of the ESB or are
enabled as an extension of it, represent a natural
part of the common infrastructure that includes
the routing, process flow, and underlying
plumbing, and that don’t require a separate
third-party BAM product be bolted onto the
side.

Auditing and tracking capabilities that are
a base part of an ESB allow you to monitor and
track the health of your business processes and
the flow of messages throughout your SOA.
Value-added services such as data caching, data
collection and aggregation, and visual rendition
of XML data can create a substrate for generating
operational alerts, notifications, and reporting
that can provide real-time insight into the state
of your business as the data traverses your
enterprise (Chapter
1).

Value-added services enabling operational awareness provide real-time insight into live business data

Figure 1-9. Value-added services enabling operational
awareness provide real-time insight into live
business data

Tracking and reporting of data in the ESB is
made possible by defining auditing and tracking
points within a process flow, which in turn
provide insertion points for collecting important
content from business messages as they travel
through a business process. Examples of trackable
data are the business messages themselves, and the
process events indicating whether a message has
passed through a particular set of business
process steps.

Advanced value-added services can provide
the data collection services, the query
mechanisms, and the reporting capabilities that
enable all this data to be gathered and presented
in a meaningful way. XML persistence services can
provide caching and aggregation points that can
collect data to be transformed for the purpose of
providing data to feed into other applications, or
to feed into human-readable reporting mechanisms
that can be used by business analysts. This means
that data flowing through the ESB can be analyzed
in real time to produce live information about the
nature of your business—for example, to provide a
realistic snapshot of where your inventory is at
all times within a supply chain.

Incremental Adoption

One of the primary differentiating qualities of an ESB
is its ability to allow incremental adoption, as
opposed to being an all-or-nothing proposition. In
the post-Y2K spending slowdown, budgets for
multimillion-dollar IT projects are just not what
they used to be. There is some indication that IT
funding is getting freed up to solve the
short-term tactical integration needs, but budgets
are still being highly scrutinized at an executive
level. At the same time, however, there is still a
desire to implement larger corporate-wide
strategic initiatives—all of which rely heavily on
integration and reuse of existing IT
assets.

Chapter 1
illustrates how an ESB can be used for small
projects that each build into a much larger
integration network. We’ll see how this is
accomplished as we delve into the details
throughout this book.

The ESB supports incremental integration, while working toward a more strategic goal

Figure 1-10. The ESB supports incremental integration,
while working toward a more strategic goal

The federated/autonomous capabilities of the
ESB also contribute to the ability to adopt an ESB
one project at a time. Incrementally staged
deployments of ESB integration projects can
provide immediate value while working toward the
broader corporate initiatives.

This notion of incremental adoption is also
further supported by the ability to bridge into an
existing integration broker hub and legacy message
brokers. Integration broker hubs and their traits
are explored in more detail in Chapter 2.