ESB Evolution

The concept of ESB was born out of the need to move away from point-to-point integration, which becomes brittle and hard to manage over the period. Point-to-point integration also results in custom integration code being spreaded among applications with no central way to monitor or troubleshoot. This is often referred to as “spaghetti situation” and does not scale because it creates tight dependencies between applications. It initially examines the present condition of venture integration and the problems of the hub-and-spoke topology, and then it initiates a new approach for EAI based on ESB called EAI-ESB. This approach which encloses Message building, Messaging channels, Listener, Decryptor, Validator, Enricher, Transformer and Router eight steps guarantees reliable and authenticated data which will be routed all through the ESB between numerous functions.

ESB Trends

An Enterprise Service Bus (ESB) is fundamentally best of breed architecture. It is a set of design patterns and principles for integrating numerous applications together over a bus-like infrastructure. ESB architecture based middleware products enable users to build this type of architecture, but vary in the way that they do it and the capabilities that they offer. The core concept of the ESB architecture is that you integrate different applications by putting a communication bus between them and then enable each application to talk through bus. This decouples systems from each other, allowing them to communicate without dependency on or knowledge of other systems on the bus. ESB is part of middleware that permits applications written in dissimilar language to effortlessly converse with one another. Unlike the hub-and-spoke architecture model, enterprise application integration (EAI) architectures, an ESB has no middle point that all communications must pass through. Instead, it has various access and outlet points beside a spectrum, so there is no central point of breakdown.

ESB Tools

Operation and management module enable reliable information of the ESB. Statistics and status provides their number of errors, minimum and maximum response times and number of processed messages. Alert mechanism can be used to sending alert messages that can be sent to various channels so that existing monitoring environment can also be incorporated.

To track the message, Message Tracking should be activated whenever required, so as to minimize any associated overhead. Message Redelivery ensures that messages that aren’t processed immediately are automatically resent after a pre-defined period of time. The number of attempts and the interval between them can be configured. This component is primarily suitable for technical errors that only last a certain length of time, such as temporary network outages. Endpoint Failover enables the option of specifying an alternate service provider that is automatically called whenever the primary service provider is not available.

Mediation module can be used to implement the message flow of an ESB Service. Message Routing allows messages to be forwarded to a particular service endpoint depending on their content. Message Transformation enables conversion from one message format to another that is applicable to text and binary messages as well as XML formats and it is applicable to EDI, HL7 and CSV.

Message Exchange Pattern (MEP) is the support of message exchange patterns, such as synchronous and asynchronous request/reply, one-way call, and publish/subscribe. Transaction allows ESBs offer transactional integrity through message processing. The persistent queues that the ESB uses to support Reliable Messaging generally work as transactional data sources, and can therefore participate in heterogeneous transactions. In addition, the ESB can offer a distributed transaction manager that can coordinate distributed transactions via heterogeneous data sources using the two-phase commit protocol.

Security module supports both the transport-level and message-level security using a number of components. Authentication authenticates service consumers when they access the service in the ESB and verifies ESB authentication for the service provider. Authorization provides an authorization system for services that can often be configured via XACML to be assigned to users or roles. Security Mediation provides support for interactions that communicate outside of security domains by converting credentials from one domain to the corresponding credentials of the other domain. Encryption/Decryption supports the encryption and decryption of the content of a message.

An adapter or connector that sits between the application and the ESB and manages data transformation between the two. The adapter uses an API to bridge the connection between the application and the bus. Other than collecting, storing, maintaining, and queuing information between applications, the adapter also performs a range of other activities such as security maintenance, data monitoring and error handling.

ESB architecture complements all the core principles of integration, such as orchestration, transformation, transportation, mediation, and event handling. It facilitates complex event processing, provides support for synchronous and asynchronous transport protocols, validates messages, governs business rules, and provides interoperability between applications irrespective of operating systems and programming languages used in each of the tools.

Core ESB Patterns with adapters

Protocol Switch Pattern

It enables service requestors to dispatch their messages using variety of interaction protocols or APIs. Such as SOAP/HTTP, FILE, FTP, SMTP, JMS, DB and MQ etc. like JCA

Modifier Pattern

Transformation Sub pattern

Message payload is transformed from one format (schema) to another to match definition of a message of the requestor to that of a provider, Includes “enveloping and de-enveloping,” which is the process of putting a message in one network format inside the format envelope needed for transmission over another network or corresponding removal of an envelope and Includes encryption

Enrichment sub pattern

Message payload is updated by adding information from external data sources such as custom parameters of the mediation or database queries.

Discovery Patterns

Set of possible service providers is not preconfigured at the mediation. Providers match requestor’s message format, QOS required or the protocol supported from the mediation to the possible providers

Monitor

  • Used to observe messages as they pass through the ESB without changing them in any way. Examples:
    • Monitor service levels
    • Provide data for problem determination
    • Chargeback Metering
    • Record business-level events (transactions exceeding a certain revenue)
    • Log messages for audit e.g., Reporting providers or BAM adapters

Correlate

Derives complex events from message or event streams. Includes rules for pattern identification and rules that react to pattern discovery e.g., business rules, human task

Canonical

Set of messages and business objects used by all parties is normalized to a canonical format. This is an aggregate of Protocol Switch and Transform patterns

 Proxy or Gateway Pattern

Variant of routing or protocol switch pattern which maps service endpoints, possibly providing security functions (authorization and access control) and logging or auditing capabilities. May incorporate transform and monitor mediations to provide encryption and logging, or auditing. It may also aggregate and disaggregate messages in a one-to-many relationship.

Example: Service portals which act as a single point of contact for multiple services and hide the details of “internal” services

Conclusion

With all proven BUS architecture, prebuilt design patterns, adapter, Services, XML, monitoring, rules engine and all of above drag-drop approach of development made the integration job much easier. Today, ESB plays key role in many enterprises for integration both internal and external systems.

Note: The above article is authored by Mr.Upahara Arunraj from Enterprise Integration & Information Management Practice @ Aspire Systems.