Decisions and Complex Event Processing

Complex event processing (CEP) continually evolves as a technology to help organizations react to various business conditions such as threats and market opportunities. CEP technologies excel at taking raw, voluminous amounts of event data, accumulating it, correlating it and aggregating it in forms that facilitate critical decision-making.

Events are simply something that happens within a company: a point of sale purchase, a dip below minimum quantity threshold in a warehouse or a fraudulent credit card transaction. Equally important, events can represent something that doesn’t happen: no sales for a product within a given time frame, or no response from an online CRM session for more than 5 minutes. All of these examples give cause for managing real-time events, and equally as important, making well-informed decisions what to do about them.

Traditional examples of business events reference voluminous data sources such as stock tickers, however complex event processing does not necessarily imply thousands of events per second. Events can be complex in nature yet less frequent in number. For example, managing the events of a warehouse may not produce a data stream of thousands of events per second, however the relationship between the events can be complex enough to warrant a CEP solution that can produce meaningful correlations of the data.

Due to the fact that well-informed decision making is a critical outcome of event processing, it makes sense to pay considerable attention to how those decisions are managed. After all, why go through the time and effort to process events without the ability to intelligently and consistently make decisions about them?

Within the context of CEP there are commonly two types of decisioning – manual and automated. The former sends processed event data to a dashboard for manual decisioning by a business resource – perhaps initiating product orders or kicking off another business process. The latter is the focal point of this paper – automated decisioning entities that are called by the CEP engine to perform some action.

CEP solutions typically provide a mechanism to maintain decisioning entities as “rules” that enable the implementation of logic to make decisions based on patterns of events. But successful CEP rule maintenance requires more than just capability. Rule maintenance for CEP should be governed by the same principles as traditional rule authoring: rules should be well-defined, not duplicated, traceable to business requirements and efficiently re-used where possible. Decision Management can help.

If CEP needs decisions, CEP needs Decision Management

Decision Management has long been established as a strategic approach for maintaining and automating decisions. In a simplified perspective Decision Management is comprised of an approach, governance and technology. The approach and governance helps define how to manage the decisions, while the technology defines how to execute the decisions. Because a CEP implementation requires decisioning to deliver value, it makes sense that Decision Management principles should be applied, at least in part, to the decision entities.

Benefits of leveraging a Decision Management approach for CEP decision entities:

Decisions can be evolved over time independent of the events and processes that invoke them
Decisions are maintained by the business users that understand them best
Decisions can be leveraged in scalable and re-usable ways

Benefits of leveraging Decision Management governance for CEP decision entities:

Clear definitions of the roles and responsibilities of decision-managing resources
Guidelines for decision traceability
Identification of key management personnel and stakeholders

While Decision Management approaches and governance are wholly applicable to managing CEP decisions, technology is often more difficult to discern. This is because CEP technologies and Decision Management technologies (such as Business Rule Management Systems - BRMS) provide overlapping capabilities which make it difficult to identify when one is clearly more applicable than the other. Consider the following question that can foster a productive discussion and produce valuable insight into how Decision Management technologies can and should apply in a given CEP scenario.

Question: What types of automated decisions are being managed?

A CEP implementation typically leverages two types of automated decisions: event-driven and business-driven.

Event-driven decisions are those that are closely tied to the events themselves. Consider an event correlation in which two credit card transactions are posted in close time proximity but far geographic proximity. An example of an event-driven decision is (in pseudo-code):

If travel time between geographic locations A & B is greater than the amount of time between two transactions in locations A & B, flag the account as fraud-compromised and deny the second transaction.

The decision to flag the account and deny the second transaction is closely tied to the events themselves. If the logic for identifying a fraud pattern in event data is changed, the decision must be changed as well or be rendered meaningless. Furthermore, within this context the decision is not generally applicable in other application areas outside of event processing. It therefore makes sense that the deployment and execution of these decisions is performed inside the event processing engine.

Business-driven decisions are those that are independent of the events themselves, but define how decisions are made under certain circumstances. Consider the same credit card transaction scenario. An example of a business-driven decision is (in pseudo-code):

If customer account is flagged as fraud-compromised and customer rating is “average”, notify customer, suspend account and initiate fraud resolution business process.

The decision to notify customer, suspend account and initiate the fraud resolution business process is independent of the events that may have invoked them. If the logic for identifying a fraud pattern in the event data is changed, the decision remains relevant. Additionally, because the business decision could be re-used in various business processes, it makes sense that it is deployed external to the event processing engine.

The discussion of CEP technology versus Decision Management technology continues to create understandable debate among vendors and professionals alike, however this question and example are not intended to provide “fuel to the fire”. They are simply provided to convey the fact that the current evolution of these technologies provides options for how to best leverage each technology in certain circumstances. There are challenges that only require CEP technologies, some that only require Decision Management technologies such as BRMS, and some that require a hybrid approach. Despite the importance of investigating the pros and cons of each technology and supporting circumstances, it is more important that the technological debate not overshadow the fact that decisions are critical to CEP, and the way we manage them is of equal criticality.

To maximize the success of CEP implementations there must be sufficient focus on managing the decisions that are to be made when certain events or even correlations present themselves. Whether those decisions are executed internal or external to the event processing engine, Decision Management lends proven practices to analyze, trace, document, manage and maintain decisions.