I have been involved in a number of discussions recently on the topic of what it means for software to be service oriented vs. service enabled. As a software vendor that markets service oriented software that service enables existing infrastructure this distinction is clearly important to us, but why should you care? This distinction is relevant to anyone investigating or applying SOA business solutions, using SOA suites or ESB technologies because understanding this distinction will inform you of what you are getting into and what you can expect for short and long term TCO and ROI.

There is much discussion on the topic of what level of granularity or coarseness right-sizes your services. The reality is that to get the full benefit of service enabling an existing system one has to implement a pattern of services that are semantically related and scale from fine-grained to coarse-grained services many times over.

For example, if we look at the data access services, we should see a large collection of really fine grained services that operate at a resource or transaction level of scope. This builds the pattern for information access and defines what is available. Ideally, these services would deliver their payloads through composition with a Canonical Information Service, but that topic is for another blog post.

As we pull back to “zoom out” of the coarseness of services, we suddenly see that the set of services are semantically related but providing composite services such as BPM sub-processes, embedded data flows and higher level services that drive workflows assembled directly from the collection of finer grained services and composed with business rules, analytic models and other services.

Now we truly have the infrastructure of services required to power business process agility and compose composite applications that can draw on and mash together business information in a dynamic manner with business rules enforced at every level of service utilization.

Because the pattern is the same at every level of granularity, I call this Fractal SOA. You can use Fractal SOA as a test to determine if you are looking at something that is service oriented or just service enabled. You can also use Fractal SOA as a model for implementing your services when planning and deploying your SOA. Ideally you would use a technology that is service oriented and supports this architectural approach directly (here comes the shameless plug) such as our SOAppliance.

You expect your existing IT to get service enabled, but how will that be done? Here you want a technology set that is service oriented, not service enabled. When you peel back the veneer of that ESB or other SOA infrastructure suite your vendors are showing you, you should see a set of services, not a wrapper-ed EAI or a semi-proprietary publish/subscribe messaging infrastructure. If it doesn’t look SOA on the inside, its not Fractal and therefore is itself a service enabled technology that will ultimately limit the ability to truly service enable your information environment.

SOA brings many promises to the Enterprise. Chief among these is the ability for IT to respond to ever-shifting business needs. In the current SOA landscape the answers to accessing Enterprise data are the same as they were pre-SOA; namely: SOAP, WS-*, proprietary APIs, etc… Time and experience have told us that these technologies do not in fact really free Enterprise data. This is not a new or hidden concept, yet the technology landscape (driven by SOA), is not changing to adopt to this reality.Right now there are legions of Enterprise IT developers that are either writing custom code to consume complex non-standard Web Services APIs or they themselves are writing their own complex non-standard Web Services APIs. Meanwhile, we are in the midst of a revolution in the human-web. Concepts such as mash-ups, Web 2.0, etc…, are transforming the web into a user-driven, world-wide, application platform. This is a world in which users can mesh and extend data in ways that the shepherds of these data could never have imagined. All of this is possible because the human-web is simple: HTTP, URIs, and (X)HTML documents.

In the computer-web, especially in the Enterprise, there is no unified way to access data. Every data access service, web or not, has its own proprietary access mechanism. Or at best one of thousands of competing “standards” are used. This prevents the computer-web from experiencing the revolution that the human-web is undergoing. The development of novel, user-driven applications and IT’s ability to respond to inevitable business shifts is greatly hampered as a result. The good news is that the tools for solving these problems are already here: HTTP, URI, XML and they are already ubiquitous. Also we already have a set of design criteria (REST) that help usher in this new age of true SOA data access. We can now make the data accessible to users; give developers a unified interface to operate on the data; give Business Analysts common (canonical) representations of the data; and more. Finally, we can leverage all of the lessons learnt (scalability, reliability, security, etc…) by the human-web.

I propose that we call these SOA data access services Resource Services. Resource, since that is a fundamental concept in REST; and Service, well because they are to be services, web services in particular.

As an example, imagine letting a Google Search Appliance loose on a set of corporate data exposed as Resource Services. The resulting index could cross all data silos and harness the power of the Google search algorithms to give (the appropriate) users a wealth of information about their data. Clearly this raises all sorts of security and governance concerns, but that is a topic for another day. This is just a simple example, the key is that we can not possibly think up all the ways that this data might be harnessed, if it were easily accessible via a standard mechanism. It is time to stop imagining this brave new world and to start enabling it.

Every interdepartmental “Enterprise” application in your company is likely integrated in some fashion with just about every other “Enterprise” application in your company. Enterprise Project Management Applications however, are often left out of the overall enterprise integration architecture and only point to point integrations are attempted- usually with less than spectacular results. The end result is that EPM applications have largely been left out in the cold and core adoption of this “Enterprise” function does not occur. This leaves EPM information and processes an island to themselves.

Even with many “Integrated Project Management” solutions, the integrations are EAI style in nature and not process execution sensitive. What use is having the project and financial systems in sync at 10PM every Friday or on some published data change event if the project data is not status ready? In fact, this kind of behavior does more harm than good as inaccurate information is propagated across the IT landscape and introduces erroneous reporting. This erodes confidence in EPM and EPM processes and any ability for EPM to become a truly enterprise level inter-departmental competency.

There are a number of reasons those solutions good enough for ERP, CRM, SFA and the like failed for EPM. The first and most important reason is that EPM is a process centric view of projects and project information while most other enterprise functions provide a data centric model. It is no wonder that a capability set built around managing static structures would fail when asked to manage dynamic processes.

The second reason is that most enterprise applications, perhaps because they are data centric, only rely on light integration points and small amounts of “keying” and meta-data to relate like information across systems. EPM however, is essentially a nexus of static structures whose intersections are governed by EPM dynamic processes. Simply put, EPM requires dynamic, workflow driven data access and data flow across many enterprise applications.

Another contributor to the mess of legacy EPM integrations is that the “system of record” for EPM data has a granularity at the field level, and the “system of record” for many fields migrate from application to application depending on the current process state of any given project. Try managing that with an EAI and you’ll end up talking gibberish to the nice folks in the white jumpsuits. (They will take good care of you, but I still don’t recommend it.)

Lastly, EPM has the misfortune of being an adolescent. Project Management has for years languished as a highly specialized discipline that is poorly understood outside of PMO and other PM disciplined departments. Recently however, Project Management technologies have finally leaped off the desktop onto server and web based tools allowing for wider participation and audiences. This jump was accompanied by government accountability mandates essentially dictating enterprise project management best practices be adopted en mass. The corporate world also seems to have awakened to the very real business benefits derived from practicing enterprise project management as a core competency.

The good news is that unlike a human adolescent, all this attention is maturing the EPM vendor market rapidly while spurning rapid innovation by new and established EPM ISVs concurrently. The bad news is that this brings EPM up to where ERP was ten years ago.

This configuration creates an opportunity to solve for the failure modes described earlier. Technology and tools now exist to rescue EPM from its island of obscurity and we can do it rapidly (as in– not over years, over the next few weeks). You have probably guessed by now what I’m going to say next.

SOA.

Dash! There are almost no SOA EPM products on the market. There are a few that have Web Services and XML APIs but these rarely provide complete coverage of the business objects and processes required to truly enable EPM to be Enterprise.
There is a ray of hope however. EPM applications can be Service Enabled and become consumable as Data Access Services using Web Services or REST. Advanced EPM applications will even provide event notification capabilities that can be extended into the SOA to further enhance dynamic processing of EPM process activities.

Once service enabled, EPM information and processes can be extended across applications, users and workflows using standards based technologies.

Business Process Management (BPM), or building, executing and managing business process has already become a standards based commodity SOA technology. BPM coupled with workflow driven composite applications completely solve the process v. data issue that failed EAI and other legacy integration technologies.

The loosely coupled nature of SOA solutions allows for data access across many systems without the introduction of physical data dependencies across systems (information addiction codependence).

BPM and Workflow services that consume business rules can deliver process controlled “system of record” data integrity logic at the field level. A best practices solution will consume and fire business rules on a canonical information model rather than a data structure tied to particular applications and application versions.

Composite Applications or mash-ups, delivered through enterprise portals and corporate intranets allow EPM process and human interaction points to be tailored and extended across the landscape of project participants.

Shameless Plug: SOALogix makes a product that does this.

(A note on SOALogix inaugural blog entry)

Up until now I have resisted throwing in and adding yet another finite opinion on exactly what SOA is; but now here I am. Two events have conspired to bring this change about:

The first is that we just changed our brand to SOALogix. It now seems appropriate for me to suddenly start blogging my adventures, learning and journeys related to SOA. As we have been practicing SOA principles and building SOA technologies and solutions since 2003, there are many adventures to be retold.

The second is that as part of our re-branding, our new website sports this spiffy blog feature that needs to be fed interesting content on a regular basis and again, it seems appropriate that I should be here.

You will also find upcoming entries from our technical, consulting and project services staff on a variety of topics covering SOA, business process integration, enterprise projects, composite applications and more.

We hope that everyone finds this forum to be interesting, relevant and available for open and sincere community dialog.

An SOA Reference Model

Since there is no single “master” specification for an SOA, we and everyone else wade through a plethora of overloaded terminology and expectation alignment. Terms like “choreography”, “workflow” and “integration” have different meanings to almost everyone and the question of dictionary invariably comes up during SOA related discussions.

Over the years, a few organizations have published “SOA Reference Models”, each with its own unique qualified rendering of what an SOA looks like. There are models that come from many different aspects: logical architecture, interface architecture, standards architecture, etc. The problem we have is that they mostly push an agenda and some rely on visual metaphors to “vendor enhance” the model or are based on a vendor specific product offering.

We could at least standardize the SOA dictionary for ourselves and create a way to communicate to our customers where our software capabilities and strengths for their situation are. To that end, we looked at all the SOA Reference Models we could find on the Internet, got all our senior technocrats together with double rounds of espresso and hashed out what we think is a solid abstraction for an SOA Reference Model. To all those who have gone before us, we give thanks for their work as the prior-art here was instrumental in shaping our own thinking and work product.

The first thing we wanted to accomplish with our reference model was to establish the concept of service domains bounded by specified separation of concerns. Of course, attempting to do this naturally led to a two hour SOA dictionary discussion, more espresso and ultimately a much refined SOA terminology set for our model.

After the preliminary work, we thought that producing the model would be cake, but again we were foiled in our initial attempts. The first person outside of the modeling team we showed the picture to said, “Oh, SOA is a layered architecture” and that is not what we wanted at all!

So back to the espresso machine, and then to the proverbial and literal drawing board. After a few more rounds, a few test subjects and one very unsuspecting sales person we arrived at our final reference model work products and were able to clearly communicate at least to each other what SOA looks like to us.

Even if you don’t agree with or like our model, I highly recommend that you get one, or put in the time to make one, and then educate everyone involved in your SOA effort on what it is and what it means. You can always revise the model as you go and there are enough examples (now including ours!) running around on the web to bootstrap the effort.

The picture below is the SOA reference we model finalized on. Its hard not to, but don’t look at the vertical as a “stack” or layers. Put your self directly on top of the picture and look “down” on it so that the ESB is “underneath” the service domains.

The one compromise we made on our level of abstraction for the rendering is to show an ESB in the diagram. We did this mainly to show that in our view, ESBs as well as SAP’s ESA provide “Backplane Services” to an SOA; which is different from the central role they are given in many other representations.

We also tried to stay with popular terminology semantics as much as possible, but there are a couple of notable exceptions. Remember that we wanted to create our service domains for our model based on a separation of concerns independent of any specific product offerings, technology packaging or specific standards implementations. That information we have captured in lower level models which are mapped up to our reference model.

The most notable distinction of our model is that we have separated out the “workflow” domain into Choreography services that invoke primarily BPM services and Information Services. We have carried this concept forward into our model out of the EA world from Zachman’s EA framework, where BPM and Workflow are clearly shown as separated concerns.

Another specific area we felt needed to be provided its own concern is Canonical Data Services. While most models contain an Information Services category, “integration server” is implied by the context in many. We combined these two service categories into a single service domain as they deal with different aspects of providing information services: integration and mediation services for information products consumable by business and user level services. This formalization of the business contextualized information provides a clean de-coupling from individual system’s data structures and fine-grained services and message formats from business rules, workflows and business processes without boundless transformation business logic scattered across architectural components.

We purposely left out the Portal component found in many of the published models because we felt that it was too specific a delivery technology to show in a reference model. We landed on Display/User Interactivity Services to allow for a wide variety of interpretations for presentation and activity implementations beyond the “corporate portal”. There are mash-ups, cell phones, cell phone mash-ups, streaming clients, etc that can all leverage services.

So that’s the SOA reference model we came up with, and it is my SOA reference model. By that I am not asserting personal ownership, but rather personal buy in. I use this model on a daily basis and carry it around in my head. I have found that it makes navigating the land of SOA more manageable and (hopefully) more objective while at the same time providing the foundation for a standard SOA dictionary, at least for 1 organization.

Like I said back at the beginning- if you like our model, I hope it brings you value; if you don’t like our model, go make your own!