RESTful Web Services - SOA Made Simple?

In the SOA community there has been ongoing debate over SOAP versus REST for a number of years now. I've decided to add my 'pennies worth' to the argument. I feel I can add a unique perspective on this subject, as during 1998/99 I was heavily involved in the early Beta-testing of Microsoft Biztalk and had access to a lot of the formative thinking that eventually went into defining SOAP and the WS-* standards.

To really put the debate in perspective you need to understand the roots and history of the SOAP protocol and the people behind it.

The story starts back in 1998 when the XML 1.0 specification was agreed, IT vendors like Microsoft and IBM were beginning to get their heads around the idea of distributed computing using HTTP. Let's not forget, the 90's was all about the transition to client / server computing then n-tier architectures and middleware such as CORBA, J2EE, Microsoft Windows DNA (now .NET). At that time I was working with a start-up company that produced one of the first commercial COTS products based on Windows DNA. In my view, at that time, Sun Java J2EE lagged Microsoft DCOM MTS in defining n-tier architecture best practice.

The starting point for SOAP was XML-RPC. XML-RPC was pretty straightforward, it took the view that you could use the core HTTP mechanisms of GET, POST, PUT and DELETE to call a remote host API with XML as the payload. Everyone then understood that HTTP and XML was going to become the heterogeneous holy grail that we needed to promote open interoperable distributed computing.

A typical XML-RPC fragment looks something like this:

POST /myservice HTTP/1.0
User-Agent: Frontier/5.1.2 (WinNT)
Host: myhost.com
Content-Type: text/xml
Content-length: 181

<?xml version="1.0"?>
  <methodCall>
    <methodName>myService.getMethodName</methodName>
    <params>
      <param>
        <value><i4>41</i4></value>
      </param>
    </params>
  </methodCall>

And a response that looks something like:

HTTP/1.1 200 OK
Connection: close
Content-Length: 158
Content-Type: text/xml
Date: Fri, 17 Jul 1998 19:55:08 GMT
Server: UserLand Frontier/5.1.2-WinNT

<?xml version="1.0"?>
  <methodResponse>
    <params>
      <param>
        <value><string>Returned Value</string></value>
      </param>
    </params>
  </methodResponse>

Pretty simple.

In 2000 I worked on a Touch-Screen Kiosk system that brought content from different web systems together in a client portal. We hooked into services such as Multimap and UpMyStreet all using an XML-RPC approach and it worked great.

Of course there are issues with this. How do you discover what services are available? How can a client understand what the description of the API is, data types etc? How do you secure an API on a public network? How do handle transactions?

These issues kicked started the vision for SOAP and WS-* standards. It may be a surprise to some, but the key organisation behind SOAP in the early days was Microsoft, that bastion of interoperability and open standards. The original contributors to SOAP worked on the Microsoft DCOM MTS team. It is, therefore, not surprising that the original SOAP proposal assumed a COM type system underneath the hood. DCOM supported HTTP as well as TCP/IP as a network transport. There was also an internal battle going on between the DCOM based SOAP and XML data teams within Microsoft as to how these two technologies should interact. This delayed the ratification of SOAP 1.0 until late 1999.

Knowing the background of the people who drove the early days of SOAP, it should be no surprise that SOAP is effectively DCOM for XML over HTTP. The architectural principles behind SOAP can be found in all the RPC middleware solutions of the 90s, i.e. Windows DNA, CORBA and J2EE. Also, with key vendors such as Microsoft and IBM driving the SOAP and WS-* standards, I suspect their motives behind architectural decisions were partly driven by commercial opportunities for selling products such as development tools, application servers etc.

Around this time, Roy Fielding, co-founder of the Apache project, was working on a dissertation for his PhD at University of California. Roy Fielding was one of the pioneers behind HTTP and HTML. Roy's dissertation, Architectural Styles and the Design of Network-based Software Architectures, probably didn't register any further than a few interested academic circles at the time, but now is regarded as the definitive guide to the architecture of the Web. In some ways, Roy's dissertation was a way to try and put some solid theory behind the evolution of technologies that became the Web. Sort of a retrospective design document if you like.

REST, as Roy articulates in his thesis, is an architectural style, not a protocol. Systems that are 'RESTful' adhere to the key architecture principles of REST. Unlike SOAP, REST is not a standard.

The key principles of REST are:

• Application and State are abstracted into Resources
• Every resource is uniquely addressable
• Client Server
• Stateless
• Cacheable
• Layered

Fielding also laid out the key goals of REST:

• Scaleability of component interactions
• Generality of interfaces
• Independent deployment of components
• Intermediary components to reduce interaction latency, enforce security, and encapsulate legacy systems

In simple terms, the SOAP vs. REST debate is a false one. You cannot compare SOAP and REST, they are completely different entities. SOAP is a standards based RPC middleware protocol, REST is an architectural style guide.

Fast forward to the present, 10 years since the release of SOAP v1.0, and where are we now with SOAP, REST and web services? Well for me a good indication of where web services are heading is to look to the major Internet and eCommerce players, I'm talking the Google, Yahoo, eBay and the Amazons of this world, and how they are providing access to their services.

It's interesting that most of these big players offer both SOAP and RESTful APIs to their services, yet reports have shown that the majority of developers prefer accessing the RESTful APIs over SOAP. Why is this? Simple, like electricity finding an earth, developers will always take the path of least resistance when developing any app. In essence, RESTful web services have proven to be substantially easier to use in the field than SOAP. This is not surprising either. If you read through the SOAP and WS-* specifications they are, in my view, near on unintelligible, and full of inconsistencies. If you want a light hearted criticism of SOAP, see this article by Pete Lacy. It's also interesting that the likes of Google who started providing both SOAP and RESTful access to their services have, in a lot of cases, deprecated their SOAP access.

So where do I stand on the SOAP vs. REST debate? The shear success of HTTP, therefore the principles of REST, coupled with its inherent simplicity, places me in the RESTful camp. Building SOA and web services on RESTful principles means you inherit all the proven advantages of HTTP. Re-read Fielding's goals for REST, these are entirely compatible with the goals of SOA.

It's my view that SOAP is 'tainted' with the client / server RPC thinking of the 1990's grafted on to HTTP. Looking at SOAP's history and the people who drove it's initial specification, you can understand why this could be the case.

Tim Berners-Lee's vision of the Web was simple:

"The Web is simply defined as the universe of global network-accessible information"

Berners-Lee view was that the primary purpose of the Web is to establish a globally shared information space, and 'legacy systems' could participate by publishing objects and services into this space. SOAP, on the other hand, talks about the web as a network transport, endpoints and bindings.

Taking Berners-Lee's core vision, SOAP, for me, adds an unnecessary level of indirection between the consumer and provider of information. In my view, the HTTP specification, used in conjunction with XML, already provides a robust mechanism for SOA across TCP/IP networks, what's the value of adding further complexity on top?

SOAP advocates will point to the issues of service discovery, security and transactions, I raised earlier, and argue this is what SOAP provides in conjunction with the WS- standards. I would argue that the proliferation of WS-* standards came about as sticking plasters to SOAP to make the Web Services behave more like traditional client / server RPC technologies.

It almost feels to me like SOAP and the WS- standards are 'fighting' against the purity and goals of HTTP, the primary protocol they run on.

Will REST win out? It's interesting to trawl the blogs and forums of the major vendors such as Oracle, Sun, IBM and Microsoft (coincidently the key vendors driving the web services standards) , and from around 2006 onwards you'll find an explosion of articles on REST. Yet go back further than that and it was all SOA equals Web Services equals SOAP.

Like a lot of software innovations these days, REST grew ground up from developers and Open Source communities, starts to become mainstream and the big players pick up on it. Ironically, considering REST is a style not a standard, you've now got the likes of Sun and w3C promoting standards around REST such as WADL

Even though I'm a fan of REST and Fielding's view of 'Resource Orientated' computing, I can still see a place for SOAP and WS-*. For example if you need to provide a service that requires transaction support, then SOAP's ideal. I've just got this feeling though, that the simplicity, purity, compatibility and scaleability of REST will win through in the end.

Time will tell!

If you're looking for a good technical overview of REST, I would recommend Elkstein's tutorial as a good starting point. I would also recommend listening to a podcast round table discussion of REST by a team from ThoughtWorks, including Martin Fowler. The postcast is split into two parts, the first part giving an overview and history of REST, the second going into more detail around modelling resources and deploying REST in the Enterprise.

Programming with Eclipse BIRT

A couple of years ago I started working on an Agent-Based Simulation System. This system simulated the behaviour of fleets of complex assets, such as aircraft, to support predictive analysis of maintenance, availability, system failure etc.

One of the (many) key challenges of the system architecture was the shear volume of data each simulation could potentially generate and the visualisation of the results. At the time the organisation I was working in hadn't decided on a corporate Business Intelligence / Analytics solution, so I had the task of looking for an appropriate technology to plug in to the system architecture.

It made sense that the solution was Open Source, looking around there were a few options including Jasper Reports, in the end I decided on Eclipse BIRT. I choose BIRT mainly because we were already using the Eclipse Platform for the rest of the system architecture development effort and, to be honest, I was quietly impressed with the capability, power and ease of use. Also BIRT has the advantage of being Open Source but with a solid commerical entity behind it in the form of Actuate.

BIRT's architecture is, essentially, split into two components, a Designer tool to maintain the report design, and a Java runtime Report Engine that manages and executes the report. The Designer component can either run as an Eclipse RCP client or Eclipse IDE Plug-In. The Report Engine will deploy to any standard Servlet Engine / J2ee Application Server environment, e.g. JBoss AS, Tomcat, Websphere etc.

An overview of the architecture is shown below.



The architecture of BIRT is pretty open and extensible. For example data source connectivity is provided by ODA which allows you to connect to different physical data sources, such as JDBC RDBMS, Web Services, Flat Files or Java Beans, yet treat them all as logic result sets. In fact, one of my first 'evaluations' of BIRT's capabilities involved building a web reporting front-end to Amazon's ECS (now known as AWS Associates Web Service).

For me, the real power of BIRT is the Server Event model that allows you alter the look, feel or behaviour of the report at runtime. An API is provided for the event model and can be accessed via Rhino (Javascript based scripting for Java) and native Java. An overview of the event model can be seen below.


Events are exposed on numerous report objects, for example DataSet.beforeOpen(), ReportElement.onPrepare() etc.

These events allow you to do all sort of 'clever stuff' at runtime. For example, change the SQL of a query based on a parameter or the outcome of a previous query, change repot or chart colours based on calculated values.

For example, the following code changes the SQL expression dependent on a runtime parameter rp_sortorder:

myDataSet.beforeOpen() {
  this.queryText=this.queryText.replace('sortlistorder',params["rp_sortorder"]);
}

You can also get access the the J2EE runtime container. For example, the following code gets access to a Http Session object.

report.initialize() {
  importPackage(Packages.javax.servlet.http);
  var thisSession=reportContext.getHttpServletRequest().getSession();
  var mySessionObject=thisSession.getAttribute("mySessionObject");
  var mySessionObjectValue=mySessionObject.getValue();
}

As well as a Report Design and Runtime (Engine) API, there's a Chart API that allows to extend the built in Chart functionality. For example, the code below alters the scale of a chart dependent on two global variables "ymindate" and "ymaxdate".

function beforeGeneration(chart, icsc) {
  importPackage(Packages.org.eclipse.birt.chart.model.data.impl);
  var scriptObj=icsc.getExternalContext().getScriptable();
  var yaxisminDate=scriptObj.getPersistentGlobalVariable("yminDate");
  var yaxismaxDate=scriptObj.getPersistentGlobalVariable("ymaxDate");
  var yaxismin=Date.parse(yaxisminDate);
  var yaxismax=Date.parse(yaxismaxDate);
  var xaxis=chart.getBaseAxes()[0];
  var yaxis=chart.getOrthogonalAxes(xaxis,true)[0];
  var yaxisScale=yaxis.getScale();
  yaxisScale.setMin(DateTimeDataElementImpl.create(yaxismin));
  yaxisScale.setMax(DateTimeDataElementImpl.create(yaxismax));
}

I am impressed with BIRT and I haven't yet found a reporting / analytics requirement it can't handle, coupled with it's extensibility and the backing of the Eclipse community and Actuate I highly recommend it.

If you want to find out more about BIRT:

• BIRT Eclipse - the home of BIRT development
• BIRT Exchange - a well supported and active BIRT community site
• BIRT BlogSpot - a blog about BIRT run by key BIRT team members