realprogrammers.com

The XML Web

Bill Zissimopoulos (zed@realprogrammers.com)

The first generation of the Web was about human-readable data. Technologies such as URI, HTTP and HTML allowed the creation of a shared information space accessible from a variety of computing environments. No prior coordination was needed to publish new information or to have it link to other information. This allowed the information space to scale virtually infinitely and form the World Wide Web as we know it today.

In his PhD dissertation "Architectural Styles and the Design of Network-based Software Architectures", Roy T. Fielding analyzed and studied the Web's architectural style. He identified certain architectural elements which he coined with the term REST (Representational State Transfer).

REST hypothesizes that having a limited but uniform interface between components that can be applied to a potentially infinite number of data elements is what let the Web achieve its success. In the Web architecture the interface is HTTP and the methods are the familiar GET/PUT/DELETE and POST (in practice only GET and POST are commonly used). The data elements called resources have names, furthermore these names can be used to locate the resource. Resources are not necessarily static data, in fact often they are services that produce data dynamically. [For further discussion on REST visit the RESTwiki.]

While HTML (and to a lesser degree standards such as GIF and JPEG)was crucial in the success of the web as a human-readable information space, it was the URI and HTTP technologies that gave to the Web its scalability and versatility. Using the same technologies (URI and HTTP) as foundations to create a machine-readable information space is not only interesting but also highly desirable.

The second generation Web is slowly emerging. XML, Schema and related technologies allow us to describe structured and typed data. XPath and XPointer allow us to address XML fragments using normal URIs. XInclude and XLink allow XML documents to embed and refer to other XML documents. W3C's Semantic Web is an effort to attach meaning to data. Some people hypothesize that the URI, HTTP and XML technologies will make current RPC-over-the-web technologies such as SOAP and XML-RPC irrelevant. [See Paul Prescod's Second Generation Web Services.]

Clearly the Web is undergoing a transformation from a human-readable Web to a machine-readable (and potentially universally machine-understandable) Web. What is needed to realize this potential is software that allows the storing, accessing and manipulating of XML data using standard and proven Web technologies. In particular we need Web servers that can store and efficiently query and manipulate XML data. We also need Web clients (browsers, application libraries, applications, etc.) that can meaningfully deal with XML.

Software for the XML Web

There exists today a variety of Web software, both client and server. Much of this software can be leveraged for enabling the XML Web. New software will also have to be built. It is important to note here that for maximum applicability, we are only interested in software that is not tied to a particular application. For example, we are not interested in software that can pull and render XHTML (a browser), as this is only one application of the XML Web.

In fact most client software is rather uninteresting for enabling the XML Web. Client software typically consists of software components that deal with connectivity and data transfer and software components that deal with data meaning. The connectivity software components are a problem that has been already solved since XML data is just data. The software components that deal with data meaning are fundamentally tied to one particular application domain and are therefore uninteresting for a generic XML Web client layer.

The situation on the server side is much more interesting. Although one can use a normal Web server to serve XML documents, this is not sufficient to realize the true potential of the XML Web. The potential applications are far more interesting if we allow XML fragments to be addressable so that they can be accessed and manipulated.

Consider for example a next generation personal addressbook application which stores personal address data somewhere on the Web instead of a local SQL database. This makes the data available from everywhere in the world. More importantly the actual data is available and not simply a user interface into the data. [In the first generation Web one can create an addressbook application that implements a user interface into the data and is accessible from everywhere in the world. The limitation is that no other application has access to the data. Alternatively one can design an application that enables programmatic access to the data. However this is usually done in an ad-hoc and non-extensible fashion.]

One person's addressbook data might look like this:

<addresses>

  <address id="Home">

    <street>000 Davis Ct.</street>

    <city>San Francisco</city>

    <zip>94111</zip>

  </address>

  <address id="Work">

    <street>466 Ellis St.</street>

    <city>Mountain View</city>

    <zip>94043</zip>

  </address>

</addresses>

With current Web server software one can make this document available under a URL, for example: http://addressbook.example.com/bill.xml. Client software that attempts to fetch the resource pointed to by the URL will fetch all of the person's addresses. There is no way to perform a query that fetches a specific address or addresses and minimizes the amount of transferred data. More importantly if the software wishes to update the details of a single address element it must update the whole document.

Let us now consider Web server software that allows portions of the above XML document to be accessed and manipulated. Let us further consider the problem of finding all addresses in San Francisco. The client software issues a request for the resource pointed to by the URL: http://addressbook.example.com/bill.xml#xpointer(//address[city%3D'San%20Francisco']) [XPath: //address[city='San Francisco']]. Only the San Francisco addresses are returned. Similarly if one wanted to change the home address, the client software would issue an update of the resource pointed to by the URL: http://addressbook.example.com/bill.xml#xpointer(//address[%40id%3D'Home']) [XPath: //address[@id='Home']]. Only the home address would be updated.

Marrying an HTTP server with an XML database

From the client perspective, the Web server of the previous example is just a common Web server that is based on the common URI and HTTP technologies. A common Web client (such as Microsoft's Internet Explorer) is all that is needed to view the XML data. Although interesting for testing, such a client would not be as useful in general, since it cannot do anything meaningful with the data.

From the server perspective, things are far more interesting. This next generation Web server, must be able to perform dynamic queries and updates on data. In effect this Web server is an (XML) database with an HTTP interface. The traditional query operation (SELECT) is mapped to HTTP GET. The traditional data manipulation operations (INSERT/UPDATE/DELETE) are mapped to HTTP PUT and HTTP DELETE. XPath/XPointer is used to reference XML data. Data aggregation and linking can be supported by leveraging XInclude and XLink. Finally HTTP POST can be used to send XSLT to the server that can then be used to update referenced XML data in place.

[This last idea is contrary to the ordinary client-server model. In traditional client-server, the data travels to the client and processing happens there. In this model, the client sends a program to the server (the program travels) and processing happens on the server. This can be quite advantageous in situations where, for example, a 2K program must update a database of millions of records.]