Defining a Common Web Service Interface

Introduction

This paper was drafted following a discussion between the developers of the xmms and noautun web interfaces.

The premise of the discussion was to provide a common means of publishing and accessing the services available in an applications web interface. As such, the following pertains to, but is not limited by, the web interfaces of the existing applications.

Terminology

There are two terms which will be used throughout this document:

Portal - a users single point of contact with the http services on offer
Transients - any http server that registers itself or is registered with a Portal

This document will focus on the interaction between the Portal and its Transients.

Background

The ease of implementation of http services and the prevalence of http clients makes it an ideal means of providing remote access to the functionality of applications.

With the ease of implementation and the existence of components to provide the functionality, the natural progression is into the form of transient servers - servers that are embedded in applications or are providing a very specific service with user interfaces being rendered in web browsers or accessed via other http enabled clients.

Identification of the kind of applications that are applicable to this kind of treatment are wide ranging and are beyond the scope of this document. The scope includes applications that are purely of interest within an intra-net to those which want some, or all, of their functionality exposed to the Internet.

Standard web/application servers (apache, iis, jrun, orion, et al)
Audio, Video streaming/selection services
Home surveillance
Centralised e-mail Clients
Centralised Internet Book-marking services
Control of Heating/Lighting
Control/Administration of set-top boxes
System Administration
Interaction with Robots
etc

Implementations of this form offer an ease of access which is unparalleled in terms of cross platform support - essentially any client wishing to access a specific service will need little more than a web browser and a network connection.

Further to this, transients may have a commonality which allows a more general access directly from the portal.

For example: If one or more transient offers an audio stream as a publicly accessible URL, ideally, the portal will be able to offer a view on all audio streams that it knows about.

Thus, the portal should allow you to bypass the interfaces of each individual server and allow direct access to the services provided.

Again, it is beyond the scope of this document to identify the commonality which may exist - however, the document will define the means by which a transient exports many specific service entry points so that any portal can implement views based on these.

Opportunity

Ultimately, implementations of the Transient server will lead to a proliferation of http servers which in turn introduces a number of logistical problems:

Security - how do we offer functionality without compromising security
Identification - what services does a server offer
Location - how a user is to find a service
Operability - different vendors will provide similar services in different ways

These technical issues are the driving force behind the proposals in this document.

[It is also the origin of the acronym which I tentatively forward as our project label - SILO].

The SILO Framework

Overview

The SILO framework is designed as a minimalistic collection of URL entry points.

As such, a Portal must provide the means to:

Register URLs
Unregister URLs

Within the acceptance/rejection process, an implementation of the a Portal may carry out any hand shaking it wishes with the Transient. It should inform the requester (by means of clear text), the outcome of its decision (in order that a transient/end user is able to determine what steps it should take to gain acceptance).

A portal may also offer Proxying functionality to provide access from internet connections to intranet transients. If it doesn't offer this facility, then it should not expose the internally registered transients to the outside world (hence a non-proxying portal is only useful to intranet users).

A Transient needn't implement anything, providing the security model of the portal allows it.

However, it may expose it's entry points to specific services:

Optional Entry Point Document URL

It may further reject any subsequent requests which are supplied via the portal (or those which don't come through the portal).

It may also register itself on start up and unregister itself on shutdown, but both of these actions may be carried out independently of the server itself.

A Minimal Portal

In it's most basic form, a Portal will accept registration requests for any URL (be it intranet or extranet) and a corresponding description. It will also accept an unregister request.

It will accumulate these and provide a menu which links the user to those sites on request.

The common interface that all portals must implement is as follows:

/silo/?action=register&url='[url]'&description='[text]'

/silo/?action=unregister&url='[url]'

To indicate success, the request should return with a valid HTTP/1.x response with a 20x code.

To indicate failure, the request should return a valid HTTP/1.x response with a non 20x code.

Any additional information can be optionally returned in the response (text is preferable to html).

Any additional security checks can be carried out to determine if the portal will accept the url. For example, it may only accept urls which are within the intranet, or it may only accept registration requests coming from an intranet ip. There may also be a Portal implementation defined handshaking between the portal and it's transients.

Subsequent requests on the registered urls may or may not be proxied through the portal. Ideally, the portal should know the difference between subsequent intranet requests and extranet requests and proxy as appropriate (see Strategies below).

In other words, the menu presented to an intranet user will consist of urls derived wholly from the registered URLs, while an extranet user will be presented with nothing or a unique publicly accessible URL for each registered intranet system.

Any requests proxied via the Portal (or any processes it starts to do it) must block usage of /silo/* to avoid security leaks.

Transients may register themselves multiple times with a portal without any corresponding unregister requests. In this situation, it is expected that the portal will shed the information stored from the previous registration and repeat the registration process and any handshaking that that implies.

An Advanced Portal

An advanced portal may further collect information from a transient during the registration.

As such, a transient must return an document XML document from the following request:

/silo/?action=entries

The content of the XML must have the following [rough - very rough - even at the point of writing I see flaws here] format:

<SILO>

<HEAD>

<LOGO>[uri to an image]</LOGO>

<PUBLIC>[uri to public entry point]</PUBLIC>

<PRIVATE>[uri to private entry point]</PRIVATE>

</HEAD>

<LOGO>[uri to an image]</LOGO>

<PUBLIC>[uri to public entry point]</PUBLIC>

<PRIVATE>[uri to private entry point]</PRIVATE>

<PARAMETERS>[uri to user defined xml doc]</PARAMETERS>

</LINK>

...

</LINKS>

</SILO>

A portal may or may not be capable of interpreting this information meaningfully. It may not even request it.

A transient can reject a request to supply this information and does not need to implement it.

A portal may expose an accumulative document based on the obtained public information from any of its registered transients to a portal that it registers with (ie: a portal is free to implement its own entries action - hence a Portal can become a Transient).

It should be free to build it's own document based on the basic registration information obtained (ie: from accepted transients which do not have their own entries implementation). It should take care to expose the URL of each transient that is applicable to the portal it is registering with (ie: that all incoming requests are passed through itself when necessary).

Implementation

Overview

This section provides a brief overview of the more tricky concerns regarding implementation. It is the intention that the portal, and any silo related functionality of a transient can be implemented on any web server which supports a bear minimum of CGI scripting functionality.

Minimal Portal

While the implementation of a minimal portal is a simple task, there are issues which need to be redressed regarding the implementation of the proxy functionality and other fine points based on security strategies.

There are a number of strategies which can be applied to provide the proxying functionality.

1) All transient servers must implement an homogeneous web hierarchy with the root being the portals root and all subsequent requests will need to be routed through a script which will proxy the incoming request to the internal server

2) Each registered server is allocated a unique subdomain and proxying is carrying out via the server

3) The server sets up a unique port per registered server and decorates the outward links with those allocated ports

Option 1 is weak since it means that the structure of each internal site must be coordinated to avoid clashes.

Option 2 is a good solution, but it excludes the ability for a non-domain registered network to use it.

Option 3 is possibly the simplest to implement since port redirectors or individual stand-alone proxies are readily available. It supports ip address usage, can potentially support many protocols but, if implemented using standalone applications, it introduces all kinds of wildcards into the system (the requests are handled independently of the portal, hence, it is not possible to ensure a centralised security model or a controlled resource utilisation).

The initial ACP implementation will revolve around the 3rd option (and latterly, the 2nd), but all proxy/redirectors will be handled within the single portal instance. The security implementation and resource usage will be controlled in a single process.

[There maybe other implementation strategies here].

Advanced Portal

An advanced portal may only implement a specific service.

For example, say an access granting Proxying Portal accepts registration requests from 5 transients. 3 of these transients have an entries document which the Proxying Portal picks up. Since it is doing nothing but proxying, it simply accumulates the 3 and adds the basic information of the other 2 to its own data.

When it registers itself with another Portal (it may do this each time a transient is successfully registered or unregistered), the second portal can retrieve the entries document and use specific links in there to provide a specific view of the original portals transients:

it could be an audio stream portal which show all the information regarding audio streams of all its connected transients/portals
it could be a centralised file searching portal which provides the ability to fire a query at all its known transients/portals
it could be a better file searching portal which requests a list of all shared files on each transient/portal and search results are generated entirely on the portal
it could be combination of all 3 of these or many more

Conclusion

The SILO framework has the intention of providing a general, infinitely extendible framework for all http services, though it is also possible to expose non-http services through the same mechanism.

The level of sophistication employed by the portals is implementation defined. The services provided by a transient are at the discretion of a particular vendor - if they wish to adopt and publish an open standard to allow multiple implementations, that is also at their own discretion.

Addendum:

Also note, since this is a first draft of the framework it's probably got and introduces more holes than are intended. The author is by no means an expert on the functionality of general web servers or even the entirety of the http protocol (he knows enough to be able to implement everything in this document using his own web server implementation and possibly in a decent servlet engine [orion for example], and at a push he could possibly write an apache mod to do it, but it is highly unlikely that he ever will... the intention is to provide a basic portal and a few transients which do specific tasks and register themselves with that portal).