Oct 10, 2011

PubSubHubBub and ColdFusion

I came across a publisher\subscriber protocol called PubSubHubBub. It is a server-to-server web-hook-based pubsub (publish/subscribe) protocol - an extension to Atom and RSS. Here the parties (servers) get an instant notification when a feed URL, that  they are interested in is updated.

Traditionally a subscriber would subscribe to a feed and poll for it at regular intervals, to see if there is an updated feed available. In this protocol rather than polling for a feed, the content is pushed out from the publisher. The theory here is that the subscriber can subscribe to a feed via a 'Hub', which then would inform the subscribers when the feed is updated.

How PubSubHubBub works?
  • A Publisher instead of sending an update to every subscriber, it includes a in its feed URL and sends an update to the Hub.
  • A Subscriber sends a subscription request to the Hub with the feed URL that it is interested in. The request also contains a callback URL to which the Hub should send an update.
  • To verify the subscription request, the Hub sends a GET request to subscribers' callback URL. The
    Subscriber then verifies itself by responding to the request.
  • When the Publisher posts new content, it notifies the Hub of the updates by sending a ping notification (POST request).
  • Hub on receiving a notification from the Publisher, fetches the new content and then POSTs an update to the Subscribers callback URL.
If the feed has multiple subscribers, then the Hub would send an update to each of these subscribers. The Subscription flow posted on the PubSubHubBub site is here:

Where is the Hub?

Anybody can run a hub, it is not owned by any company. The protocol is decentralized and free. A couple of implementations include pubsubhubbub.appspot.com and superfeedr.com


Whenever a publisher adds new content (a new blog post), the subscribers are to be notified of the updated feed. A notification (POST request) is sent to the Hub by the Publisher. The Content-Type header must be set to application/x-www-form-urlencoded and the request should contain the parameters 'hub.mode' and 'hub.url' in the request body. The parameter hub.mode is set to 'publish' and the hub.url is set to topic URL that is updated.

The Hub then accepts the POST request and fetches the new content by sending a GET request to the topic URL. After fetching the content the Hub determines whether the feed has changed. The Hub would then send this information about the changes to each subscriber.


The Subscriber initiates its subscription by sending a POST request to the Hub URL. The Content-Type header must be set to application/x-www-form-urlencoded and it should contain the following parameters in the request body:
  • hub.mode - set to 'subscribe'
  • hub.callback - subscribers callback URL where the notifications should be delivered by the Hub.
  • hub.topic - the topic URL to which the subscriber wishes to subscribe
  • hub.verify - it can be either sync/async. 
    • sync when the verification request must occur before the subscription request's HTTP response is returned.
    • async when the verification request may occur at a later point after the subscription request has returned.

Verifying the subscription:

The Hub on receiving a subscription request verifies the same by sending a GET request to the subscriber's callback URL. The request contains the following query strings:
  • hub.mode - subscribe or unsubscribe depending on what was set in the original request.
  • hub.topic - the topic URL given in the subscription request.
  • hub.challenge - a random string.  
The subscriber's callback URL should respond to this request by echoing the hub.challenge string in response.


Registering the Publisher with the Hub:

The Publisher and Subscriber can be registered at pubsubhubbub.appspot.com and it is a one time activity. However, one can do the same programmatically:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="pub_result> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp> <cfif #pub_result.Responseheader.Status_Code# eq 204> Publisher registered successfully <cfelse> Publisher failed to register returned #pub_result.Responseheader.Status_Code# </cfif>

The Hub responds with a '204 No content' if the publishers feed is successfully registered.

Publishing content:

A form accepting the post title and description:
<cfparam name="FORM.feedTitle" default="" > <cfparam name="FORM.feedDescription" default="" > <cfif FORM.feedTitle NEQ "" AND FORM.feedDescription NEQ ""> <cfset publisherObject = createObject("component","Publisher") > <cfset publisherObject.publish_update(FORM.feedTitle,FORM.feedDescription) > </cfif> <form action="publisher_form.cfm" method="post"> Add a new post here:
Title: <input name="feedTitle" type="text" style="width:300px">
<textarea name="feedDescription" rows="5" cols="50" ></textarea>
<input type="submit" value="Submit"> </form>

A ping notification is sent to the Hub, to notify of the new content. The function publish_update does that:
<cffunction name="publish_update" access="remote" returntype="String"> <cfargument name="feedTitle" required="true"/> <cfargument name="feedDescription" required="true"/> <cfscript> if(!isDefined("application.feedQuery")) { application.feedQuery = queryNew("title,description,pubdate"); } queryAddRow(application.feedQuery, 1); querySetCell(application.feedQuery, "title", '#arguments.feedTitle#'); querySetCell(application.feedQuery, "description", '#arguments.feedDescription#'); querySetCell(application.feedQuery, "pubdate", '#DATEFORMAT(now(), "mm/dd/yyyy")#'); </cfscript> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="new_content"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish"> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp>

The Hub then fetches the feed from the mentioned URL (hub.url). Here it is the getFeed method in Publisher.cfc:

<cffunction name="getFeed" access="remote"> <cfscript> /* Feed metadata structure */ feedMetaData = structNew(); feedMetaData.author = arrayNew(1); //arrayAppend(feedMetaData.author,{name:'Sagar Ganatra',email:'sagar@sagarganatra.com'}); feedMetaData.encoding = "UTF-8"; feedMetaData.link = arrayNew(1); //arrayAppend(feedMetaData.link,{href:'http://pubsubhubbub.appspot.com',rel:'rel'}); feedMetaData.title = structNew(); feedMetaData.title.type = "text"; feedMetaData.title.value = "Sagar's Kitchen"; feedMetaData.updated = "#now()#"; </cfscript> <cfset colMap = {publishedDate="pubdate", title="title", content="description"}> <cffeed action="create" query="#application.feedQuery#" properties="#feedMetaData#" columnmap="#colMap#" xmlvar="xmlResult"> <cfcontent type="text/xml" reset="true"> <cfoutput>#xmlResult#</cfoutput> </cffunction>

Registering a Subscriber with the Hub:

The Subscriber can subscribe to the feed by specifying the topic URL in the request body:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="sub_result"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="subscribe"> <cfhttpparam type="url" name="hub.verify" value="sync"> <cfhttpparam type="url" name="hub.topic" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> <cfhttpparam type="url" name="hub.callback" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/subscriber.cfc?method=receiveFeed> </cfhttp> <cfif #sub_result.Responseheader.Status_Code# eq 204> Subscriber registered successfully <cfelse> Subscriber failed to register. Returned <cfoutput>#sub_result.Responseheader.Status_Code#</cfoutput> </cfif>


As mentioned earlier, Hub checks the Subscribers' intent by sending a GET request to the callback URL. The Subscriber is then required to respond, by echoing back the 'hub.challenge' parameter present in the URL:
<cffunction name="receiveFeed" access="remote" returnformat="plain" returntype="string" output="false"> <cfheader name="Content-Type" value="text/plain"> <cfif structKeyExists(URL, "hub.challenge")> <cflog text="#url['hub.challenge']#"> <cfreturn #url['hub.challenge']> <cfelse> <cfset feeddata="getHTTPRequestData().content"> <cfset feeddata="feeddata" feedsource.xml="feedsource.xml" filewrite="filewrite" ram:="ram:" trim="trim"> <cffeed action="read" query="feedReceived" source="ram:///feedSource.xml"> <cfdump output="console" var="#feedReceived#"> <cfheader statuscode="200"> <cfreturn> </cfif> </cffunction>
As seen in the above code the feed is read in else part. One can store the read content in a database for later retrieval or send the same over a websocket to the clients so that it can be read live.

Who is using PubSubHubBub?

Google products - Google Alerts, FeedBurner, Blogger and Google Reader are using PubSubHubBub.


Project Home - http://code.google.com/p/pubsubhubbub/
Protocol Basics - http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html


  1. Thanks for this post. It's been helpful to me in learning how to implement the PubSubHubbub protocol in ColdFusion.

    FYI: I think there is an error in line 11 of your getFeed() method. The link to the hub server should have rel="hub" but your code uses rel="rel".

  2. Thanks Richard,

    I've updated the blog post now.

  3. Thanks a lot for the useful and detailed post, I am currently looking for a publisher for myself and consider sticking to this one. Seems to be a little complicated at first sight, but I am sure I will fiagure all the features out with such helpful tips.