Skip to main content

PubSubHubBub and ColdFusion

I came across a publisher\subscriber protocol called PubSubHubBub. It is a server-to-server web-hook-based pubsub (publish/subscribe) protocol - an extension to Atom and RSS. Here the parties (servers) get an instant notification when a feed URL, that  they are interested in is updated.

Traditionally a subscriber would subscribe to a feed and poll for it at regular intervals, to see if there is an updated feed available. In this protocol rather than polling for a feed, the content is pushed out from the publisher. The theory here is that the subscriber can subscribe to a feed via a 'Hub', which then would inform the subscribers when the feed is updated.

How PubSubHubBub works?
  • A Publisher instead of sending an update to every subscriber, it includes a in its feed URL and sends an update to the Hub.
  • A Subscriber sends a subscription request to the Hub with the feed URL that it is interested in. The request also contains a callback URL to which the Hub should send an update.
  • To verify the subscription request, the Hub sends a GET request to subscribers' callback URL. The
    Subscriber then verifies itself by responding to the request.
  • When the Publisher posts new content, it notifies the Hub of the updates by sending a ping notification (POST request).
  • Hub on receiving a notification from the Publisher, fetches the new content and then POSTs an update to the Subscribers callback URL.
If the feed has multiple subscribers, then the Hub would send an update to each of these subscribers. The Subscription flow posted on the PubSubHubBub site is here:



Where is the Hub?

Anybody can run a hub, it is not owned by any company. The protocol is decentralized and free. A couple of implementations include pubsubhubbub.appspot.com and superfeedr.com

Publishing

Whenever a publisher adds new content (a new blog post), the subscribers are to be notified of the updated feed. A notification (POST request) is sent to the Hub by the Publisher. The Content-Type header must be set to application/x-www-form-urlencoded and the request should contain the parameters 'hub.mode' and 'hub.url' in the request body. The parameter hub.mode is set to 'publish' and the hub.url is set to topic URL that is updated.

The Hub then accepts the POST request and fetches the new content by sending a GET request to the topic URL. After fetching the content the Hub determines whether the feed has changed. The Hub would then send this information about the changes to each subscriber.

Subscribing

The Subscriber initiates its subscription by sending a POST request to the Hub URL. The Content-Type header must be set to application/x-www-form-urlencoded and it should contain the following parameters in the request body:
  • hub.mode - set to 'subscribe'
  • hub.callback - subscribers callback URL where the notifications should be delivered by the Hub.
  • hub.topic - the topic URL to which the subscriber wishes to subscribe
  • hub.verify - it can be either sync/async. 
    • sync when the verification request must occur before the subscription request's HTTP response is returned.
    • async when the verification request may occur at a later point after the subscription request has returned.

Verifying the subscription:

The Hub on receiving a subscription request verifies the same by sending a GET request to the subscriber's callback URL. The request contains the following query strings:
  • hub.mode - subscribe or unsubscribe depending on what was set in the original request.
  • hub.topic - the topic URL given in the subscription request.
  • hub.challenge - a random string.  
The subscriber's callback URL should respond to this request by echoing the hub.challenge string in response.

Code:

Registering the Publisher with the Hub:

The Publisher and Subscriber can be registered at pubsubhubbub.appspot.com and it is a one time activity. However, one can do the same programmatically:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="pub_result> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp> <cfif #pub_result.Responseheader.Status_Code# eq 204> Publisher registered successfully <cfelse> Publisher failed to register returned #pub_result.Responseheader.Status_Code# </cfif>

The Hub responds with a '204 No content' if the publishers feed is successfully registered.

Publishing content:

A form accepting the post title and description:
publisher_form.cfm:
<cfparam name="FORM.feedTitle" default="" > <cfparam name="FORM.feedDescription" default="" > <cfif FORM.feedTitle NEQ "" AND FORM.feedDescription NEQ ""> <cfset publisherObject = createObject("component","Publisher") > <cfset publisherObject.publish_update(FORM.feedTitle,FORM.feedDescription) > </cfif> <form action="publisher_form.cfm" method="post"> Add a new post here:
Title: <input name="feedTitle" type="text" style="width:300px">
Description:
<textarea name="feedDescription" rows="5" cols="50" ></textarea>
<input type="submit" value="Submit"> </form>

A ping notification is sent to the Hub, to notify of the new content. The function publish_update does that:
<cffunction name="publish_update" access="remote" returntype="String"> <cfargument name="feedTitle" required="true"/> <cfargument name="feedDescription" required="true"/> <cfscript> if(!isDefined("application.feedQuery")) { application.feedQuery = queryNew("title,description,pubdate"); } queryAddRow(application.feedQuery, 1); querySetCell(application.feedQuery, "title", '#arguments.feedTitle#'); querySetCell(application.feedQuery, "description", '#arguments.feedDescription#'); querySetCell(application.feedQuery, "pubdate", '#DATEFORMAT(now(), "mm/dd/yyyy")#'); </cfscript> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="new_content"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="publish"> <cfhttpparam type="url" name="hub.url" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> </cfhttp>

The Hub then fetches the feed from the mentioned URL (hub.url). Here it is the getFeed method in Publisher.cfc:

<cffunction name="getFeed" access="remote"> <cfscript> /* Feed metadata structure */ feedMetaData = structNew(); feedMetaData.author = arrayNew(1); //arrayAppend(feedMetaData.author,{name:'Sagar Ganatra',email:'sagar@sagarganatra.com'}); feedMetaData.encoding = "UTF-8"; feedMetaData.link = arrayNew(1); //arrayAppend(feedMetaData.link,{href:'http://pubsubhubbub.appspot.com',rel:'rel'}); feedMetaData.title = structNew(); feedMetaData.title.type = "text"; feedMetaData.title.value = "Sagar's Kitchen"; feedMetaData.updated = "#now()#"; </cfscript> <cfset colMap = {publishedDate="pubdate", title="title", content="description"}> <cffeed action="create" query="#application.feedQuery#" properties="#feedMetaData#" columnmap="#colMap#" xmlvar="xmlResult"> <cfcontent type="text/xml" reset="true"> <cfoutput>#xmlResult#</cfoutput> </cffunction>

Registering a Subscriber with the Hub:

The Subscriber can subscribe to the feed by specifying the topic URL in the request body:

<cfset machineIP = CreateObject("java", "java.net.InetAddress").getLocalHost().getHostAddress()> <cfhttp method="POST" url="http://pubsubhubbub.appspot.com" result="sub_result"> <cfhttpparam type="header" name="Content-Type" value="application/x-www-form-urlencoded"> <cfhttpparam type="url" name="hub.mode" value="subscribe"> <cfhttpparam type="url" name="hub.verify" value="sync"> <cfhttpparam type="url" name="hub.topic" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/Publisher.cfc?method=getFeed"> <cfhttpparam type="url" name="hub.callback" value="http://#machineIP#/ZeusTutorial/pubsubhubbub/example/subscriber.cfc?method=receiveFeed> </cfhttp> <cfif #sub_result.Responseheader.Status_Code# eq 204> Subscriber registered successfully <cfelse> Subscriber failed to register. Returned <cfoutput>#sub_result.Responseheader.Status_Code#</cfoutput> </cfif>

Subscribing

As mentioned earlier, Hub checks the Subscribers' intent by sending a GET request to the callback URL. The Subscriber is then required to respond, by echoing back the 'hub.challenge' parameter present in the URL:
<cffunction name="receiveFeed" access="remote" returnformat="plain" returntype="string" output="false"> <cfheader name="Content-Type" value="text/plain"> <cfif structKeyExists(URL, "hub.challenge")> <cflog text="#url['hub.challenge']#"> <cfreturn #url['hub.challenge']> <cfelse> <cfset feeddata="getHTTPRequestData().content"> <cfset feeddata="feeddata" feedsource.xml="feedsource.xml" filewrite="filewrite" ram:="ram:" trim="trim"> <cffeed action="read" query="feedReceived" source="ram:///feedSource.xml"> <cfdump output="console" var="#feedReceived#"> <cfheader statuscode="200"> <cfreturn> </cfif> </cffunction>
As seen in the above code the feed is read in else part. One can store the read content in a database for later retrieval or send the same over a websocket to the clients so that it can be read live.

Who is using PubSubHubBub?

Google products - Google Alerts, FeedBurner, Blogger and Google Reader are using PubSubHubBub.

Reference:

Project Home - http://code.google.com/p/pubsubhubbub/
Protocol Basics - http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html

Comments

  1. Thanks for this post. It's been helpful to me in learning how to implement the PubSubHubbub protocol in ColdFusion.

    FYI: I think there is an error in line 11 of your getFeed() method. The link to the hub server should have rel="hub" but your code uses rel="rel".

    ReplyDelete
  2. Thanks Richard,

    I've updated the blog post now.

    ReplyDelete
  3. Thanks a lot for the useful and detailed post, I am currently looking for a publisher for myself and consider sticking to this one. Seems to be a little complicated at first sight, but I am sure I will fiagure all the features out with such helpful tips.

     

    ReplyDelete

Post a Comment

Popular posts from this blog

Custom validation messages for HTML5 Input elements using the constraint validation API

HTML5 has introduced several input types such as EMAIL, URL, RANGE, SEARCH, DATE, TIME, etc,. Most of the modern browsers have implemented them and are ready to be used in a HTML document. Another exciting feature introduced in HTML5 is the form validation. Instead of writing JavaScript to validate users input, browsers can now validate it and show an appropriate message if the validation fails. The validation message is shown in line with the field for which the validation has failed. The default error message is shown when the validation fails. In this post I'll explain how these error messages can be changed.

Adding beforeRender and afterRender functions to a Backbone View

I was working on a Backbone application that updated the DOM when a response was received from the server. In a Backbone View, the initialize method would perform some operations and then call the render method to update the view. This worked fine, however there was scenario where in I wanted to perform some tasks before and after rendering the view. This can be considered as firing an event before and after the function had completed its execution. I found a very simple way to do this with Underscore's wrap method.

A cheat sheet of keyboard shortcuts in ColdFusion Builder

In my last post I have explained about keyboard shortcuts in ColdFusion Builder 2.0. This blog post contains a list of all shortcut keys and I have listed these keyboard shortcuts based on the categories that it falls into. Well, this post was not planned since users can easily get to know the keyboard shortcuts by navigating to the preferences (ColdFusion -> Profiles -> Keys). However, I met Joshua at Scotch on the rocks in Edinburgh and he suggested that it would be nice to have the list of keyboard shortcuts handy. So this post is for those who would like to have the list with them and refer it whenever required.


Keyboard shortcuts for inserting text:

These are the keyboard shortcuts which are used to insert some text into the editor:

CommandKeyboard shortcuts on WindowsKeyboard shortcuts on MacInsert anchor tagCTRL + T, LCMD + T, LInsert bold tagCTRL + T, BCMD + T, BInsert br tagCTRL + T, RCMD + T, RInsert cfabortCTRL + T, ACMD + T, AInsert cfdumpCTRL + T, DCMD + T, DInsert cfs…