Bolinfest Changeblog

It turns out the iGoogleBar needed a one-line fix to address what I can only assume was a recent change in Gmail. I updated the code in the Chickenfoot script wiki as well. Version 0.5.2 is now the latest version, so go to Tools -> Add-ons in Firefox to see if you have the latest version — hit the Find Updates button if you don’t!

iGoogleBar is my latest Firefox extension that adds Google Apps favicons to the Google Apps Bar, using them as triggers for the Apps’ respective iGoogle Gadgets. As a courtesy to some of my colleagues, I included their projects in the bar so they’re easier to get to (or just preview) from Gmail and Calendar.

I built this extension using Chickenfoot, which hit version 1.0 recently. Part of that release included improvements to the extension-packaging tool, which made it much easier for me to convert my iGoogleBar prototype into a full-fledged Firefox extension!

The iGoogleBar page acknowledges that there are some missing features from the extension, so I put the source code in the Chickenfoot Scripts Wiki. That means, if you’d like to see a new feature added, please go and update the Wiki instead of just nagging me :) If I like your patch, then I’ll make a new release. And if I don’t like your patch, then you can at least install your modified version of iGoogleBar as a trigger in Chickenfoot.

Will this method of software development actually work? Probably not, but it should be fun to try!

I noticed that when I visit my Twitter page, I see some tweets on twitter.com that do not show up when I look at my Twitter feed in Google Reader. I checked to make sure that the missing tweets were in Twitter’s feed by loading it in my browser, and they were, but only when I was logged in to Twitter.

The “missing” tweets belonged to my friends who make their tweets protected, which is why I can only see them when I’m authenticated by Twitter. Like most things on the web, authentication is simply the matter of providing the appropriate cookie with an HTTP request, so I decided to write a little script to get my authenitcated Twitter feed. To do that, first I needed to get my Twitter cookie, which you can get by entering this in the address bar when logged in on any twitter.com URL:

javascript:void(document.body.innerHTML = document.cookie.match(/auth_token=(.*);/)[1])

That should replace the text of the page with your Twitter cookie, making it easy to copy and paste in the script below:

wget --quiet --cookies=off --header "Cookie: auth_token=COOKIE" TWITTER_FEED_URL -O SECRET_OUTPUT_FILE

• COOKIE should be the Twitter cookie that you just copied
• TWITTER_FEED_URL should be the URL to your Twitter RSS feed
• SECRET_OUTPUT_FILE should be a file on your Linux machine that is served publicly, but has a long, unguessable URL that people can’t stumble upon and use to read your friends’ secret tweets! Using /usr/bin/uuidgen may be a good way to come up with a filename (though remember to add an .xml suffix to the filename to ensure your browser serves the URL with the correct MIME type). This URL is what you will add to Google Reader.

Now that you have your one-liner, you’ll want to put it in a cronjob so you will periodically update your local file with the contents of your full Twitter feed. I just made a shell script with the command above (remember to chmod 700 your shell script and to chmod 640 your output file!), followed by this command to ping Google Blog Search so that Google will know about your updated feed:

curl --silent http://blogsearch.google.com/ping?name=bolinfest+and+friends&url=YOUR_SECRET_URL > /dev/null

Once your file is written, add YOUR_SECRET_URL to Google Reader, and you’re all set!

Because I haven’t posted in awhile, you might think that I haven’t been hacking, but I assure you that is not the case! Between the Facebook Platform and the iPhone, there are many new opportunities to explore lately. Nevertheless, my most recent (published) work is still related to my favorite product!

Last week, I posted about the introduction of Calendar Gadgets on the Google Code Blog. Although most people are probably most excited about the new horoscope and sudoku calendars, my biggest contribution is my updated xkcd calendar:

Normally, if you looked at Friday’s xkcd comic in Firefox and moused over it, you would see Could be worse. The last guy in that situation fell for one of the transient trumpeting a… because Firefox 2 clips the title attribute of the image at 80 characters. After great debate on the subject, the 80-character limit is going to be fixed in Firefox 3, but until then, I created a Google Gadget that displays an image with its full tooltip (you can optionally hyperlink your image as well) that I use to host the content of the xkcd calendar. (This was inspired by Mihai Parparita who fixed this for Google Reader.)

Also, since school is back in session, Mike Lambert and I updated mitcalendars.com with links to all of the Fall 2007 MIT course schedules hosted on wikicalendars.com. Sadly, wikicalendars.com has not taken off as we had hoped (it does get a lot of spam, though!), but if we ever have time to make it as easy to add events to wikicalendars.com as it is to add events to Google Calendar, then I think we’ll do a lot better.

Oh! And unfortunately I missed Google Developer Day this year because I was too busy working on the Google Calendar gallery, so I didn’t have a chance to comment on mapplets, which are really cool! With a few PHP includes, I was able to turn CalMap into a mapplet. I’m sure you can do the same with your own Maps mashups, so give it a try!

Facebook recently announced that it has dramatically expanded its API to allow deep integration into facebook.com, aptly naming the project Facebook Platform. The potential integration points for a Facebook Application look very appealing, but developing for the Platform itself still seems a bit rough around the edges.

To start, I just wanted to put my own IFRAME inside Facebook. (As you may have noticed, I like sticking IFRAMEs where they don’t belong.) I didn’t feel like watching any of the videos to learn about the platform because I preferred to do textual searches. Unfortunately, the search box on http://www.facebook.com/developers/ searches Facebook profiles, not developer docs, so I’m not sure whether this is possible on Facebook. (Using site:developers.facebook.com on Google seems to work, though.)

On the Platform Wiki, I found a link to an external tutorial on creating apps with canvas pages which seemed promising, but it also addressed authentication issues and required me to download some PHP code, neither of which I was particularly interested in (this seemed like overkill just to embed an IFRAME), so I moved on.

I tried to start configuring my application directly, so I added the Developer Application like Facebook told me to, created a new Facebook app, and started editing its settings. You’ll likely discover that the textboxes for URLs are way too small, but you can fix that with one line of Chickenfoot:

for each (var t in find('textbox')) t.element.size = 80;

Now that that was out of the way, I tried to wrap my head around the difference between a URL for my web site and a Facebook canvas URL. Apparently, you get some namespace under http://apps.facebook.com/example.com/ which maps to real URLs on your own domain, such as http://example.com/facebook/. The developer has to keep track of this mapping himself, so when asked for something such as the Side Nav URL, you need to supply the apps.facebook URL instead of the one from your own domain. Why isn’t this mapping done automatically? Why can’t I use the URLs that I already know instead of the ones that Facebook invented for me?

There’s also something strange about the role of your Callback URL (or is it “Url?”). I tried to use the authentication overview link to learn about it, but that page (http://mit.facebook.com/developers/documentation.php?doc=auth) gave me a 404.

Fortunately, there is some information on the wiki about the callback page — it seems that is serves both as your login handler as well as your home page, which I suppose works out all right if you’re authenticating users’ Facebook tokens, but if you just want to show a static page, there isn’t much point.

Finally, Facebook appears to be finicky about using explicit URLs (perhaps it’s a security thing?), so using http://example.com/facebook/ as your Side Nav URL is automatically converted to http://example.com/facebook/index.html even if your Apache DirectoryIndex directive points to something else. For a site who probably has its DirectoryIndex set to index.php, I find this a bit surprising.

As you can imagine, I spent quite a bit of time playing around with the Edit page, as I imagine most new developers do. It would be really nice if the Save button were an Apply button instead that applied the changes without reloading the page since there’s a good chance that the URL that you just entered was the wrong one, so you’re going to have to return to the Edit page again, anyway.

In the coming weeks, I’m sure we’ll see a number of these issues get cleaned up (as well as adding *cough* search *cough* to the Application Directory) — you never really know what bugs your API has until developers start using it. I trust that the Facebook developers were working very hard to get the platform ready for F8, but their job is certainly not done — no rest for the weary, kids!

Some of you have probably seen the experiment on google.com that brings Google apps closer to the home page. But if you find that they still aren’t close enough for you, then I have a new gadget for iGoogle (which is a much better name than Google Personalized Homepage) that might help you out. My gadget is called Your Page Here because it lets you use any page you like as the content for one of your iGoogle tabs:

Note that if you add this gadget to a tab, it eclipses all of the other gadgets on the page, so you probably want to create a new tab for Your Page Here before installing it. Once you have it on your page, the settings are pretty simple:

All you have to do is enter a URL and you’re ready to go. Come on! Try it for yourself: 

And if Calendar isn’t your thing, then I recommend trying out Google Reader (http://reader.google.com/) with Your Page Here.

Thanks to Chris McAndrew for suggesting this hack to iGoogle.

Not too long ago, I imported MLB schedule data into WikiCalendars.com. This made it simple to see when the Mets were playing next, but not where they would be playing next, so I decided to create CalMap: a mashup that takes a calendar feed and plots the locations of the events on a map:

Currently, CalMap only plots the locations it knows, which are listed in a JSON data file. I also use this to assign marker colors so baseball teams in the same division will have the same color — this makes the interleague games easy to pick out on the map!

If you inspect the default URL for CalMap, you will see that the URL to the calendar feed that you want to load is simply passed as a GET parameter:

http://calmap.bolinfest.com/?url=http://www.wikicalendars.com/wiki/MLB_2007_nym_protected

For WikiCalendars, you can also splice a number of calendars together and plot them as one calendar. For example, you can combine the calendars for all of the NL East teams and display them as one on CalMap:

http://calmap.bolinfest.com/?url=http://www.wikicalendars.com/splice.php?cal=MLB_2007_nym_protected,MLB_2007_phi_protected,MLB_2007_atl_protected,MLB_2007_fla_protected,MLB_2007_was_protected

The supplied calendar feed needs to follow the GData JSON conventions, so it needs to honor alt=json-in-script, start-min, start-max, etc. This means that, yes, CalMap may execute arbitrary JavaScript from foreign sites, which is why I put it on its own subdomain of bolinfest.com where there are no cookies to steal.

Also, so that my demo does not get trashed, I created copies of the original MLB calendars and made versions of them that only I can edit so that the CalMap home page does not become spammy. I have written a Chickenfoot script to update these new versions of the calendars with data from mlb.com, so if you added one of the old MLB calendars to your Google Calendar account, I recommend replacing it with one from the list of protected calendars as the new calendars will be updated fairly frequently with legitimate data. (You can also subscribe to one of my “protected” MLB calendars by following the Subscribe link at the bottom of CalMap.)

The only thing left to do is to figure out how to run Chickenfoot reliably as a cronjob…

About a month ago, I spent an hour on the phone screaming (yes, literally screaming) at Dell because they had sold me a machine with Windows Vista. Admittedly, I knew that it would have Vista because when I ordered it online, I was not given the option to select XP. Fortunately for me, Dell recently reversed its decision to impose Vista on the world, so as soon as I saw the post on Slashdot, I got on the phone with Dell and demanded XP again. (After the previous conversation with them, I was planning to return the machine altogether.) This time, they acceded.

You may be wondering why I wanted Windows so badly, and in particular, why Windows XP? A couple of years ago, I bought a Dell touchscreen (model E153FPT) in hopes of building my own little kiosk. It would have been ideal to hook it up to my Mac Mini, but I could only find drivers for Windows. Also, there are still a few things that I use that only work on Windows, such as X10 ActiveHome and the Google Talk client, both of which I would like to use on my kiosk.

When I ordered the machine, I thought it might be fun to try out Vista, but instead it was a nightmare. The new UI paradigm where they hide the menubar by default drove me nuts (as I expected it would), and the popup situation is just as bad as the I’m a Mac “Security” claims it is. But the real deal breaker was when I discovered that there were no drivers for the E153FPT – I got on the phone and was furious that Dell would force me to buy a new machine with an operating system that was incompatible with other hardware I bought from Dell. The first time I called, the best they would do was sell me XP for $120 instead of the retail price of $200. I found this pretty outrageous as the machine I bought cost less than $500 in the first place.

I’m happy that the problem is resolved, but after the second phone call, I had spent a total of two hours on the phone and had talked to at least seven people, all because I wasn’t offered the option to select XP with my new machine on their web site in the first place.

After my recent post requesting information about a Firefox-friendly Jabber library, it dawned on me that I should try searching for firefox xmpp library instead of firefox jabber library. Lo and behold, I discovered xmpp4moz! From the project web site, I downloaded the SamePlace Suite Firefox extension, which is a small suite of applications built on top of xmpp4moz. I fired it up and discovered that it contained a nice Jabber client written in XUL with explicit support for Google Talk. The only remaining question was: how did it work?

I checked out the xmpp4moz code from the repository and started looking through it to figure out how it made a secure connection to talk.google.com. I immediately found some code that mentioned the starttls exchange that had thwarted my previous efforts to build a Jabber library for Firefox, but unfortunately it was commented out! After shooting in the dark for awhile with grep, I decided to find the entry point to the sign-on code and step through it.

Two important differences from the Smack library were (1) xmpp4moz connected on port 5223 instead of port 5222, and (2) the initial stanza that it sent to the server included my username and password:

<iq to={JID(jid).hostname} type="set">
  <query xmlns="jabber:iq:auth">
    <username>{JID(jid).username}</username>
    <password>{password}</password>
    <resource>{JID(jid).resource}</resource>
  </query>
</iq>

I found this odd, as this was not what was described in the jabberd 2.0 protocol document I had discovered earlier. I searched on Google to find out what the difference between using ports 5222 and 5223 was, and found out that port 5222 offers TLS support while port 5223 offers SSL support. So by using port 5223, I could connect to Google Talk by using an SSL transport in Firefox, created as follows:

const srvSocketTransport = Components.classes["@mozilla.org/network/socket-transport-service;1"]
    .getService(Components.interfaces.nsISocketTransportService);
var transport = srvSocketTransport.createTransport(['ssl'], 1, "talk.google.com", 5223, null);

From what I have seen of the code so far, xmpp4moz looks to be an organized, well-designed library. I noticed that the developers make generous use of E4X, which is both convenient and brilliant since XMPP messages are in XML: all the more reason to develop a Jabber client within Firefox!

But wait! Didn’t I just say that the SamePlace suite already has a nice Jabber client for Firefox? Yes, it certainly does, but I want to explore building an HTML client with some of my own features, so leveraging xmpp4moz looks like the best way to do it!

For quite some time, I have wanted to have a Jabber library in Firefox that I could build extensions around. For my experiments with Google Talk on the desktop, I have been using the Smack API. Smack has served me well, so I decided to try to use it as a model for an equivalent library written in JavaScript for Firefox, leveraging XPCOM. It turns out that Smack has a kickass debug mode that shows all of the XML messages that are sent back and forth while communicating with the server. Below, you can see the messages that Smack and Google Talk exchange when initiating a connection (client messages are in red, server messages are in blue):

<stream:stream to="gmail.com" xmlns="jabber:client" xmlns:stream="http://etherx.jabber.org/streams" version="1.0">

<stream:stream from="gmail.com" id="ABAC7E59A46C522E" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
<stream:features>
  <starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls">
    <required/>
  </starttls>
  <mechanisms xmlns="urn:ietf:params:xml:ns:xmpp-sasl">
    <mechanism>X-GOOGLE-TOKEN</mechanism>
  </mechanisms>
</stream:features>

<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>

<proceed xmlns="urn:ietf:params:xml:ns:xmpp-tls"/>

By following a XULPlanet tutorial on sockets, I successfully duplicated the above exchange in Firefox using trusted JavaScript. Unfortunately, then I hit a snag when I tried to replicate the following Java code in JavaScript:

/**
 * proceedTLSReceived() in org.jivesoftware.smack.XMPPConnection
 */
void proceedTLSReceived() throws Exception {
  SSLContext context = SSLContext.getInstance(”TLS”);
  // Verify certificate presented by the server
  context.init(null, // KeyManager not required
      new javax.net.ssl.TrustManager[]{new ServerTrustManager(serviceName, configuration)},
      new java.security.SecureRandom());
  Socket plain = socket;
  // Secure the plain connection
  socket = context.getSocketFactory().createSocket(plain,
      plain.getInetAddress().getHostName(), plain.getPort(), true);
  socket.setSoTimeout(0);
  socket.setKeepAlive(true);
  // Initialize the reader and writer with the new secured version
  initReaderAndWriter();
  // Proceed to do the handshake
  ((SSLSocket) socket).startHandshake();
	
  // Set that TLS was successful
  usingTLS = true;
	
  // Set the new  writer to use
  packetWriter.setWriter(writer);
  // Send a new opening stream to the server
  packetWriter.openStream();
}

As you can see, this requires some logic to deal with security. Although it appears that Firefox has code to handle SSL and TLS, I haven’t figured out a way to access it from JavaScript. One option would be to write my own XPCOM object in C++, which is what I suspect Process-one did for their XUL-based XMPP client. Unfortunately, I am not that comfortable with C++, nor am I interested in the overhead involved in compiling for multiple platforms, which is really why I was looking for an existing, scriptable XPCOM object to do the heavy-lifting for me in the first place.

Another possibility is writing an XPCOM object in a language other than C++. I started to explore PyXPCOM for writing modules in Python, but got discouraged when I discovered that Brendan Eich himself commented that PyXPCOM does not make it practical to write Firefox extensions in Python because it requires bundling Python with the extension. I’m guessing that the XPCOM language bindings for Perl, Ruby, and Java have similar limitations.

Then again, what about Java? Unlike Perl and Ruby, there is a standard Java plugin for Firefox, so maybe using JavaXPCOM is a possibility. And even if not, LiveConnect might do the trick, since we have successfully used it for Java-JavaScript communication in Chickenfoot. Being able to leverage Smack as-is would certainly be a windfall if it worked!

Though I am likely going to pursue the Java path to make this happen, I think it would be great if someone with the right know-how created a C++ XPCOM object that exposed a scriptable interface that would let you connect to a Jabber server (using TLS) and provide access to input and output streams for that socket. If anyone knows his way around the GAIM code, he or she could likely leverage what is already there to make this happen. I expect the “plumbing” involved in bringing in the existing C++ code, creating the IDL, and compiling for the major platforms, is likely where more than 80% of the work is – the new code that needs to be written should be fairly minimal.

For the end-user, this would provide the best experience since he would be able to install a Firefox extension without having to install the Java plugin on top of it. Also, the C++ XPCOM object would likely be faster, because in developing Chickenfoot, I learned that crossing the Java/C++ process boundary can be expensive. (This is described in section 8.1.1 of my thesis.) So if anyone is interested in taking on this project, or if you know of an existing solution, then please let me know!