Skip to content

Outfoxing Gmail with Greasemonkey

Can you remember a time when the title of this blog post might have landed me in a straight jacket? Can you believe that was just a few short years ago? Yea, I can’t either.

Anyway, Gary’s post Outfox: speech, sound, and more for Firefox talks about a new Firefox extension. He’s using it to create cross-platform, self-voicing Web apps for kids with disabilities using a pure JS API. He hopes to extend his work to support alternative input devices such as game pads and switches as the Outfox extension matures and grows more flexible.

One of the other potential uses listed on the Outfox homepage is Adding new I/O to web sites with Greasemonkey. Interesting. It’s one thing to include Outfox explicitly in a page, but can it possibly work when injected by GM? What about for a complex app like Gmail with multiple iframes, dynamic changes, refreshing, etc.?

To learn about Outfox (and for fun), I decided to write a quick GM script for Gmail that announces the senders and times of new messages (bold items) in the inbox. (I would have done subject and summary too, but Outfox 0.1.0 appears to have some unicode issues and balked at some of the Gmail separator characters. Less is more at this point.) The script makes the announcement when the Gmail interface first loads, any time Gmail automatically refreshes its inbox view, or when the user clicks the refresh link to check for new mail. It is smart enough to announce a given message only once, however, so you don’t hear the same message over and over again on each refresh.

Yes. It does actually work.

To try this script, make sure you have the Greasemonkey 0.8 and Outfox 0.1 extensions installed on Firefox 3. (Or use the latest available version of each.) Then visit the following link to have GM install the script: gmail_announcer.user.js.

For reference, the entire script is listed below:

// ==UserScript==
// @name Gmail Announcer
// @namespace http://www.mindtrove.info/
// @description Speaks new Gmail inbox messages using Outfox
// @include https://mail.google.com/mail/*
// @include http://mail.google.com/mail/*
// @require http://outfox.googlecode.com/svn/trunk/js/outfox.js
// ==/UserScript==

var need_say = null;
var ids = {};

function sayMessages(msgs) {
    if(!outfox.defaults.config) {
	// outfox really needs a better way to detect ready ...
	need_say = msgs;
	return false;
    }

    var header = 'New messages';
    for(var id in msgs) {
	// say all messages
	var msg = msgs[id];
	var segs = msg.split(’»’);
	var sender = segs[0];
	var time = segs[1].slice(segs[1].search(’…’)+2);
	if(header) {
	    outfox.say(header);
	    header = null;
	}
	outfox.say(sender + ‘ at ‘ + time);
    }
    return true;
}

function onOutfoxReady() {
    if(need_say) {
	// say anything already queued
	sayMessages(need_say);
	ids = need_say;
	need_say = null;
    }
}

function onTableChange(event) {
    var div = event.target;
    var trs = div.getElementsByTagName(’tr’);

    var count = 0;
    var new_ids = {};
    var curr_ids = {};
    for(var i=0; i < trs.length; i++) {
	var tr = trs[i];
	if(tr.innerHTML.search('<b>') != -1) {
	    // marked as a new message
	    if(ids[tr.id] == undefined) {
		// never announced
		new_ids[tr.id] = tr.textContent;
		++count;
	    }
	    // curr is announced + new
	    curr_ids[tr.id] = tr.textContent;
	}
    }

    // report if we can
    if(sayMessages(new_ids)) {
	ids = curr_ids;
    }
}

function onDocumentChange(event) {
    if(event.target.tagName == 'DIV') {
	var div = event.target;
	var tables = div.getElementsByTagName('table');
	for(var i in tables) {
	    var table = tables[i];
	    if(table.id != '' && !table.getAttribute('role')) {
		// watch just table changes from now on
		var div = table.parentNode.parentNode;
		div.addEventListener('DOMNodeInserted', onTableChange, false);
		document.removeEventListener('DOMNodeInserted',
					     onDocumentChange, false);
		// start outfox
		var div = document.createElement('div');
		document.body.appendChild(div);
		outfox.init(div, onOutfoxReady);
		// kick off initial read manually
		onTableChange({'target' : table.parentNode});
	    }
	}
    }
}

document.addEventListener('DOMNodeInserted', onDocumentChange, false);

Accessibility Daily Moved

If you were tracking the Accessibility Daily feed I set up on Google Reader, you’ll want to update your reader to point to the new feed aliased by http://a11y.mindtrove.info. I’ve switched over to using a clone of Eitan’s Yahoo! Pipes aggregator to get more control over the output (and because I forgot the credentials for the Reader account! Doh!)

If you’re interested, the raw URL is http://pipes.yahoo.com/pipes/pipe.run?_id=6280947538e79c29bdef3017f1f6846e&_render=rss.

Dr. Horrible

I never watched Buffy or Firefly on TV, but seeing the movie Serenity and then purchasing the Firefly DVD set turned me on to Joss Whedon. He’s taken his talents online now with a three act musical, Dr. Horrible.

http://www.drhorrible.com/

I didn’t know what to think of the opening with Neil Patrick Harris (Dr. Horrible) recording his vlog. But as soon as he started singing, I started laughing. By the time Nathan Fillion (Captain Hammer) appeared, I was loving it.

Acts II and III are due in a few days. Catch them before they disappear.

Rich Audio MUDs

Gary has mentioned that sound adventure games like Descent into Madness and The Last Crusade have served as effective rewards in some local schools. Kids with visual impairments work hard in order to earn time playing them.

I’ve been brainstorming a bit about open-ended multi-user dungeons (MUDs) with rich sound and speech. Gary thinks it would be beneficial to use the MUD for educational purposes, not just as a reward after the work is done. I tend to agree, as long as its easy for teachers, older students, parents, and so on to translate lessons into in-game puzzles and adventures.

Here are some ideas I think could go into such a system to make it fun and rewarding for the kids, and an interesting platform for games.

  • Rooms and items. A simple setup including the entire dungeon, rooms with arbitrary connections, items in rooms, and users in rooms can should account for a large number of adventure game designs.
  • A basic command set. Take, drop, give, use, go, and a few other very simple commands can be supported everywhere to let the user navigate and interact with the environment.
  • An extensible parser. Items can define additional supported commands, even to the extent where the become…
  • In-world games. To go beyond exploring rooms, picking up items, and using items, items and rooms themselves can become full games. Imagine a puzzle game as an item or a room where everyone is participating in a guessing game.
  • Other input methods. Items can reconfigure the keyboard to support simpler methods of interaction. For example, an multiple choice game might require only use of the arrow keys instead of requiring the user to enter full sentences. A game might enable other devices too (e.g., DDR pad).
  • A rich audio client. Most MUD clients are text-only. When used with a screen reader, the game experience is entirely spoken. With a custom client, the game logic can provide responses the client renders as speech and sound in any number of streams.
  • A client/server configuration over XMPP. The dungeon lives on a server, though not necessarily the same machine as the XMPP server. Instead, it’s just another XMPP client with a well known JID. A bot of sorts. The rooms and items can be objects managed by that bot, or even become other bots themselves. The dungeon can exist across multiple machines.
  • Collaboration. Rooms in the dungeon are like chat rooms where some text entered in the client is broadcast to everyone, and other commands are addressed to items in rooms. Going beyond simple text, clients could implement XMPP Jingle to support voice chat.

Is anyone working on a similar project? Heard of a similar project?

Dojo 1.1 Charting

Dojo has a charting module capable of rendering a few different types of graphs. Here’s an example rendering of one sine and cosine period in SVG.

The code is straightforward:

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Dojo 1.1 Charting Example</title>
    <script type="text/javascript" src="/scripts/dojo/dojo.js">

    <script type=”text/javascript”>
        dojo.require(”dojox.charting.Chart2D”);
        dojo.require(”dojox.charting.themes.PlotKit.blue”);
        dojo.addOnLoad(function() {
          var period = 2 * Math.PI;
          var tick = Math.PI / 180.0;
          var step = 5*Math.PI / 180.0;
          var id = ‘chart_area’;

          var chart = new dojox.charting.Chart2D(id);
          chart.setTheme(dojox.charting.themes.PlotKit.blue);
          chart.addAxis(”x”, {min: 0, max: period, majorTickStep: tick*30,
                              minorTickStep: tick*10, minorLabels: false});
          chart.addAxis(”y”, {vertical: true, min: -1.01, max: 1, majorTickStep: 0.5,
                              minorTickStep: 0.1, minorLabels: false});
          chart.addPlot(”default”, {type: ‘Lines’});
          chart.addPlot(”grid”, {type: “Grid”, vMinorLines: true});

          var series = {’sin’ : [], ‘cos’ : []};
          for(var i = 0; i < period; i+=step) {
              series.sin.push({'x' : i, 'y' : Math.sin(i)});
              series.cos.push({'x' : i, 'y' : Math.cos(i)});
          }

          chart.addSeries('sin', series.sin);
          chart.addSeries('cos', series.cos);
          chart.render();
        });
    </script>
  </head>
  <body>
        <div id="chart_area" style="width: 100%; height: 400px;"></div>
  </body>
</html>

Basic animations are possible by invoking chart.updateSeries(name, data);. The result is pretty smooth in Safari, but eats a ton of CPU if done with a delay under 50ms. There also appears to be a lack of support for controlling updates to other chart features such as axes in the 1.1 version.

Other features I have yet to find (possibly because they don’t exist) include labels for axes, legends, chart titles, and a simple way to set event handlers for various chart components and actions.