Anyway, Gary’s post Outfox: speech, sound, and more for Firefox talks about a new Firefox extension. He’s using it to create cross-platform, self-voicing Web apps for kids with disabilities using a pure JS API. He hopes to extend his work to support alternative input devices such as game pads and switches as the Outfox extension matures and grows more flexible.

One of the other potential uses listed on the Outfox homepage is Adding new I/O to web sites with Greasemonkey. Interesting. It’s one thing to include Outfox explicitly in a page, but can it possibly work when injected by GM? What about for a complex app like Gmail with multiple iframes, dynamic changes, refreshing, etc.?

To learn about Outfox (and for fun), I decided to write a quick GM script for Gmail that announces the senders and times of new messages (bold items) in the inbox. (I would have done subject and summary too, but Outfox 0.1.0 appears to have some unicode issues and balked at some of the Gmail separator characters. Less is more at this point.) The script makes the announcement when the Gmail interface first loads, any time Gmail automatically refreshes its inbox view, or when the user clicks the refresh link to check for new mail. It is smart enough to announce a given message only once, however, so you don’t hear the same message over and over again on each refresh.

Yes. It does actually work.

To try this script, make sure you have the Greasemonkey 0.8 and Outfox 0.1 extensions installed on Firefox 3. (Or use the latest available version of each.) Then visit the following link to have GM install the script: gmail_announcer.user.js.

For reference, the entire script is listed below:

// ==UserScript==
// @name Gmail Announcer
// @namespace http://www.mindtrove.info/
// @description Speaks new Gmail inbox messages using Outfox
// @include https://mail.google.com/mail/*
// @include http://mail.google.com/mail/*
// @require http://outfox.googlecode.com/svn/trunk/js/outfox.js
// ==/UserScript==

var need_say = null;
var ids = {};

function sayMessages(msgs) {
    if(!outfox.defaults.config) {
	// outfox really needs a better way to detect ready ...
	need_say = msgs;
	return false;
    }

    var header = 'New messages';
    for(var id in msgs) {
	// say all messages
	var msg = msgs[id];
	var segs = msg.split(’»’);
	var sender = segs[0];
	var time = segs[1].slice(segs[1].search(’…’)+2);
	if(header) {
	    outfox.say(header);
	    header = null;
	}
	outfox.say(sender + ‘ at ‘ + time);
    }
    return true;
}

function onOutfoxReady() {
    if(need_say) {
	// say anything already queued
	sayMessages(need_say);
	ids = need_say;
	need_say = null;
    }
}

function onTableChange(event) {
    var div = event.target;
    var trs = div.getElementsByTagName(’tr’);

    var count = 0;
    var new_ids = {};
    var curr_ids = {};
    for(var i=0; i < trs.length; i++) {
	var tr = trs[i];
	if(tr.innerHTML.search('<b>') != -1) {
	    // marked as a new message
	    if(ids[tr.id] == undefined) {
		// never announced
		new_ids[tr.id] = tr.textContent;
		++count;
	    }
	    // curr is announced + new
	    curr_ids[tr.id] = tr.textContent;
	}
    }

    // report if we can
    if(sayMessages(new_ids)) {
	ids = curr_ids;
    }
}

function onDocumentChange(event) {
    if(event.target.tagName == 'DIV') {
	var div = event.target;
	var tables = div.getElementsByTagName('table');
	for(var i in tables) {
	    var table = tables[i];
	    if(table.id != '' && !table.getAttribute('role')) {
		// watch just table changes from now on
		var div = table.parentNode.parentNode;
		div.addEventListener('DOMNodeInserted', onTableChange, false);
		document.removeEventListener('DOMNodeInserted',
					     onDocumentChange, false);
		// start outfox
		var div = document.createElement('div');
		document.body.appendChild(div);
		outfox.init(div, onOutfoxReady);
		// kick off initial read manually
		onTableChange({'target' : table.parentNode});
	    }
	}
    }
}

document.addEventListener('DOMNodeInserted', onDocumentChange, false);

If you’re interested, the raw URL is http://pipes.yahoo.com/pipes/pipe.run?_id=6280947538e79c29bdef3017f1f6846e&_render=rss.

http://www.drhorrible.com/

I didn’t know what to think of the opening with Neil Patrick Harris (Dr. Horrible) recording his vlog. But as soon as he started singing, I started laughing. By the time Nathan Fillion (Captain Hammer) appeared, I was loving it.

Acts II and III are due in a few days. Catch them before they disappear.

I’ve been brainstorming a bit about open-ended multi-user dungeons (MUDs) with rich sound and speech. Gary thinks it would be beneficial to use the MUD for educational purposes, not just as a reward after the work is done. I tend to agree, as long as its easy for teachers, older students, parents, and so on to translate lessons into in-game puzzles and adventures.

Here are some ideas I think could go into such a system to make it fun and rewarding for the kids, and an interesting platform for games.

• Rooms and items. A simple setup including the entire dungeon, rooms with arbitrary connections, items in rooms, and users in rooms can should account for a large number of adventure game designs.
• A basic command set. Take, drop, give, use, go, and a few other very simple commands can be supported everywhere to let the user navigate and interact with the environment.
• An extensible parser. Items can define additional supported commands, even to the extent where the become…
• In-world games. To go beyond exploring rooms, picking up items, and using items, items and rooms themselves can become full games. Imagine a puzzle game as an item or a room where everyone is participating in a guessing game.
• Other input methods. Items can reconfigure the keyboard to support simpler methods of interaction. For example, an multiple choice game might require only use of the arrow keys instead of requiring the user to enter full sentences. A game might enable other devices too (e.g., DDR pad).
• A rich audio client. Most MUD clients are text-only. When used with a screen reader, the game experience is entirely spoken. With a custom client, the game logic can provide responses the client renders as speech and sound in any number of streams.
• A client/server configuration over XMPP. The dungeon lives on a server, though not necessarily the same machine as the XMPP server. Instead, it’s just another XMPP client with a well known JID. A bot of sorts. The rooms and items can be objects managed by that bot, or even become other bots themselves. The dungeon can exist across multiple machines.
• Collaboration. Rooms in the dungeon are like chat rooms where some text entered in the client is broadcast to everyone, and other commands are addressed to items in rooms. Going beyond simple text, clients could implement XMPP Jingle to support voice chat.

Is anyone working on a similar project? Heard of a similar project?

The code is straightforward:

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Dojo 1.1 Charting Example</title>
    <script type="text/javascript" src="/scripts/dojo/dojo.js">

    <script type=”text/javascript”>
        dojo.require(”dojox.charting.Chart2D”);
        dojo.require(”dojox.charting.themes.PlotKit.blue”);
        dojo.addOnLoad(function() {
          var period = 2 * Math.PI;
          var tick = Math.PI / 180.0;
          var step = 5*Math.PI / 180.0;
          var id = ‘chart_area’;

          var chart = new dojox.charting.Chart2D(id);
          chart.setTheme(dojox.charting.themes.PlotKit.blue);
          chart.addAxis(”x”, {min: 0, max: period, majorTickStep: tick*30,
                              minorTickStep: tick*10, minorLabels: false});
          chart.addAxis(”y”, {vertical: true, min: -1.01, max: 1, majorTickStep: 0.5,
                              minorTickStep: 0.1, minorLabels: false});
          chart.addPlot(”default”, {type: ‘Lines’});
          chart.addPlot(”grid”, {type: “Grid”, vMinorLines: true});

          var series = {’sin’ : [], ‘cos’ : []};
          for(var i = 0; i < period; i+=step) {
              series.sin.push({'x' : i, 'y' : Math.sin(i)});
              series.cos.push({'x' : i, 'y' : Math.cos(i)});
          }

          chart.addSeries('sin', series.sin);
          chart.addSeries('cos', series.cos);
          chart.render();
        });
    </script>
  </head>
  <body>
        <div id="chart_area" style="width: 100%; height: 400px;"></div>
  </body>
</html>

Basic animations are possible by invoking chart.updateSeries(name, data);. The result is pretty smooth in Safari, but eats a ton of CPU if done with a delay under 50ms. There also appears to be a lack of support for controlling updates to other chart features such as axes in the 1.1 version.

Other features I have yet to find (possibly because they don’t exist) include labels for axes, legends, chart titles, and a simple way to set event handlers for various chart components and actions.

The second half of the video struck me as novel. Image based reconstruction of 3D environments is nothing new. Doing it using images mined from Flickr or other large databases is as far as I know.

I Googled for the project home page for Photosynth, intending to play with their demo, but didn’t bother when I saw it was Windows only.

Gary came up with an interesting idea: create a shared Google Reader aggregate feed. The result is something very much like a Planet instance, but with Google doing the aggregation and hosting. I added most of the feed URLs from David’s existing site, slapped a subdomain together to (partially) hide the big, ugly Google permalink, and voila: Accessibility Daily.

I’d like to give it a test run as an alternative to a Planet for now. Feel free to subscribe to the Atom feed or visit the HTML page. If you’d like your site listed, post a comment or shoot me an email with a link to your feed (preferably one that includes accessibility related news only).

I’m all for using what’s already up and running. Perhaps some simple domain redirection would make it slightly easier to locate in Google? Linux Foundation already owns a11y.org and uses it to redirect to http://www.linux-foundation.org/en/Accessibility. Maybe they wouldn’t mind pointing planet.a11y.org to David’s ATRAC site?

A reference in the sidebar of planetplanet.org might also help.

Let me know if you’re interested. I will volunteer time, effort, and maybe even hosting (if necessary) if public interest reaches some critical mass.

The kids always have a blast, and this year was no exception. Some of the 21 projects demoed this year included the following:

• Carolina Rocker: A rocking-horse game that combines exercise and acoustic localization.
• Carolina Beat: An accessible version of Dance Dance Revolution for a good exercise workout to music.
• Braille Twister: A twister game using a DDR pad to form Braille letters and get exercise.
• Sweet Beat: A vision-based, edible (yes, edible) music tracker.
• Deep View: Auditory display of complex diagrams.

I believe the estimated attendance this year was something like 80 kids and 90 adults. About 30 or so stopped by to try my demo of Clique. While many of the kids were young and not yet familiar with using desktop applications, they still enjoyed figuring out Clique’s commands, typing messages, and hearing Clique’s output. And when they got bored, they jumped to the ever popular Nanomanipulator demo in the same room to touch carbon nanotubes using a PHANTOM and Atomic Force Microscope data.

While the day is intended to give these kids a trip to remember, I think I have just as much fun watching their reactions to the various projects. I find it equally wonderful to see the CS department staff, faculty, and students band together to create a memorable experience for all of the visitors.