For completeness, here's the code I sent him that accomplishes the same thing using the older Glade.XML object for those that are interested. It answers a long standing mailing list question about using Glade.XML.Autoconnect in IronPython.

import clr
clr.AddReference('gtk-sharp')
clr.AddReference('glade-sharp')
import Gtk
import Glade
 
def PyGladeAutoconnect(gxml, target):
    def _connect(handler_name, event_obj, signal_name, *args):
        name = ''.join([frag.title() for frag in signal_name.split('_')])
        event = getattr(event_obj, name)
        event += getattr(target, handler_name)
 
    # add all widgets
    for widget in gxml.GetWidgetPrefix(''):
        setattr(target, gxml.GetWidgetName(widget), widget)
    # connect all signals
    gxml.SignalAutoconnectFull(_connect)
 
class Application:
    def __init__(self):
        gxml = Glade.XML("test.glade", "window1", None)
        PyGladeAutoconnect(gxml, self)
        # window1 comes from glade file
        self.window1.ShowAll()
 
    def onWindowDelete(self, o, args):
        # connected via glade file definition
        Gtk.Application.Quit()
 
Gtk.Application.Init()
app = Application()
Gtk.Application.Run()

Can you remember a time when the title of this blog post might have landed me in a straight jacket? Can you believe that was just a few short years ago? Yea, I can't either.

Anyway, Gary's post Outfox: speech, sound, and more for Firefox talks about a new Firefox extension. He's using it to create cross-platform, self-voicing Web apps for kids with disabilities using a pure JS API. He hopes to extend his work to support alternative input devices such as game pads and switches as the Outfox extension matures and grows more flexible.

One of the other potential uses listed on the Outfox homepage is Adding new I/O to web sites with Greasemonkey. Interesting. It's one thing to include Outfox explicitly in a page, but can it possibly work when injected by GM? What about for a complex app like Gmail with multiple iframes, dynamic changes, refreshing, etc.?

To learn about Outfox (and for fun), I decided to write a quick GM script for Gmail that announces the senders and times of new messages (bold items) in the inbox. (I would have done subject and summary too, but Outfox 0.1.0 appears to have some unicode issues and balked at some of the Gmail separator characters. Less is more at this point.) The script makes the announcement when the Gmail interface first loads, any time Gmail automatically refreshes its inbox view, or when the user clicks the refresh link to check for new mail. It is smart enough to announce a given message only once, however, so you don't hear the same message over and over again on each refresh.

Yes. It does actually work.

To try this script, make sure you have the Greasemonkey 0.8 and Outfox 0.1 extensions installed on Firefox 3. (Or use the latest available version of each.) Then visit the following link to have GM install the script: gmail_announcer.user.js.

For reference, the entire script is listed below:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95

// ==UserScript==
// @name Gmail Announcer
// @namespace http://www.mindtrove.info/
// @description Speaks new Gmail inbox messages using Outfox
// @include https://mail.google.com/mail/*
// @include http://mail.google.com/mail/*
// @require http://outfox.googlecode.com/svn/trunk/js/outfox.js
// ==/UserScript==
 
var need_say = null;
var ids = {};
 
function sayMessages(msgs) {
    if(!outfox.defaults.config) {
	// outfox really needs a better way to detect ready ...
	need_say = msgs;
	return false;
    }
 
    var header = 'New messages';
    for(var id in msgs) {
	// say all messages
	var msg = msgs[id];
	var segs = msg.split('»');
	var sender = segs[0];
	var time = segs[1].slice(segs[1].search('…')+2);
	if(header) {
	    outfox.say(header);
	    header = null;
	}
	outfox.say(sender + ' at ' + time);
    }
    return true;
}
 
function onOutfoxReady() {
    if(need_say) {
	// say anything already queued
	sayMessages(need_say);
	ids = need_say;
	need_say = null;
    }
}
 
function onTableChange(event) {
    var div = event.target;
    var trs = div.getElementsByTagName('tr');
 
    var count = 0;
    var new_ids = {};
    var curr_ids = {};
    for(var i=0; i < trs.length; i++) {
	var tr = trs[i];
	if(tr.innerHTML.search('&lt;b&gt;') != -1) {
	    // marked as a new message
	    if(ids[tr.id] == undefined) {
		// never announced
		new_ids[tr.id] = tr.textContent;
		++count;
	    }
	    // curr is announced + new
	    curr_ids[tr.id] = tr.textContent;
	}
    }
 
    // report if we can
    if(sayMessages(new_ids)) {
	ids = curr_ids;
    } 
}
 
function onDocumentChange(event) {
    if(event.target.tagName == 'DIV') {
	var div = event.target;
	var tables = div.getElementsByTagName('table');
	for(var i in tables) {
	    var table = tables[i];
	    if(table.id != '' && !table.getAttribute('role')) {
		// watch just table changes from now on
		var div = table.parentNode.parentNode;
		div.addEventListener('DOMNodeInserted', onTableChange, false);
		document.removeEventListener('DOMNodeInserted', 
					     onDocumentChange, false);
		// start outfox
		var div = document.createElement('div');
		document.body.appendChild(div);
		outfox.init(div, onOutfoxReady);
		// kick off initial read manually
		onTableChange({'target' : table.parentNode});
	    }
	}
    }
}
 
document.addEventListener('DOMNodeInserted', onDocumentChange, false);

Gary came up with an interesting idea: create a shared Google Reader aggregate feed. The result is something very much like a Planet instance, but with Google doing the aggregation and hosting. I added most of the feed URLs from David's existing site, slapped a subdomain together to (partially) hide the big, ugly Google permalink, and voila: Accessibility Daily.

I'd like to give it a test run as an alternative to a Planet for now. Feel free to subscribe to the Atom feed or visit the HTML page. If you'd like your site listed, post a comment or shoot me an email with a link to your feed (preferably one that includes accessibility related news only).

I'm all for using what's already up and running. Perhaps some simple domain redirection would make it slightly easier to locate in Google? Linux Foundation already owns a11y.org and uses it to redirect to http://www.linux-foundation.org/en/Accessibility. Maybe they wouldn't mind pointing planet.a11y.org to David's ATRAC site?

A reference in the sidebar of planetplanet.org might also help.

Let me know if you're interested. I will volunteer time, effort, and maybe even hosting (if necessary) if public interest reaches some critical mass.

Spatial event sounds: click on a button on the left side of your screen, and the event sound comes out of your left speaker. Click on one on the right side of your screen, and the event sound comes of of the right speaker. It’s earcandy, but I think this could actually be quite useful, but only if we get better quality event sounds, than we have right now.

While spatialized event sounds may be “earcandy” as Lennart admits, there are other benefits of using 3D audio over mono sounds in certain applications. One interesting use concerns the separation of concurrent sound streams such that a user can distinguish and “pick out” one of many. The theory of auditory scene analysis (Bregman, 1990) says (among many other things) that we humans can better segregate different sound sources and select one for attentive processing if the acoustic and semantic properties of streams from distinct sources differ along certain dimensions while certain properties within a stream remain constant over time.

For instance, say I make two audio recordings of the same person speaking two different utterances. In one recording, the person says “What a lovely bunch of coconuts.” In the other, “That dog certainly has fleas.” If I mix these two recordings into a mono track with the two utterances starting at exactly the same time, you will have a hard time determining and understanding the two independent phrases. If I create a stereo sound, with one phrase played in the left speaker and another in the right, you’ll have a much easier time identifying the original phrases. (But you’ll likely have to listen to the sound more than once before you can repeat both phrases: another tenet of auditory scene analysis.) Better still, if I apply a head related transfer function (HRTF) to each recording such that the two utterances appear to come from the left and right side of you head in a 3D space, your task becomes even easier.

In other words, spatialized sounds aid segregation and selection of independent streams of speech and sound. In fact, research (McGookin, 2004) suggests that spatialization alone is sufficient to aid recognition of information encoded in properties of concurrent musical sounds (earcons).

Applying the concept of distinguishable sound streams to screen readers is an interesting endeavor. (Or, at least, I think so.) Screen readers currently rely on a single, serial stream of speech and sound to describe the multitasking, high-bandwidth graphical desktop. In a single stream design, reports of peripheral information outside the application focus are either non-existent, delayed, or interruptions, and can be easily missed. For instance, if a screen reader is busy reading an email when the user receives an instant message in another application, the screen reader has to decide whether to keep reading the email or announce the new message in some manner. If the screen reader interjects, the user might confuse the instant message content with that of the email or become annoyed with the interruption. If the screen reader decides to wait for the email reading to finish, the late announcement about the chat message runs the risk of being stale.

Worse yet, any single-stream announcement of the new message can be inadvertantly interrupted at any time by the next user command. In such a situation, unless the user tabs around looking for the new instant message or the chat program is set to play a sound every time a message is received (which still doesn’t indicate which of potentially many chats has the new message), the user may never learn of the existence of the new message.

Concurrent streams provide an answer to this peripheral awareness problem, but only if the screen reader can present them in a way that avoids masking other simultaneous streams. And this is exactly where the ability to spatialize sound helps. Without interrupting or modifying the stream of speech reading the email, another stream can pipe up and announce the new chat message with a sound, speech, or both according to the verbosity settings of the user (Zing! or “Message from Harvey” or “Harvey says ‘Hey! Stop reading your email and answer me! This is important!’”). As long as these streams are spatially separated according to some simple rules, the user will be able to effectively distinguish them, ignore one, listen to one, and switch attention back and forth between them.

Instant messaging is just one example of a modern desktop application that begs for concurrent streams in screen readers. Just from looking at my GNOME desktop I see a system monitor, the clock applet, my network status, a popup balloon, and a log monitor all updating in the background while I write this post. Of course, a user can’t cope with all of these event sources reporting at once. But that’s where the interesting design problems start: how do we construct a usable multi-stream auditory display?

An open source library supporting spatial sound is a fundamental building-block for this investigation (and I’m certain, others). I hope Lennart pursues it.