JSonic: Speech and sound using HTML5
I've released a new library called JSonic for text-to-speech synthesis and sound playback in browsers supporting HTML5 <audio>. The code is on GitHub along with full documentation of the JS and REST APIs.
The client API is implemented as a Dojo dijit._Widget subclass. Other client implementations are possible as long as they provide the same JS interface. The TTS synthesis is implemented server-side using espeak and Tornado. Other server implementations are possible as long as they adhere to the REST API, and other speech engines can be plugged in rather easily.
The UNC Open Web group is looking to use JSonic to build self-voicing web games for kids with disabilities. I've already ported my Spaceship! game (also available on GitHub) to use it instead of Outfox, and hope deploy it somewhere in the near future.
Bug reports, bug fixes, comments, questions, uses, and so on are welcome. Please use the issue tracker on the GitHub project page when reporting bugs.
Spaceship! code at GitHub
I cloned the code for Spaceship! at GitHub to facilitate easier development by anyone interested. I'll push any future improvements I make there and consider my personal subversion repository dead.
Spaceship!
When Gary announced Outfox back in 2008, all manner of ideas for using speech and sound in the browser popped into my head. I've always had the boring demos (i.e., for adults) at Maze Day, so I decided to work first on a fun, somewhat educational, self-voicing browser game for the 2009 rendition. After all, keeping the mostly under-13, soda drinking, pizza eating, game playing clientele happy is always priority #1 at Maze Day.
The result is Spaceship!, a JavaScript game for Firefox built using Creative Commons licensed music, sound, speech, and graphics; the Dojo toolkit; and the Outfox add-on. In the primary portion of the game, the player fires shots at a grid of tiles trying to hit enemy ships. When the player runs out of ammo, he or she plays a set of minigames in an attempt to earn more shots. Of course, hazards and bonuses abound to keep things interesting.
A text description is nice, but you're better off watching the gameplay video below to really understand what I'm jabbering about. Or, better yet, grab Outfox and Firefox 3 and play it yourself online at http://spaceship.mindtrove.info.
What a great exercise this turned out to be! The payoff has been manyfold:
- I learning a ton more about Dojo and writing custom widgets.
- I developed some interesting MVC techniques for aural+visual event driven apps in Dojo. I hope to blog about these.
- I built some nice, reusable Dojo components for future browser games.
- I got to show off client-side music, sound, and speech in Firefox with pure JS. Maybe this will spur development of other audio apps?
- I drummed up some interest in extending Spaceship! with new minigames. Hopefully more coming soon.
- My wife was entertained. Yes, she will actually ask to play the game if she sees me working on it.
- I had lots of teachers ask when the game will be online at Maze Day. Well, here it is, a month later.
- And, most importantly, a steady stream of kids (and adults) got to play it at Maze Day. Hopefully even more can enjoy it now online.
If you try it out, leave a comment. It's new, there are bugs, and there is room for improvement. But anything you report will help in making the game better.
I owe many thanks to the artists who made their wonderful images, songs, and sounds available under open licenses. Their names appear in the Credits section off the main game menu. Be sure to check them out.
Oh, and of course the game code itself is BSD-licensed. Grab the code from http://svn.mindtrove.info/spaceship http://github.com/parente/spaceship if you're feeling adventurous.
Tidbit: doh.robot initialization
The doh.robot module for Dojo clicks a special text field inserted into the top-left of the test page during its initialization. If you obscure this magic field, the robot hangs.
If you're writing your test cases separate from the page under test (always a good idea), dojo.robotx craftily hides the iframe containing the page to test until the robot is ready. If you're not using dojo.robotx (my constraint), you must be careful to avoid that magic text box in a similar fashion.