Web Audio: HTML5 + Flash (in a tree)
Scott Schiller
Builds front-end things at FlickrHQ
photo credit
Likes:
Code, Sound, Records
Likes:
Fun web experiments
Likes:
Fun web experiments
Let's revisit the past.
(The web / HTML / audio part.)
Then.
<bgsound src="ghostbusters.mid" autostart="true" loop="999">
Then.
<object type="audio/x-wav" data="test.wav"></object>
<embed src="sound.wav" volume="50" controls="console">
Then.
- Flash-based player UI, SWF object/embed
- Java Applets, Quicktime etc.
- External applications launched from browser
Presently.
- 100% Flash-based audio (SWF
<object>
/ <embed>
)
- HTML5
Audio()
/ <audio>
with Flash fallback
- HTML5-only audio, MP3 and/or OGG
HTML5 Audio Caveats: iOS
- Only one sound at a time.
- No auto-play allowed (pop-up blocker-style security model.) Applies to
play()
also.
- Limited or no caching?
HTML5 Audio Caveats: General
- No HTML5 audio/video support in Safari/Windows without QuickTime.
- Windows Server may have IE 9; no "desktop experience", no
Audio()
- No panning (not in spec?)
HTML5 Audio Caveats: General
new Audio() .canPlayType('audio/mp3')
Mixed browser support for MP3, MP4, OGG, WAV
Audio()
object ceilings (32?)
- HTTP: Partials (range requests) and chunked transfers de-emphasize traditional "load progress"
Bugs, Quirks and Annoyances
- Broken HTML5 audio on Safari, Snow Leopard (OS X 10.6.3 - 10.6.7.) #32159. Fixed w/10.6.8+.
- Global audio "crash", playback failure in Chrome <18. #107933.
- Related: areweplayingyet.org (HTML5 audio test suite)
Bugs, Quirks and Annoyances
new Audio(null)
may load "null" in Webkit, vs. new Audio()
(WRONG_ARGUMENTS_ERR
in Opera 9.64)
audio.src = null
vs. ''
(Firefox likes ''
, most others don't and may fail to cancel request, may not release RAM etc.)
HTML5 Today - With Caveats
- Browser implementation bugs mean
soundManager.preferFlash = true
by default.
- Thus, MP3 + MP4 playback is flash-based (by default) if available.
- Case study: Cut The Rope (MSFT-sponsored game, 100% HTML5 audio only in IE 9 due to the above.)
General Audio Concerns
- Performance, sync, timing critical for games
- Latency between JS/browser, and browser/hardware
(per-browser, per-OS?)
- Looping (ideally, seamless. Tricky.)
Lower-level: Dynamic audio?
- Base64-encoded audio data can be loaded by
Audio()
.
- Limited (realistically) to generating static sounds from code vs. loading over HTTP.
Digging Deeper: Audio Data APIs
- Binary-level read/write. Similar to what
<canvas>
does for graphics.
- Data = remix = visualization = shiny.
- Independent APIs in development by Mozilla and Webkit teams.
- W3C now discussing an "Audio Processing API."
Fun stuff: Demos
Demos using SoundManager 2
"Survivor" Atari / Commodore 64 game remake (prototype)