In this proof of concept (currently only tested and working in Chrome 24+) I've split the mp3 in 25 parts by using the unix split command. The moment the first part is loaded then the playback starts immediately and it loads the second part. When the second part is loaded then I create a new AudioBuffer by combining the old and the new, and I change the buffer of the AudioSourceNode with the new one. At that point I start playing again from the new AudioBuffer.

A lot of thanks to Paul (@aerotwist) for his suggestions and to Theo for letting me use his awesome track Breathe In.

Download source or see the progject on Github

Debug: