If you want to install the debugger version of this release on a 64 bits Linux box you have to install a current 64-bits non-debugger version first.

After that you can replace the installed libflashplayer.so by a 32 bits debugger version of this file.

This also applies if you want to install the debugger version of the current release, see note2 below.

This is how i did it on 64bits Kubuntu 9.10:

First install the non-debugger non-free version:

sudo apt-get install flashplugin-nonfree

Check if the flashplayer is installed by typing about:plugins in the address bar of your browser. You should see something like:

The installer should have made a folder /usr/lib/flashplugin-installer/, go into this folder by typing:

cd /usr/lib/flashplugin-installer/

In this folder you’ll find libflashplayer.so, this is the 64bits non-debug version. You are going to replace this version by the 32bits 10.1 beta release debugger version, so in order not to loose this file lets rename it:

mv libflashplayer.so libflashplayer.non-debug-64

Now download the 10.1 beta 3 version from adobe into your tmp directory:

cd /tmp
wget http://download.macromedia.com/pub/labs/flashplayer10/flashplayer10_1_p3_debug_linux_022310.tar.gz

Untar the tar ball:

tar xvzf flashplayer10_1_p3_debug_linux_022310.tar.gz

Now you should find a file libflashplayer.so in your tmp directory, this is the 32 bits debugger version. Move it to the flashplugin-installer directory. You have to be superuser to get this done:

sudo mv libflashplayer.so /usr/lib/flashplugin-installer/libflashplayer.so

Now the debugger version is installed! Type about:plugins in the address bar of your browser and you should see something like:

Note1: Firefox shows both the non-debugger version and the debugger version. Sometimes you have to scroll down to the bottom of the about:plugins page to see the debugger version listed.

Note2: You can follow the same steps as described above if you want to install the current version of the flashplayer debugger version. You can download this version from the adobe site.

Windows: No known issues

OSX: Only Safari is supported. If you are on 10.5 please install Java update 6, if you are on 10.6 install Java update 1. You don’t need Humatic mmj anymore. More info on humatic.de and on apple’s developer site, scroll down to Java Sound.

Linux: Konqueror is not supported

I made a sample app that you can see over here:

http://abumarkub.net/bridge-1

Code is available in several versions:

• FDT project
• FlashBuilder 4 project
• Flash CS4 project

Please report back to me if it doesn’t work on your system.

About midi out

Now that you can select a midi out device, the problem with latency arises. Unless you’re on a Mac. On a Mac you can just select a default midi synthesizer (Java Sound Synthesizer) and start playing with no noticeable latency.

If you’re on Windows or Linux, the latency of the Java Sound Synthesizer makes it almost impossible to play. On Windows you can also choose the Microsoft GS Wavetable Synth and with some soundcards you may get a decent latency (i was told the Realtek AC97 perfoms pretty well).

Latency is caused by both the drivers of your soundcard and the way your synthesizer works. Most modern softsynths hardly cause any latency, but even with the latest M-Audio pro cards you’ll experience latency when using the Java Sound Synthesizer or the Microsoft GS Wavetable Synth.

So we need to be able to connect to some real softsynths like Pianoteq or Lounge Lizard and for this we need a virtual midi driver.

If you’re on a Mac, you’re lucky again because such a thing is already installed on your machine. It is called IAC Driver and you’ll find it if you open the Audio MIDI Setup in your Applications folder.

If you are on Windows you can download LoopBe1 from nerds.de and Linux users can check VirMidi

Below i’ll give a brief explanation for every driver.

LoopBe1

Download it from nerds.de and run the installer. After the installation has finished LoopBe is up and running and will automatically start with Windows (if you don’t want this, run msconfig and remove the LoopBe startup service).

Now if you refresh or load my sample application you should see LoopBe Internal MIDI appearing in both the midi inputs drop-down box as well as the midi outputs drop-down box. Choose your favorite keyboard for input device and LoopBe Internal MIDI for output.

Now open your favorite softsynth and go to the midi settings and set your synth’s midi input to LoopBe Internal MIDI. Here is a screendump of what this looks like in Pianteq:

You should now be able to play your softsynth while midi data is passing thru the midibridge, and dependent on your soundcard’s driver, with very low latency.

Please notice that LoopBe1 is only free for non-commercial use. For commercial use you need to acquire a license after a 30-day evolution period. But for only € 11,90 inc VAT it’s really a bargain. Check here.

IAC

Open Audio MIDI Setup from your Applications folder. You should see an icon named IAC Driver. IAC stands for Inter-Application Communication, and that is exactly what it does.

If the icon is greyed out double click it and in the popup that appears check the box “Device is online”. See screenshot:

Don’t worry if looks a little different on your machine. You should see at least 1 port in the “Ports” part of the screen. If not, simply click the plus sign to add a port.

Now close this popup and the Audio MIDI Setup and refresh or load my sample application. You should see LoopBe Internal MIDI appearing in both the midi inputs drop-down box and the midi outputs drop-down box.

Choose your favorite keyboard for input device and LoopBe Internal MIDI for output. Open your favorite softsynth and go to the midi settings and set your synth’s midi input to LoopBe Internal MIDI. Here is a screendump of what this looks like in Lounge Lizard:

Now you can play your softsynth while midi data is passing thru the midibridge.

VirMidi

I haven’t had the time to figure this out, i’ll get back to you as soon as I have it up and running. Pianoteq is also available for both 32 and 64 bits Linux, so if you want to try it yourself you can download a demo version over here.

Onwards

Unfortunately the bridge won’t work on Konqueror until this browser supports Live Connect; communication between Java applets and Javascript and Flash applets and Javascript.

On OSX, browsers that rely on the Java Embedding Plugin such as Firefox, Camino and Seamonkey are not supported either. This is because of something really weird. In Firefox an applet gets loaded twice and therefor (or for some other reason) JSObject.getWindow(this); returns ‘null’ so the connection between Java and Javascript is cut off.

I am gonna look into this asap but should you have and suggestions please mail me!

It is built for player 10 so you can load local files directly in to the player, you can also capture images from your webcam. You can find it here:

http://abumarkub.net/animatedgifmaker/

Mac OSX and Vista 64bit are now supported, check the example!

For OSX
Please download mmj first and place mmj.jar and libmmj.jnilib in your /Library/Java/Extensions directory.

Because the applet is loading the mmj library from your local system, the applet has to be signed and you have to trust it before it can run.

For Vista 64bit
Please go to Control Panel -> View 32-bit Control Panel Items -> Java -> Advanced -> Java Plug-in and deselect ‘Enable the next-generation Java Plug-in’.

In general
If the midi bridge loses its connection to your midi device, or the bridge isn’t responding at all anymore, please close your browser(s) completely. That means: close all running browser windows of all browsers that have been running the applet. This is to ensure that you kill the Java Plug-in process completely.

All midi bridge examples are updated and your platform should be detected automatically, so go ahead and test the bridge on your machine with the links in this post or in this post.

Please let me know if it doesn’t work on your system.

You can find the code overhere.

As developers started to use the midi-to-flash bridge i realized that the organization of code was not flexible enough. So i rearranged the code rather drastically. The code is now split up into two pieces: the midi bridge and the midi client. The midi bridge comprises 3 parts:

• a Java applet that receives the midi data from your system
• some JavaScript code that enables 2-way communication between Java and Flash
• a small Flash application that broadcasts the midi data to other Flash applications thru a LocalConnection

See the generated asdocs.

This is a sample of a midi bridge. You can set the bridge id and select the midi device that you want to connect to the midi bridge. These parameters can also be set as flashvars in the containing html file.

You can run multiple instances of a midi bridge at the same time, and you can connect each instance to different midi device.

Because a midi client receives midi data thru a LocalConnection it can be any type of Flash application: a swf running in a browser or in a standalone player, a projector or an AIR application.

For a midi client to receive midi data from a midi bridge, you need to create an instance of the MidiClient class in your code and pass it a midi bridge id. If a bridge with the same bridge id is running, this will get your client automatically connected to desired bridge at runtime:
var bridgeId:String = stage.loaderInfo.parameters["midiBridgeId"];
bridgeId = bridgeId == null ? "_device1" : bridgeId;
_midiClient = new MidiClient(bridgeId,false);
_midiClient.addEventListener(MidiDataEvent.CONNECTED, handleMidiDataEvent);
_midiClient.addEventListener(MidiDataEvent.MIDI_DATA, handleMidiDataEvent);

Subsequently you have to implement event listeners and actually do something with the received midi data. In the package you’ll find some handy and efficient midi data parse utilities; these might help you kick-start writing your own code.

You can connect as many midi clients to a midi bridge as you like, and you can disconnect and reconnect them at any time. You can also set up a single client to receive midi data from several bridges, or let your client connect to different midi bridges successively.

This can be very handy if you create an installation with multiple monitors, or if your application has to run on a huge plasma screen.

You can find a sample of a very simple midi client overhere, go ahead and open and close as many clients as you like. Don’t forget to start up the midi bridge first.

multi bridge scenario’s

It is advisable that you have 2 midi devices attached to your computer to test the multi bridge scenario. This is how you set it up:

• open a bridge, give it an id and press ‘ok’
• open another bridge, give it a different id and press ‘ok’
• choose a midi device that you wish to connect to this bridge, play some notes to check connectivity
• switch to the other bridge and select a different midi device, play some notes or move some controllers to check connectivity
• open a client and enter either of the 2 bridge ids you have used above and press ‘ok’
• play some notes on the midi device that is connected to the chosen bridge and you’ll see the data appearing in the client
• repeat last 2 steps for another client and connect this one to the other bridge
• connect as many clients as you like to either of the 2 bridges

Obviously, you can make set ups with more than 2 bridges as well.

Recently i have been asked several times for the code of the midi-to-flash bridge. It has always been my plan to publish the code but i could not find the time to do it. Finally today i have uploaded the code to Google Code, you can find the repository over here:

http://code.google.com/p/miditoflash/

It is published under new BSD license. Though this is a non-reciprocal license, it would be nice if you update me about your improvements.

See the generated asdocs.

You can compile the code with Flex SDK and with Adobe’s Flash IDE. Read more about compiling with the Flash IDE here.

The code of the Chord Finder app uses the net.abumarkub.harmony package and has only been uploaded partly; currently i am busy implementing more chords and modi. In the code chords and modi are represented as 12 bits numbers to get the smallest possible response time when you play a chord on your keyboard. I’ll go into further detail in a future post.

I started by looking into the SOUND_COMPLETE event. When using a loop of 2048 or less samples, the exactness of this event varies roughly between 25 and 100 ms whereas it should be a steady 46 ms (1000/44100 * 2048 ≈ 46). So i used an old trick by adding some ms of silence in front of the actual sound. Whenever a SOUND_COMPLETE occurs i calculate the deviation compared to theoretical (multiple of) 46 ms and i compensate this by starting the sound earlier or later dependent on the offset:

sound.play(offset,1);

Although this works pretty well in the standalone player, in a browser on slower processors the beat is very unstable and i experienced hiccups and fall outs. So i started studying the Popforge library renowned for its steady beats.

I took me quite some time to grasp how it actually works but i became gradually clear to me after i had made a visualization of the principle. Below you can create a pattern and by clicking on start the sequencer starts.

Next, the red bars that appear demark the audio buffers and the green bars set apart the beats. If you increase the tempo you’ll notice that more beats fit in a single buffer.

As you can see, sometimes a single sample is split up and spread over 2 buffers. Because the buffers are seamlessly glued after each other you don’t hear the joints.

The indicator on top of the pattern-edit grid is not very precise, and if you increase the tempo it gets even lesser accurate. André Michelle made an online version of the Roland TR909 and if you look closely at this implementation of Popforge, you’ll notice that the leds above the pattern buttons aren’t very exactly either. But as the emphasis of the Popforge library lies on sound, this is perfectly valid.

However, in my case i had to deal with a video aspect as well. After some fruitless experiments i came up with a rather quick and dirty trick: i adapted the tempo as one beat to match exactly with the length of a buffer.

Downside of course is that i am tied to a very limited set of tempos. Buffersize 8192 (4 times 2048) boils down to 1000/(8192*1000/44100) * 60/4 ≈ 81 bpm. In the example above i use a buffer of 12288 bytes (6 times 2048) so a tempo of approx 54 bpm will align one beat to one buffer.

Anyway the result feels rather synced and steady and it performs way better than the version in Director. I am still left with an unanswered question though: the engine that glues the subsequent buffers after each other is the SOUND_COMPLETE event that is fired by a dedicated sync sound.

This sync sound is just an array of ‘silent bytes’ with a length that matches the buffersize (minus one). As soon as the application is initialized the sync sound starts silently playing. Whenever the SOUND_COMPLETE event arises the AudioBuffer class checks if any sample data is prepared in the buffer. If so, the data is played back and you actually hear something. In any case the sync sounds starts playing again, resulting in an endless sync stream.

Because the SOUND_COMPLETE event isn’t very steady, even at higher framerates (80 – 120 fps), i would expect to hear gaps and pops because a deviation of say 20 ms of the theoretical interval between 2 subsequent SOUND_COMPLETE events should be audible, and this deviation is not compensated somehow or other in the Popforge library.

In theory the first method i tried should give the same result as the Popforge library, if even better, because of the compensation of the deviation (the offset, see above), but it doesn’t.

However, if i test the swf above on a rather old processor (AMD Turion 64 ML28 1,6 GHz), Popforge starts stammering at and above 66 bpm, whereas the FL909 works like a charm at any tempo.

Since drawing the waveform is rather CPU intensive, i guess it causes the ENTER_FRAME event to become such instable that the SOUND_COMPLETE event can’t be taken care of in time, resulting in gaps between the sample slices. When using a buffer of 12288 bytes, the framerate must be at least 1000/(12288 * 1000/44100) ≈ 3,58, say 4 fps, and apparently with heavy drawing duty, the flashplayer sometimes drops below.

I have tried to compensate this with the offset trick described above, but until now without success. Still busting my brains on this.

A new version of the bridge has been released, check here

You can find a proof of concept overhere. You will need to have flashplayer 10 installed because i use the dynamic sound generation possibilities of player 10 for producing sounds when midi note-on data is received.

If you don’t want the hassle of installing flashplayer 10 you can find a flashplayer 9 version overhere. You won’t be able to enjoy the beautiful warm sinus tone generated by flash but you can see flash responding to your midi keyboard. The Release version of Flash Player 10 is now available from the Flash Player Download Center. However, the flashplayer 9 version is still online.

connecting flash to a midi device

When the page has fully loaded, flash asks the Java-applet to investigate your midi setup and displays the result in the yellow part on the top right corner. If a midi in port is detected and available, it will automatically be connected to the flash interface. If not, you can click on one of the other detected devices: you need to look for a midi in port. If Java was not able to detect the kind of midi device (midi in, midi out, synth, sequencer), it will be displayed as ‘device’. A red bullet indicates a successful connection.

So now if you press a key or move a slider on your midi keyboard you should see this represented in the little pink square on the lower right corner. If you play 3 or more notes simultaneously, the purple square on the upper left corner will try to find out what cord you are playing (see below).

the keyboard and velocity sensitivity

At the center bottom, the matching piano keys will light up on the software keyboard while playing your hardware keyboard. You can also play on this keyboard by clicking the keys with your mouse, but the software keyboard isn’t (yet) velocity sensitive, as opposed to your hardware keyboard. That is: if you have a velocity sensitive keyboard connected.

Because velocity sensitivity of hardware keyboards is supported, i divide the volume of the sound channel by 10. Otherwise the sound might be distorted when you play 2 notes at maximal velocity. By dividing it by 10 this could still happen if you play more than 10 notes at maximum velocity but i assume that you have 10 fingers and won’t use you forearm to play clusters

I have to find another solution though once sustain pedal support is implemented.

pitch bend

In the lower left corner you can set the amount of bend that is used by your pitch bend wheel. You can either pick an interval with the drop down box (from a minor 2nd up to an octave), or choose any value in between using the slider.

the ‘color organ’

I have also implemented a very rudimentary form of a color organ which you’ll find in the black square above the keyboard. You can assign midi controllers or notes to the RGB channels represented by the 3 little accordingly colored squares.

It works like a regular midi-learn/midi-forget toggle that you can find in many offline midi programs: click on the software controller, move the hardware controller, press ‘ok’ and you’re done.

If you connect a note, the velocity is used to change the value of the channel and all notes with the same name in other octaves will be connected as well. This can be interesting: i have experimented a little with assigning specific tonic, subdominant and dominant notes to the channels, like for instance in key of C the note C to the green channel, the note B to the red channel and the note A to the blue channel.

Now when you play you’ll see that colors are more or less matched to harmonic functions: dominant chords will always be a little reddish. The color of a Dmin7 will be, dependent on how hard you play the C and the A, beautifully aquamarine. Cmaj7 is yellow, but if you play the seventh very loud, it will turn more orange.

I have to think up more refined and intelligent ways of connecting tones to harmonies, but i wanted to put this simple color organ in my proof of concept to show as an interesting possibility of a flash midi connection.

the chord finder

This is in fact the reason why started this project: i wanted to create a decent online chord finder. The chords are related to scales and are presented as modi of all scales they fit in. Not all scales are implemented yet, so there are also missing chords.

Only major, minor, minor harmonic and -melodic scales are implemented. Also, only the most common keys are used: C, F, Bb, Eb, Ab, Db, Gb, B, E, A, D, and G. This implies that if you play Db-F-Ab the chord finder will say Db major, but if you lower the third to Fb the chordfinder will enharmonise it to C#-E-G# and thus tell you it is C# minor. This is perfectly sound because C# minor (E major) is far more common then the minor parallel key of Fb (!).

Also this part needs some good reworking because i am not completely happy with the order in which the modi are displayed.

more to come

I am still working on this.