After weeks of investigation and discussion we have come up with a way to create a button on the content of a loaded in transparent png.
You can set a sprite mask to use a bitmap and it will mask successfully so long as both are cached as a bitmap. This is ok but when you put mouse listeners on the sprite it registers the whole rectangular area.
Next we thought we'd utilize the bitmapData getPixel32 method which returns an ARBG colour when passed in the x and y coordinate. This works great but you can only get 1 pixel at a time.
So then we thought a bit outside the sqaure and rather than create a dynamic hit area over the bitmap, we'd use the stage's MouseEvent.MOUSE_MOVE listener and get the x and y of the current position then pass that into the getPixel32 methods of each bitmap.
Success however for overlaying bitmaps this method is limited. Now we would need to updated the mouseEnable property on the fly to allow for bitmaps that are being obscured.
So the final thing you would need to do is check to see whether or not the ARBG value is equal to 0. If so enable the mouse add a MouseEvent.CLICK listener to the clip and set the button mode to true. I also use a boolean isActive to avoid adding multiple listeners. Once the ARBG value equals 0 then disable the actions by removing the MouseEvent.CLICK listener set the button mode to false and set mouseEnabled to false.
The reason we use the stage MouseEvent.MOUSE_MOVE listener is so we can always receive updates regardless of whether the bitmap button is enabled or not. If we set the bitmap button MouseEnabled property once we set it to false we'd never receive mouse events again.
So there we have it, still in it's infancy but nonetheless a solution to a perculiar problem. See the attached AlphaBitmapButton.as to view the workings.
View Example
AlphaBitmapButton.as (3.76 kb)