I recently completed a project that required and Android OS application to run on a large blowup of an Android phone using an lcd monitor and a touch foil overlay. There were a number of different proposed methods for getting the app to run but we decided on using an Android emulator and running the whole OS virtually on a pc.

This presented a number of challenges including how to get the emulator running at a decent speed, and how to get multitouch input transferred from the touch system to the application.

To get the emulator running quickly we utilized a system with Intel’s VT technology and an Atom based emulator device with support for Intel’s HAXM (Hardware Accelerated Execution Manager). HAXM combined with the GPU acceleration in the Android SDK emulator made for a very fast emulation.

Unfortunately, there is a bug with the GPU accelerated emulator currently which displays rendering artifacts in web view based applications, or at least when displaying the web view based application that our client required to run.

I tried multiple approaches to getting multitouch input into the emulator. I wrote a simple java desktop app that would receive TUIO events from the touch frame and then translate those into monkeyrunner automation events. This works for native application ui elements but for some reason web view buttons would not respond to automation events. I also used an OS signed Android service app that would translate TUIO events into injected touch events but the web view would also not respond to these injected events. For more info on this app check the code at RJ’s github. Also, this approach wouldn’t work in JellyBean as it appears they removed touch event injection from the API. This guy has a good detailed blog post about different methods for injecting events, but ultimately determines that it’s not possible unless you build an app signed with your OS’s system signature.

Ultimately, I decided to just control the mouse cursor of the host OS to control the emulator touch. That meant that multi-touch wasn’t an option. That created a new problem: how to get rid of the mouse cursor in the emulator. This was trickier than it seemed because even if I removed the host OS’s mouse cursor, the emulator actually has it’s own cursor which is rendered by the SDL graphics system. In order to get rid of the cursor you need to edit the source of the SDL bundled with the Android emulator source and rebuild the entire emulator.