Parsing and rendering the various SVG layers that make up each clock is a relatively expensive operation. Locking up the user interface while doing these operations is bad design and leads the user to believe that the application may not working correctly. What I’m attempting to do seems pretty simple on the surface. The following screen capture shows the (partially working) goal:

The selector consists of two columns, an image of the clock on the left and the name of the clock on the right. Using the standard ListActivity support, the goal is to show an indeterminate progress image in the left column until the layers have been loaded and rendered. At that point, the animated progress image is hidden and the clock image is displayed.

The implementation is simple in concept. The list adapter subclasses the standard BaseAdapter implementation to provide an implementation of the getView method. The getView method delegates loading of the clock’s image to an asynchronous task, keeping the user interface alive. When the asynchronous task completes, it signals the adapter by calling notifyDataSetChanged. The next call to getView will hide the progress animation and display the rendered image.

While this is mostly working, I’m hitting a random deadlock condition that locks up the UI thread, usually resulting in an “Application Not Responding” (ANR) error message to the user. All of my attempts to diagnose and resolve this issue have ended with nothing but frustration. So far, I’ve tried all of the “standard” approaches to diagnose this:

• Analyzed the traces.txt file generated by the ANR.
  Unfortunately, the process never shows up in the log file.
• Force-generated a traces.txt file using kill SIGQUIT pid while the application was still running.
  Again, the process doesn’t seem to show up. Instead, I see “W/dalvikvm(19144): threadid=4: spin on suspend #1 threadid=9 (pcf=0)” whenever attempting this.
• Attempted to use the standard Eclipse debugger to suspend the UI thread once it is locked up.
  No big surprise that it did not work.
• Added lots of Log output trying to pinpoint the hang.
  At least from that logging, it doesn’t appear to be a hang anywhere directly in my code.
• I even tried to use method tracing to see if I could figure anything out.

At this point, I’ve run out of good ideas on how to track down and fix this problem on my own. I’ve gone ahead and joined the Google android-developers group and asked for ideas on solving this. I’m still waiting for my question to clear moderation to see if anyone else can offer any insights. If anyone reading this has any ideas, I’d love to hear them. Until then, this project is officially deadlocked.

The internet offered no examples of how I might go about doing this, so I went spelunking on my own. For those that may need to do something similar, I decided to capture the steps I used to relink my Android Market applications with a single GMail address. Before I go any further, let me offer some warnings:

November 13, 2011 Update

While this appears to work in the short term, I’m noticing applications slowly migrating back to the original accounts. I’m not entirely sure where those values are being stored or how to fix this permanently.

The Market Assets Database

As of this writing, Android Market (implemented by the com.android.vending package) stores the linkage between an Android application package and the associated Android Market account in the file:

/data/data/com.android.vending/databases/market_assets.db

The ASSETS table within that SQLITE database contains the mappings as seen in this screenshot.

Relinking applications is a matter of properly updating this table. The prerequisites for this process are:

• Rooted device
• Android Debug Tools (adb) installed and working

Relinking Process

Start by making sure that Market application has been stopped:

1. Navigate to Settings -> Applications -> Manage Applications
2. Locate and select the Market application in the list
3. Press the Force Stop button

Make sure that your device is attached with USB Debugging enabled:

$ -> adb devices
List of devices attached
3231174E43AA00EC	device

Open a shell on the device and navigate to the database directory:

$ -> adb shell
# cd /data/data/com.android.vending/databases
# ls -l
-rw-rw----    1 app_57   app_57       34816 Oct 29 11:55 market_assets.db

Backup the current file:

# cp market_assets.db market_assets.db.save
# ls -l
-rw-rw----    1 app_57   app_57       34816 Oct 29 11:55 market_assets.db
-rw-rw----    1 root     root         31744 Oct 29 11:44 market_assets.db.save

Update the database file:

# sqlite3 market_asset.db
sqlite> update ASSETS set ACCOUNT = 'new.email@gmail.com'
   ...> where ACCOUNT = 'old.email@gmail.com';
sqlite> .exit
# exit

At this point, reboot your phone. If all went correctly, your free applications should now be associated with new.email@gmail.com.

Updating Paid Applications

As I mentioned earlier, this process can’t be used to relink paid applications due to DRM that is tied to your account. If you want to do this, most application developers will offer a refund if you purchase on a new account and send them the order details for both the old and new accounts. Once you’ve purchased on the new account, you can uninstall for the old account and reinstall from the new. This is a lot of effort, but can help if you need to switch market accounts.

This screen capture from the emulator shows multiple live instances of the widget running simultaneously with many different themes. I would not suggest that anyone actually do this do the amount of memory required, however it shows the power of the themes.

Android analog clock displaying many themes simultaneously.

Implementation Notes

Getting to this point has been an interesting process, as the Android widget support definitely makes this type of widget somewhat difficult to build.

AndEngine SVG Support

I’ve had this idea for quite some time. What helped me move forward was the addition of SVG support to AndEngine. Thanks to Nicolas Gramlich yet again for his excellent engine. I’ve found a few glitches along the way that I’ve started to submit patches to the project to correct, but as always his code works amazingly well.

Per-Minute Updates

As I mentioned in my inertia post, the standard Android widgets model is really more of a “pull” model versus a “push” model. The widget provider definition file (in XML) specifies the frequency that the Android framework will call the widget’s update functionality:

    android:updatePeriodMillis="1800000"

However, no matter what is specified for this value, Android limits the update frequency to no more than once every 30 minutes to avoid the battery drain associated with executing code too often. Thus, it is necessary to push changes to the widget based on our own schedule to make sure the clock is updated every minute.

My initial implementation used Java’s Timer and TimerTask to do these updates. Looking at the framework’s analog clock implementation, I discovered Android’s time-related broadcast messages:

• Intent.ACTION_TIME_TICK
• Intent.ACTION_TIME_CHANGED
• Intent.ACTION_TIMEZONE_CHANGED

Using these broadcast messages is a vast improvement over maintaining my own timer threads for this functionality. However, since a widget is nothing more than a fancy broadcast receiver, it is necessary to spin up a separate service to register a broadcast receiver for these messages. On reception of one of these messages, each clock instance is updated to match the current time. This update generates a properly sized bitmap that is pushed to the widget instance:

		RemoteViews views = new RemoteViews(context.getPackageName(), R.layout.main);
		views.setImageViewBitmap(R.id.imageview, bitmap);
		appWidgetManager.updateAppWidget(instanceIdentifier, views);

It’s important to note that there is also an updateAppWidget method call that accepts an instance of android.content.ComponentName. Using this method will update all instances for the specified provider with the bitmap. It took a bit to figure out why all of my clock instances were showing the same theme until I realized it was due to using the wrong method.

Improving Battery Performance

Given that this widget is controlling updates rather than allowing the framework to do the job, my primary concern is in terms of performance. Android devices are notorious for poor battery life, however it does seem that it is primarily due to background applications. I’ve done a couple of things thus far to attempt to minimize battery usage.

No Seconds Hand

At least at the moment, the widget removes the seconds hand to avoid pushing more updates to the screen than necessary. Assuming that Android wants to limit updates to once every 30 minutes, the widget is already pushing updates 30 times more often than the framework would like. Multiplying that yet again by 60 seconds seems like a bit too much. In the future, I may consider allowing the user to enable the seconds hands with proper warnings attached.

Manage Broadcast Receiver Messages

An unfortunate side effect of the way that widgets work is that it does not appear to be possible for a widget to determine if it is actually being displayed. If a widget is placed on screen 1, but the user is currently viewing screen 2, there is really no reason to update the widget. With that said, the implementation can still be smart about updates. The service alters the messages it listens for based on the status of the screen:

• Screen Off
• Intent.ACTION_USER_PRESENT
• Screen On
• Intent.ACTION_SCREEN_OFF
• Intent.ACTION_TIME_TICK
• Intent.ACTION_TIME_CHANGED
• Intent.ACTION_TIMEZONE_CHANGED

After the user has cleared the lock screen (ACTION_USER_PRESENT), the widget registers to hear time updates as well as the screen turning off. Once the screen turns off, the widget stops listening for time updates and switches to listening for user presence. This lowers the update frequency for the widgets when there is no chance that they will actually be visible to the user.

What’s Next?

This project has a chance of being something that I can complete and that others might be interested in actually using. I’m considering whether I want to submit this to the Android Market when it is a bit further along. If I do, I will have to decide whether or not I would charge for it which implies a certain level of required support. Either way, I’m not at the point of releasing any significant amount of source code until I decide what to do with this project.

Inertia is the resistance of any physical object to a change in its state of motion or rest, or the tendency of an object to resist any change in its motion.

For me, this also describes my tendencies toward side projects like my Pingus project. When I last worked on Pingus a couple of months ago, I updated the underlying AndEngine libraries and found a ton of breaking changes. I put Pingus on the shelf until I had more time to look at the breakage and how to solve it. The AndEngine changes are pretty significant and I’m going to need to rethink portions of Pingus in order to get things running correctly again.

Now my personal inertia is kicking in and causing me to put off this rework for a while. To me, this is the biggest difference between work and hobby projects… I don’t have to work on hobby projects if I don’t want to. Thus, Pingus is “on a break” for a while until I find the energy to bring it up to date relative to the underlying game engine.

In the meantime, I decided I wanted to spend a bit of time taking a look at Android’s App Widget support. Until I started digging into the documentation and examples, I had always assumed that a widget was provided a Canvas to draw directly on to the home screen. To me this seemed like it would have been the easiest way for developers to develop widgets.

It turns out that Android AppWidgets don’t work that way at all. AppWidgets are built on top of Remote Views. According to the Android documentation, Remote Views are

A class that describes a view hierarchy that can be displayed in another process. The hierarchy is inflated from a layout resource file, and this class provides some basic operations for modifying the content of the inflated hierarchy.

A Remote View is created in one process and passed into the process that owns the Android home screen. It is actually a Parcelable object, however due to class loading issues, there is only a very small number of Views and Widgets that are allowed to be passed across the process boundary. For anything that involves relatively complex graphical rendering, the only real way to drive the widget’s contents is by specifying a very simple widget layout:

and then sending bitmaps to the image view:

	private void updateWidget(Context context, AppWidgetManager appWidgetManager) {
		Bitmap bitmap = Bitmap.createBitmap(100, 100, Config.ARGB_8888);
		Canvas canvas = new Canvas(bitmap);

		this.drawable.draw(canvas);

		RemoteViews views = new RemoteViews(context.getPackageName(), R.layout.main);
		views.setImageViewBitmap(R.id.imageview, bitmap);

		ComponentName componentName = new ComponentName(context, MyAppWidgetProvider.class);
		appWidgetManager.updateAppWidget(componentName, views);
	}

While this seems like a high overhead way to handle updates to the widget contents, I have to assume that the Android developers had a good reason for doing things like this. I can only hope that there are some tricks being done in the Android implementation that lower the cost of this operation. Given that I’ve just managed to get this to work at all, I imagine there are is a lot of room for improvement in my use of this API. However, I found it confusing enough to figure out how to do and thought others might benefit from my pain.

In reading the Android supports-screen documentation, it is clear that it is necessary to set the xlargeScreens attribute to true. However, the xlargeScreens attribute is not supported below API level 9. Trying to shoehorn that attribute into my project that was attempting to support back to API level 5, resulted in the following error.

XLargeScreens Attribute Failure

With a bit of finagling, I was able to get things working. In order to allow the xlargeScreens attribute, it is necessary to specify a target SDK version of at least 9.

XLargeScreens Working

This screenshot shows how the minimum SDK version can be set below version 9 and the target version is set to 9, allowing the xlargeScreens attribute to be specified. In addition, it is necessary to change the Android version level in the project properties.

XLargeScreens Properties

With the project properties set to use API level 9 there does not appear to be any automated way to restrict access to API that was older than version 9. Because of this, I do worry about choosing Android API’s that will not work at the minimum SDK and will fail on-device. My plan at this point is to switch back to building primarily for low end and switch once in a while to try on my Xoom. If I were a bit more serious, I would probably handle this automatically as part of a build script.

I do wish that Google had handled things differently in regard to how this works.

Once it was clear that the correct tiles were being found and that the image alteration was working, the next step was to calculate the correct alterations to match the digger’s location as shown as red in this video.

Finally, terrain destruction was completed by clearing those same image pixels to transparent resulting in the following.

Clearing to Transparent

Clearing the image pixels to transparent turned out to be a bit trickier than I had guessed it would be. The default paint “transfer mode” is such that painting with a transparent color results in no changes to the image. In order to erase the image to transparency, the transfer mode needs to be changed like the following:

		// Set up a paint that can be used to clear pixels from a ground tile
		Paint paint = new Paint();
		paint.setColor(Color.TRANSPARENT);
		paint.setXfermode(new PorterDuffXfermode(PorterDuff.Mode.CLEAR));
		paint.setStrokeWidth(1);

With the paint set to CLEAR mode, the Android graphics functions can then be used to alter the image pixels.

New Things Uncovered

I recently picked up a Motorola Xoom that I have also thrown AndPingus on to. It showed me that there are a couple of pretty interesting issues with the current implementation:

• Proper Speed Scaling
  The digger handler doesn’t properly account for clock speed and digging happens way too fast on something as fast as the Xoom.
• Extra Large Screens
  Recent Android versions introduced extra large screen support. Unfortunately, AndPingus isn’t correctly utilizing the screen size yet. I’m still trying to understand how to properly handle older devices at the same time as the extra large screen size.

Planned Source Release

I’ve decided that I’m going to go ahead and release a couple of utility pieces of the AndPingus source code as open source for others to take advantage of. My plan is to make available the following pieces:

• QuadTree
  I created a QuadTree implementation for searching the sprites. At the moment, because of the ground tiles, that code is not being used. However, it seems like it may useful to others.
• Drawable Texture Support
  This is the underlying implementation of the destroyable terrain implementation in AndPingus.

As I mentioned before, I’m getting busy with soccer coaching these days, so I can’t offer a specific timeframe for the release. Before I can make that happen, I need to decide on appropriate licensing and hosting options. In addition, I need to do at least a bit of cleanup before unleashing it to others. Stay tuned to this blog for more information when it becomes available.

I added support to separate the sprite images when a sprite needed to be altered, but it took me a while to realize I was forgetting to set the correct position for the newly created sprite. This resulted in this confusing result.

Once I realized that the issue was due to incorrect image/sprite generation, I had a much better result.

It became clear very quickly that my current approach for ground sprites was not going to work very well. Using the same images scaled and rotated in various ways makes it very difficult to find the correct sprite and, as you can see from the red X’s, it also means that many sprite images may need to be altered in the course of digging out a particular chunk of terrain.

Ground Tiles

To improve this such that the sprites would be more aligned and easier to deal with, I’m switching to using pre-generated ground tiles. The tooling that currently generates the collision map has been extended to generate a set of square ground tiles. Starting with an image that contains all of the ground sprites:

Full Ground Image

This image can be cropped down to include only the non-transparent area:

Cropped Ground Image

Finally, it is broken down into individual tiles:

Ground Tiles

Each non-transparent tile is stored individually. A new ground tile map object tracks the images and transparent tiles. At the moment, the tiles are being generated as 128×128 pixel images, which plays well with the OpenGL requirement that textures must be sized as a power of 2. Dependent on the maximum texture size, multiple ground tiles may be laid out within the texture with a minimum wasted space. The trick will be to pick an appropriate size to balance the various costs involved in loading and manipulating the sprite textures when destruction occurs.

While I had hoped to actually show this work via video in this post, I’ve run up against a bit of a roadblock. While fixing one problem, I’ve introduced another issue that I can’t seem to resolve. At this point it is better for me to walk away from this project for a few days and come back with a fresh set of eyes. With any hope, my next entry will show a final working destroyable terrain implementation.

In Early Digger Support I covered the initial digger support. At that point I had managed to update the in-memory collision map, but updating the actual textures driving AndEngine was proving to be a bit more difficult. I’m still not there, but I think I’m moving in a positive direction. The following video shows the current state of things. The textures are being updated with a full red fill to make it clear that they have been hit.

So, why is everything turning red? Well, that turns out to be the next item that will need to be dealt with… shared textures. To save memory, many of the sprites share common textures and texture regions. Thus, in the current implementation, changing the underlying texture information affects all sprites that share that information. This is something I knew would have to be dealt with eventually, so it appears that eventually is now.

Quad Trees

When I initially started playing with altering the texture data, I was worried about performance. My first attempt to locate the sprites to be altered used the standard AndEngine functionality to query collisions using the “collidesWith” method for shapes. This proved to be really expensive for gross-level collision detection. My performance tests using the built in Android tools for capturing trace data showed that much of the cost of the terrain destruction was accountable to simply finding the sprite to be altered.

I had heard previously about the use of Octrees in 3D to help do quick searches on the boundaries of objects. In the 2D world, Quadtrees are used instead. I was surprised not to find an actual Quadtree implementation on the web, but was able to piece together a nice generic implementation based on lots of research. With the Quadtree, I was able to get closer to reasonable performance, as you can see in the video capture. My hope is that using a Quadtree and doing the necessary cloning to split sprite textures will lead to a reasonably performant destroyable terrain implementation, but that is still yet to be seen.

Java Generics Aside

My Quadtree implementation initially was built to accept a single object type. It seemed more useful to use Java Generics to make the Quadtree more generally useful. I was hung up by one thing though. I wanted to be able to allow the Quadtree to accept objects with a certain interface declaration. Basically, I wanted this:

public class Quadtree<T implements IBoundedObject> {

Where IBoundedObject is simply defined as:

public interface IBoundedObject {
	Rect getBounds();
}

However, the implements extension is not supported by the generics syntax. This had me confused for a while until I realized that it is possible to do what I wanted to do, but needed to specify extends:

public class Quadtree<T extends IBoundedObject> {

I’m sure there is some perfectly good technical reason for doing things this way, but personally I find the lack of consistency confusing and unnecessary.

Next Time

With any luck, I will be able to show a reasonably performant implementation of destroyable terrain by pulling the various pieces together.

The next step was to start implementing some behaviors for the Pingus. The digger behavior seemed a good place to start. In order to implement the digger, it is necessary to actually alter the collision map generated by the tool. In the end, this part was pretty easy to handle. The results are captured in the video capture.

While it was relatively easy to carve out a path through the in-memory collision map, updating the actual graphics is proving to be much more difficult. AndEngine implements 2D graphics using 3D/OpenGL. This implies that in order to update the graphics, the underlying texture images need to be updated. I’m in the process of building AndEngine support for altering the underlying texture images. At the moment, this appears to be slow and may need to be abandoned. Just like while I worked on the collision map, the lack of guarantee for clipping and Z buffer on Android devices further complicate the situation.

While there are times that I wonder if using AndEngine for this project is makes it more difficult, I’m not quite ready to give it up. More to come…

07-04 12:23:45.101: ERROR/copybit(2230): Error in VIDIOC_STREAMON
07-04 12:23:45.101: ERROR/copybit(2230): Fail : v4l2_stream_on()
07-04 12:23:45.106: ERROR/copybit(2230): stretch_copybit::sec_stretch fail : ret=-5

This had me stymied for a bit until I took a look at the awesome zxing (Zebra Crossing) project where I stumbled on to a solution for this problem. (All credit goes to their developers). The code in their CameraManager class and CameraConfigurationManager class goes through a number of gyrations to pick an appropriate preview size. Using that code allows the Captivate to properly render a live preview.

Thanks to the zxing crew for this little tidbit, as I’m not sure I would have figured this one out on my own.