Dec
28
Mothra Upgrade Postmortem
The Mothra upgrade (3.1.0) was fairly exciting, but not in a good way. The good news is that our updating system works very well. Within 5 hours of release 25% of our users had upgraded from 3.0.4 to 3.1.0. The bad news is that we quickly realized that there was a strange bug in 3.1.0 and released 3.1.1 and subsequently 3.1.2 the next morning (sorry about that).
The day after we released 3.1.1 and 3.1.2 only 6% of our users were still using the problematic 3.1.0 version and by the next day only 1.3% were still using 3.1.0. If we keep getting traffic from 3.1.0 users we can assume the bug didn’t affect their installs (i.e. we think it only affected 10% of users anyway). As I mentioned before, the good news is that our updating mechanism on ShopSavvy is effective. One of the biggest challenges for remote, native applications - like web browsers is updating (Microsoft still wishes people would finally upgrade past IE 5).
Of course, the data generated by our application is one thing, actual communication from our users is another thing. Almost immediately after the update we began getting emails explaining we had broken user’s phones - that ShopSavvy no longer worked. Some users feedback:
- “Why have you ruined Christmas?”
- “You guys suck!”
- “Why didn’t you test this before releasing, noobs!”
- “I hate you!”
- “DIE!!!!!!!!!!”
We received complaints from around 30 unique users, which isn’t many when you consider our user base consisting of tens of thousands of people. But let me tell you, it seemed like the sky was falling. I was freaking out. Here is the scoop: Fifteen of these users were running 3.1.1 when they complained (one was running our 3.0.4 beta). Updating their application resolved their issues. Their main issues revolved around a black screen after scanning an item resulting in a freeze of the application. The good news is that we haven’t received a single email complaint from a 3.1.2 user - seems like it is stable and hopefully bug-lite. There is one user who had an issue upgrading from 3.0.4 to 3.1.2, but we think this might have something to do with the camera driver optimizations we had in the beta and removed in 3.1.1. We will continue to keep an eye on this issue.
The results: 10% of our users experienced a problem with 3.1.0, but within a day 99% of our users were no longer experiencing any issues. 3.1.2 seems like a big improvement over 3.0.4’s wish list and memory bug reports. Let us know what you think…
I hate to be the bearer of bad news, but I’m running 3.1.2 and still am unable to even get to any of my wishlist data. Whenever I click the Wishlist link on the main menu or try to save something to a wishlist, it only gives me the option to force close. Let me know what I can do to help troubleshoot the issue.
Comment by Sean Mc. — December 28, 2008 @ 8:49 pm
Well, I take that back. I uninstalled and reinstalled (was holding off and figured I’d go ahead and give it a whirl) and that fixed the issue, at the cost of my saved wishlists. I’m going to assume my wishlist data got corrupted somehow. Still a great program. And I’ll have fun rebuilding those lists. =]
Comment by Sean Mc. — December 28, 2008 @ 8:55 pm
Sorry about that. The wishlists should now be stable. We will soon have a feature that allows for export and import of the lists. I don’t think we ever realized how important the wishlists would be…
Comment by Big in Japan — December 29, 2008 @ 1:17 am