The Value of the Beta Test

Before I start getting into the whole Windows 8.1 App thing, I decided to bang of a few quick Windows Phone apps.  After all, the Developer Movement is in play in Canada and there are some mighty good things up for grabs.  The first three I did were actually pretty simple and straight forward.  They had some pretty cool bits like integration with Windows Azure and use of the Nokia Imaging SDK.  It's pretty cool how so much of the heavy lifting for Windows Phone has been done for you.  You can focus on the idea and the design.

But that's not I'm talking about today.  It's the 4th one I'm focusing on today.  I actually received a great idea based on my Track My Trip app (please note that info from customers is GREATLY valuable).  So I started building the app.  Now, I had no use for the app suggested but like to think I can build any idea somebody comes up with.  So I built it.  It was REALLY cool.  It did all sorts of things and was really cool about doing it.  I tested it (it seemed to work) so I was ready to push it out to the world for, no doubt, universal adoration and respect for the awesomeness I had wrought.

This particular customer asked if he could beta test the app.  Hey, that sounded like a good idea, why not let him be the first to praise my work. Well, turns out that what I THOUGHT was cool was just a pain in the butt.  A good beta tester that provides lots of feedback on what they like and don't like and will use the app is gold, gold, gold.  I learned very quickly (and have probably known a very long time really) that users do not ever use a piece of software the way it was originally designed. 

There are two ways to deal with a beta.  The first is through the store.  When you go to the Windows Phone Dev Center and "Submit a new App" at the bottom of step one is a "More Options" drop down.  If you open this you will see that you can publish your app as a beta.  When you choose this option and put in the Microsoft Ids of your beta testers (always include your self so you can try it first) the app will be published within hours rather than days.  Only those users with a designated Microsoft Id can download your app.  You will receive an email with a deep link to the beta that you can distribute to your testers.

One caveat to be aware of is that it has been my experience that "updating" a beta doesn't seem to work properly.  If a beta tester had previously downloaded to their phone version 1.0.0.0 and you publish an updated XAP of version 1.0.1.0, no matter what that user does, they will never get anything other than version 1.0.0.0 for some reason.  So you need to publish a new beta, then redistribute the deep link to the new app and the users will need to uninstall the previous one.

NOTE:  Make sure that the version of your app is visible somewhere in the app so they can be sure what version they are testing for reporting purposes.

The second method only works if your beta tester is an app developer themselves and you have a solid trust relationship between your tester and yourself.  Maybe co-workers and such would be ideal.  You can send them the XAP file and have them use the Windows Phone Deployment tool.

Either way, remember that feedback is important.  Feelings about your app are important.  If something doesn't "feel" right then it's probably not.  Make sure you focus in on what's important.  Know what you are trying to do and focus on that.  Don't let feature suggestions cause you to have feature-creep.  Keep it tight, and make it work.

Here's to great apps!