There are also very few benchmarks that are capable of testing the speed of these two phone operating systems without bias one way or the other. So how do you compare them? One way would be to run both on identical virtual machines to see what you get. We thought about this and would be able to get our hands on a VM image for Android 2.2 (possibly 3.0) but for the iPhone OS we would need a little over $500 to achieve this. We would also be required to sign an NDA agreement that would prevent us from using the VM image in this manner and contains serious legal consequences for doing so.
We are back to square one. I guess there is almost no way to test these two head to head. So let’s talk about subjective performance. This type of performance testing is a very iffy topic. There are no fixed numbers for comparison; instead user experience is the key here. To test this type of performance we took three users that had been iPhone users for more than two years and gave them each an Android based phone to see how Android stacked up to the iOS. My users broke down into three categories; Power User/Tinkerer (this would be me), Average User, Teen/Media Oriented User. We looked at five aspects of the user experience to form our final results. These aspects were; General Speed, Web Performance, Customization, Media Experience/Gaming and Battery Life. This article is not meant as a full blown review and there are very few numbers listed, we are looking at a general in use comparison based on our three user profiles and some common usage models.
The test platforms -
Our three test platforms were broken down into popular styles of Android based on the user type. I chose two HTC EVO 4G phones for their power (CPU and Memory) claims of multimedia performance and gaming. The last choice was one that was found to be very popular amongst the teenage group this is the Samsung Intercept, we heard very good things about its performance as a media phone and also for its speed in texting and messaging. The two EVOs were running Android 2.2 (HTC/Sprint’s Over the Air Update) and used the HTC Sense UI. The Intercept remained on Android 2.1 as at the time of this writing no update for this phone was available. During this test we were unable to play with the 4G functions on the EVO as the closest 4G stations are about 4-5 hours from our lab. We have heard from Sprint that 4G validation and testing will begin locally in the next couple of months with the system going live by the end of summer 2011. We will be taking a look at 4G speeds later when we have time to linger in a 4G “live” area for an extended period of time.
The Tests -
The way we setup the testing was to gauge actual usage over a period of about three weeks. Part of the testing period included a “getting to know Android” time as each of the users had just given up use of the iPhone. From there we wanted to see how intuitive Android is in addition to how fast and functional the Google phone OS is. By using three different user types we were able to get a much broader feel for the OS in comparison to the iOS. So let’s dive in and see what we found out.
General Speed -
For general speed I have to say that the iPhone seems to have the upper hand here. Even with an update to Android 2.2 and cleaning up the memory in use; similar apps between the two OSes seem to open slower on Android. Even on the EVO with its Snapdragon CPU things were a tad slower to open and on the Intercept it was more pronounced. At times the stock music application would lag between inputs on the keyboard or from the touch screen. To give you an apples-to-apples example, launching Atomic Web on the iPhone (4 or 3G S) takes about 2 seconds. Launching the default web browser on the Intercept takes around 3-4 seconds with some pages not displaying. The Evo takes less time but still comes in just behind the iPhone at 3 seconds average for a non-cached page. The Facebook app for Android also seems to be slower. This was noticed by our average and teen users who both complained that the app did not refresh properly and could take as many as 5-6 seconds to pull up information and that was if you hit refresh as soon as you opened the app. There were many other examples of this type of behavior with apps from MSN Messenger, Twitter and others that lagged a little behind the iPhone apps by a second on the Evo and 1-3 on the intercept. There was also an issue with the Intercept and the screen; for some reason the screen would not register taps, it needed a fairly firm touch. This was a problem for the Teen user and we heard complaints of, “the stupid thing won’t work” when trying to play music and videos.
Web Performance -
The first thing that I need to qualify here is that as of the time of this writing Sprint’s CMDA network cannot do both voice and data together when 3G is is all that is available. Because of this we have found that all three Sprint phones lag when connecting to the internet and a non-cached page. This is a fault of the carrier and not the OS. Still for our testing purposes we heard grumblings from both the average user and the teen. These complaints were that many pages are very slow to load. Pages like MSN.com, Live.com, BSN, TweakTown and of course our site here DecryptedTech.com can take a very long time to load and render. Now, that having been said, there is nothing like being able to actually view and USE flash on the Android. It was nice not see animations and videos visible on the pages that were being rendered. I was able to hit up YouTube without the need for a secondary application, embedded videos were visible and playable in all pages. Of course this was not available on the Intercept (and won’t be until 2.2 is available) but the pages were still more “live” than we see on the HTML5 reliant Apple devices (even with Frash installed). So this one is a toss-up; the iPhone is a little faster, but the Android devices were a much better experience.
