There are also very few benchmarks that are capable of testing the speed of these two phone operating systems without bias one way or the other. So how do you compare them? One way would be to run both on identical virtual machines to see what you get. We thought about this and would be able to get our hands on a VM image for Android 2.2 (possibly 3.0) but for the iPhone OS we would need a little over $500 to achieve this. We would also be required to sign an NDA agreement that would prevent us from using the VM image in this manner and contains serious legal consequences for doing so.
We are back to square one. I guess there is almost no way to test these two head to head. So let’s talk about subjective performance. This type of performance testing is a very iffy topic. There are no fixed numbers for comparison; instead user experience is the key here. To test this type of performance we took three users that had been iPhone users for more than two years and gave them each an Android based phone to see how Android stacked up to the iOS. My users broke down into three categories; Power User/Tinkerer (this would be me), Average User, Teen/Media Oriented User. We looked at five aspects of the user experience to form our final results. These aspects were; General Speed, Web Performance, Customization, Media Experience/Gaming and Battery Life. This article is not meant as a full blown review and there are very few numbers listed, we are looking at a general in use comparison based on our three user profiles and some common usage models.
The test platforms -
Our three test platforms were broken down into popular styles of Android based on the user type. I chose two HTC EVO 4G phones for their power (CPU and Memory) claims of multimedia performance and gaming. The last choice was one that was found to be very popular amongst the teenage group this is the Samsung Intercept, we heard very good things about its performance as a media phone and also for its speed in texting and messaging. The two EVOs were running Android 2.2 (HTC/Sprint’s Over the Air Update) and used the HTC Sense UI. The Intercept remained on Android 2.1 as at the time of this writing no update for this phone was available. During this test we were unable to play with the 4G functions on the EVO as the closest 4G stations are about 4-5 hours from our lab. We have heard from Sprint that 4G validation and testing will begin locally in the next couple of months with the system going live by the end of summer 2011. We will be taking a look at 4G speeds later when we have time to linger in a 4G “live” area for an extended period of time.
The Tests -
The way we setup the testing was to gauge actual usage over a period of about three weeks. Part of the testing period included a “getting to know Android” time as each of the users had just given up use of the iPhone. From there we wanted to see how intuitive Android is in addition to how fast and functional the Google phone OS is. By using three different user types we were able to get a much broader feel for the OS in comparison to the iOS. So let’s dive in and see what we found out.
General Speed -
For general speed I have to say that the iPhone seems to have the upper hand here. Even with an update to Android 2.2 and cleaning up the memory in use; similar apps between the two OSes seem to open slower on Android. Even on the EVO with its Snapdragon CPU things were a tad slower to open and on the Intercept it was more pronounced. At times the stock music application would lag between inputs on the keyboard or from the touch screen. To give you an apples-to-apples example, launching Atomic Web on the iPhone (4 or 3G S) takes about 2 seconds. Launching the default web browser on the Intercept takes around 3-4 seconds with some pages not displaying. The Evo takes less time but still comes in just behind the iPhone at 3 seconds average for a non-cached page. The Facebook app for Android also seems to be slower. This was noticed by our average and teen users who both complained that the app did not refresh properly and could take as many as 5-6 seconds to pull up information and that was if you hit refresh as soon as you opened the app. There were many other examples of this type of behavior with apps from MSN Messenger, Twitter and others that lagged a little behind the iPhone apps by a second on the Evo and 1-3 on the intercept. There was also an issue with the Intercept and the screen; for some reason the screen would not register taps, it needed a fairly firm touch. This was a problem for the Teen user and we heard complaints of, “the stupid thing won’t work” when trying to play music and videos.
Web Performance -
The first thing that I need to qualify here is that as of the time of this writing Sprint’s CMDA network cannot do both voice and data together when 3G is is all that is available. Because of this we have found that all three Sprint phones lag when connecting to the internet and a non-cached page. This is a fault of the carrier and not the OS. Still for our testing purposes we heard grumblings from both the average user and the teen. These complaints were that many pages are very slow to load. Pages like MSN.com, Live.com, BSN, TweakTown and of course our site here DecryptedTech.com can take a very long time to load and render. Now, that having been said, there is nothing like being able to actually view and USE flash on the Android. It was nice not see animations and videos visible on the pages that were being rendered. I was able to hit up YouTube without the need for a secondary application, embedded videos were visible and playable in all pages. Of course this was not available on the Intercept (and won’t be until 2.2 is available) but the pages were still more “live” than we see on the HTML5 reliant Apple devices (even with Frash installed). So this one is a toss-up; the iPhone is a little faster, but the Android devices were a much better experience.
Customization -
Here, well there simply is no competition. The Android even an un-rooted Android phone is 100 times more customizable than any of the stock iPhones. You can download and install themes, music players, browsers, UIs, Icons, you name it you can do it. With the iPhone, unless you jailbreak it (and continue to jailbreak it with each new update) you are stuck with what Apple gives you. Yes you can download a couple of new browsers and there are even a few additional video players, but for the most part you are stuck. I did not like the stock text messaging app that came with the Android so I grabbed a new one and have that set as the default. Try that on an un-jailbroken iPhone. The same thing with the actual UI, I grew tired of the Sense UI and launcher screen so I use the one I like. These are things you just cannot do on the iPhone without jailbreaking and you are still stuck within a certain framework. I can run UI’s and apps on top of the UI that mimic extra screens (or mimic a sandboxed Multitasking layer). There is just so much more you can do with an Android based phone than you can with the iPhone.
Media Experience/Gaming -
This was a tough one as well. The screen on the iPhone 3G S was pretty good. It was not great but it was good. The iPhone 4’s screen was excellent the sharpness of the 720p videos that were rented, purchased or ripped (using the Videora and the Apple TV settings) looked great on those screens. The iPod and Video apps (really a full time subsystem) are also very easy to use and perform as they should. I was not a fan of breaking out the video and the music sections of the iOS, but I can see the reasons behind it. Still, there is nothing wrong with either of these apps when in use, they are fast and responsive. The UI is clean and easy to navigate. The problem; how you get them on the phone. Being forced to use iTunes for everything means having to jump through several DRM hoops just to get content you may already own on your phone. I do not want to buy iTunes digital copies. I want to use ones I have that are not tied to the Apple name and logo, there are ways around this but again, you are always fighting to keep up with Apple as they work very hard to lock down the ecosystem. This behavior is good for Apple but for someone like me, it stinks. I simply despise iTunes. It is bloated resource hog that slows down my system, refuses to sync with outlook 2010 and continuously wants to try and shove Safari down my throat.
With all of the Android devices we tested getting music and video onto your phone was as simple as drag and drop. You connect the phone, tell it to operate in drive mode and drag the files to the removable disk that pops up. There is no need to install any software at all. I dragged the same 720p HD videos that I played on the iPad and iPhone 4 to the EVO and without a hitch they were visible and playable. The picture on the much larger 4.3-inch 800x480 screen (Vs. the iPhone 4’s 3.5-inch 960x640) gave a better representation of colors even at the lower resolution. It was also easier to watch most movies on the EVO Vs the iPhone 4 or iPhone 3G S. Now, on the Intercept with its 3.0-inch screen things were a little different. The resolution here is 240x400 and the screen is a simple TFT/LCD unlike the Retina display on the iPhone 4. The color saturation is very similar between the phones but that can also be a function of the surface. When I put an anti-glare screen protector on the Evo the clarity and sharpness dropped, with that removed; it all returned. The customization aspect plays in here as well. As I mentioned above our teen user was not happy with her player. Because of the ability to customize the Android experience completely we were able to easily replace the music player with one that was more to her liking.
Battery Life -
Sorry Android users; the iPhone does beat many of the Android based phones and certainly did win out over the three we used for this subjective comparison. Put simply, the Evo has poor battery life. I cannot go a full day without needing to charge the phone. On the iPhone I was usually able to get about a day and a half of battery life. Of course this was dependent on the signal strength. When I first bought the iPhone 4 the battery would drain quite quickly. This seemed to be due to a lack of proper signal as I could also see the signal constantly fluctuate between no bars and five bars. This will always have an adverse effect on battery. This was mitigated somewhat with the first iOS 4.0 update, but was not completely gone. The battery life did improve though. The Evo has a very similar issue, but it also has a ton of items running when using the stock HTC Sense UI. You can extend the battery life a little using applications like Advanced Task Killer (which even allows you to kill core phone tasks), but you are still not getting battery life near what the iPhones get. We see the same thing with the Samsung Intercept. The battery just does not last as long. The one thing you really have going for you with most of the Android based phones is that you can buy an extended battery. With Apple you are again; stuck with what Steve gives you.
Final Thoughts -
To sort of wrap things up here, we are seeing that while the iPhone can be faster in many cases it lacks the ability to really give the end user their own experience. Our Teen user still laments the loss of her iPhone and complains about the Facebook app, but finds new things to tinker with on the Intercept almost daily. Our average user was unhappy with the Sense UI, but now that it has been replaced with items like Launcher Pro, Quick Desk, Beautiful Widgets and a host of other customized features and default apps the Evo is much more to her liking. I even managed to find a lockscreen that emulates the slider from the iPhone instead of the drag down style one that is default on the Evo. It is the ability to change little things like that that really makes the Android a much better value even for the mainstream consumer. A phone based on Android (even one that is not rooted) truly is your phone. Meanwhile the iPhone, while faster in some respects, remains very firmly Steve’s Phone. This is even truer since the latest update to the iOS 4 has apparently removed the typical exploit used for jailbreaking. It might be legal to jailbreak your phone, but Apple is going to prevent you from doing it anyway. They continue to tighten the noose and keep their leech-like hold on your credit card through that money draining software called iTunes. Although I still maintain it for my iPad (which I rarely use now), I will be happy on the day I can pick up an Android, Ubuntu, or Windows based Tablet and put that thing on e-bay. Sorry to the Apple fans, but I have become too used to being able to do what I want with my systems, I balk at the control Apple has on their mobile devices just like many consumers are beginning to with more joining the crowd almost daily. Maybe one day Apple will loosen their grip (and AT&T will have a better network) but even with the iPhone coming to Verizon (another nickel and dime carrier) I cannot see being tethered to Apple’s hip. Not after getting a taste of the freedom and performance that can be available with an Android based phone. We should caution you though, not every Android phone will be a great performer. We found this out with the hardware on the Intercept. It is limited by some of the hardware choices made by Samsung to reduce price. You can still get some of the speed by killing off a few start up and non-essential tasks but there is no getting around the screen issue. Still after some heavy customization (by yours truly) our teen user is much happier with her Android phone and the statements of “I miss my iPhone” are happening less and less often.
