Monday, December 15, 2008

uTEST BROWSER REPORT: CHROME IS THE BUGGIEST


As Firefox and Chrome battle it out with new betas and rumored release dates, a newly completed analysis is answering the question over which browser has the most bugs.
Mozilla released Firefox 3.1 beta 2 this week and is now working on a third beta set to debut in early January. Google executives, meanwhile, reportedly indicated during a Web conference interview that Chrome is on the brink of coming out of beta altogether.
Testing, 1, 2, 3
In contrast to what those timelines might suggest, Chrome is being called the buggiest browser of all in a review released Wednesday. Open software testing community uTest used its crowdsource-based model to sign up 1,300 volunteer testers for the project. Each tester compared Firefox 3.1's first beta with Chrome's third beta (0.3.154.9) and Internet Explorer 8 beta 2. Neither Opera nor Safari was included in the analysis.
The testers competed for eight days to see who could uncover the most problems. uTest then verified and compiled the results to determine how each browser fared.
General Findings
The test's overall message is positive: uTest uncovered no major security flaw in any of the three browsers covered. "We're very encouraged by it," says spokesperson Matt Johnston. "In years past, we wouldn't have been able to say that."
The community did, however, discover 672 bugs across the browsers -- an average of almost one bug found for every 15 minutes of testing. All were user interface-related functional or technical issues, Johnston says. One bug, for example, caused a newly opened tab to have faulty color properties. Another caused a blip when trying to import favorites from another browser.
"They were general usability and functional things," Johnston explains.
Browser Breakdown
So here's the breakdown, browser-by-browser:
• Internet Explorer 8 beta 2: 168 bugs identified
• Firefox 3.1 beta 1: 207 bugs identified
• Chrome 0.3.154.9: 297 bugs identified
Among those bugs, 101 were labeled as "showstoppers," or more serious flaws that warrant immediate attention. Firefox had the highest percentage of those, at 24 percent. Twelve percent of Chrome's bugs were deemed "showstoppers," while 9 percent of IE's received the designation.
As for overall usability, Firefox had the highest number of "excellent" ratings, followed by Chrome, then IE -- which did not receive a single "excellent" score. Firefox also came out on top in ratings for general preference.
(The uTest community, by the way, is by its own reporting a fairly tech-savvy crowd. Nearly 80 percent of the members list "software tester" as their full-time professions.)
Putting It All Together
The Chrome-tarnishing results may not be entirely surprising. Chrome -- the newest browser of the three -- has seen numerous bugs and compatibility issues since its debut. Still, each release has made the product more stable than the last. And though Chrome has the smallest market share of the three browsers included here, Google has suggested its biggest push won't start until the software is deemed stable enough for a full release. Then, recent reports indicate, the company will "throw [its] weight behind it," heavily promoting Chrome and possibly even inking deals to have it preinstalled on new PCs.
With the notion that such a full release could be right around the corner, though, one has to hope Google has one serious update ready to roll out. Otherwise, if the uTest results are to be believed, that first full version may still be a bit too buggy for the masses.

JR Raphael

Stumble Upon Toolbar

3 comments:

Anonymous said...

This is a very interesting test carried out to determine which browser had the most bugs. Now I understand better why the preference on the www still goes for IE, followed by firefox and then safari.

BTW, are we in telepathy mode here:) I just released a post on browser market share today. Maybe you want to check it out. Together with your article, I believe people will understand better why Chrome has not yet conquered a lot of fans.

Fantastic post.

Anonymous said...

That is not surprising since Chrome is still in beta version. I'm using Chrome because it seems to be the fastest.

Anonymous said...

@ditesco..considering that IE and FF are the old guards in the browser line and each have their own loyal followers.
@ham..you're right, ham, and that's what betas are for,feeling through for bugs and users' reactions. and as we know this is chrome's first entry in the browser market.
well, this uTest report is very timely as google officially announced that chrome's final version is about to be release.

Search Engine Spider Simulator

Enter URL to Spider