Front-end Performance in Firefox 3.5 and Beyond

One of the major changes in Firefox 3 was the amount of browsing history saved. In Firefox 2, the default amount of history saved was 9 days, so just over a week. The default in Firefox 3 is 180 days. Or 6 months. Or *20 times* the amount that Firefox 2 stored. However you slice it, it’s a far greater amount of data than was previously stored. An example benefit is that we can use this set of data to more accurately predict what you’re searching for in the Awesomebar. However, the challenge is to keep the browser *fast*, at least as fast as if we still only stored 9 days, and not only in the Awesomebar, but everywhere and anywhere that this data is exposed.

Firefox 3.5 contains a large number of internal changes to how we store, manage and display this data, in order to improve performance. Shawn Wilsher and Marco Bonardo spent an immense amount of time and effort altering almost every SQL query in Places, the bookmark and history infrastructure, so that it no longer needs to write to the database every time you visit a page, a project well chronicled on Shawn’s blog. Spurred on by one of Shawn’s performance wins, Ed Lee was able to drastically speed up searching in the Awesomebar. Even now, we’re working on changes that will vastly improve the performance of the history menu, sidebar, and smart folders, as well as just browsing around the Library. To try out some of these changes, download the latest beta of Firefox 3.5.

This work has been aided by better visibility and diagnosis of performance regressions due to tools such as Shark and DTrace, and all the work Alice and others have done in performance monitoring on Tinderbox. Also, Shawn’s work on mozStorage, the SQLite interface in Firefox, has provided us with reporting of poorly performing queries in debug builds.

A recent effort to gather real world usage data is the Places Statistics project, run by Drew Willcoxon and David Dahl. Drew created a script that culled statistical information about browsing behavior from the database that stores a user’s history and bookmarks. Users who wish to participate can anonymously submit their statistics to a website that aggregates the data. David wrote a Python tool that generates databases based on the aggregate data, that we can use as canonical datasets to test against. One of the first ways we’ll put this data to use is to run our current performance test suite against minimum, average and maximum sized databases generated by this tool.

Firefox will never be fast enough, so the work continues into Firefox.next. We’re using asynchronous SQL queries to move more database activity into the background, so as to not block the browser UI. This will surface in the Awesomebar (testable via Shawn’s extension) and Drew Willcoxon’s work on the Places query system.

You can help: If you think something is too slow, please file a bug, and add “[TSnappiness]” to bug’s whiteboard field, or in your report. Here’s the current list of TSnappiness bugs.

[UPDATE: Thanks jst for pointing out my humble interpretation of 180 days as only 3 months]