Front-end Performance in Firefox 3.5 and Beyond

One of the major changes in Firefox 3 was the amount of browsing history saved. In Firefox 2, the default amount of history saved was 9 days, so just over a week. The default in Firefox 3 is 180 days. Or 6 months. Or *20 times* the amount that Firefox 2 stored. However you slice it, it’s a far greater amount of data than was previously stored. An example benefit is that we can use this set of data to more accurately predict what you’re searching for in the Awesomebar. However, the challenge is to keep the browser *fast*, at least as fast as if we still only stored 9 days, and not only in the Awesomebar, but everywhere and anywhere that this data is exposed.

Firefox 3.5 contains a large number of internal changes to how we store, manage and display this data, in order to improve performance. Shawn Wilsher and Marco Bonardo spent an immense amount of time and effort altering almost every SQL query in Places, the bookmark and history infrastructure, so that it no longer needs to write to the database every time you visit a page, a project well chronicled on Shawn’s blog. Spurred on by one of Shawn’s performance wins, Ed Lee was able to drastically speed up searching in the Awesomebar. Even now, we’re working on changes that will vastly improve the performance of the history menu, sidebar, and smart folders, as well as just browsing around the Library. To try out some of these changes, download the latest beta of Firefox 3.5.

This work has been aided by better visibility and diagnosis of performance regressions due to tools such as Shark and DTrace, and all the work Alice and others have done in performance monitoring on Tinderbox. Also, Shawn’s work on mozStorage, the SQLite interface in Firefox, has provided us with reporting of poorly performing queries in debug builds.

A recent effort to gather real world usage data is the Places Statistics project, run by Drew Willcoxon and David Dahl. Drew created a script that culled statistical information about browsing behavior from the database that stores a user’s history and bookmarks. Users who wish to participate can anonymously submit their statistics to a website that aggregates the data. David wrote a Python tool that generates databases based on the aggregate data, that we can use as canonical datasets to test against. One of the first ways we’ll put this data to use is to run our current performance test suite against minimum, average and maximum sized databases generated by this tool.

Firefox will never be fast enough, so the work continues into Firefox.next. We’re using asynchronous SQL queries to move more database activity into the background, so as to not block the browser UI. This will surface in the Awesomebar (testable via Shawn’s extension) and Drew Willcoxon’s work on the Places query system.

You can help: If you think something is too slow, please file a bug, and add “[TSnappiness]” to bug’s whiteboard field, or in your report. Here’s the current list of TSnappiness bugs.

[UPDATE: Thanks jst for pointing out my humble interpretation of 180 days as only 3 months]


11 Comments on “Front-end Performance in Firefox 3.5 and Beyond”

  1. Brian says:

    Hopefully by the time 3.5 is in it’s final form it will be faster but right now, it’s not and I run the latest nightly builds of 3.5b5pre.

  2. Brian, can you say exactly what is not faster?

  3. Paul says:

    When the history storage capabilities are that large, why are they measured in days rather than by urls or some other metric?

    3 months of history for me is a dramatically different amount of data to 3 months of history for my mother.

  4. Zoltan says:

    Firefox on leopard (mac os x) is slow as hell, I upgraded my Mac to 4GB for a decent performance. I deactivated all plugins, the same problem.. start-up is slow compared to safari

  5. pd says:

    Fx 3.0.10 History and Library are too slow. It’s good to hear that plenty is being done about this. Will TraceMonkey itself help as well?

  6. Kim Sullivan says:

    What’s the difference between browser.history_expire_days_min and browser.history_expire_days? Why are they set to wildly conflicting values? I checked http://kb.mozillazine.org/Browser.history_expire_days and http://kb.mozillazine.org/Browser.history_expire_days_min but that doesn’t explain it.

  7. RyanVM says:

    TraceMonkey shouldn’t have any affect on 3.5 since it’s only enabled for content, not chrome. I suppose it *might* speed things up when chrome JIT is enabled, but it seems that SQLite is the bottleneck, not JS execution.