We quietly shipped Firefox 11 with a bunch of performance fixes that both reduce the amount of memory that Firefox uses, and improve the responsiveness of it’s user interface.
These types of changes are not easy to talk about. Often they’re very technical, and meaningless to anyone but the developers involved, which is probably why we usually don’t enumerate them in the the release notes or other public announcements. “Firefox is 74% faster when you open menu X, and twice as fast in some garbage collection scenarios!” Yeah, not an eye-popping headline. We could do a lot better in communicating these improvements in broadly meaningful ways though – nice graphs or some competitive site like arewefastyet would help a lot. But until then, here’s a short summary of the improvements in Firefox 11. And if you know of other performance fixes that don’t fall into the categories below, please add them in the comments!
Memory Use (aka “memshrink”)
The Memshrink project has been going for quite a while now, led by Nicholas Nethercote. He blogs weekly updates on the project’s activity. According to Bugzilla, there were 29 memshrink bugs marked fixed during the Firefox 11 development cycle – four of which were P1, or very high priority. Some of the fixes were related to tools and detection methods, but many are actual reductions in memory use. The changes that made it into Firefox 11 include fixes for detected leaks, removing of zombie compartments, lazy-loading data, reducing the size of some caches, reducing memory used while scrolling, and many more.
UI Responsiveness (aka “snappy”)
The Snappy project started last December, and is run by Taras Glek. Its aim is to improve the responsiveness of the Firefox UI. Taras has been posting weekly updates on Snappy activity on his blog. Bugzilla shows 15 snappy bugs marked fixed during the Firefox 11 development cycle. The project had just started, but there are still some significant wins in this release! Firefox 11 includes reductions in queries in the bookmarks system, reduced preference lookups, faster data collection for session restore, and various improvements in the DOM code.
While it’s not related to performance, I do want to call attention to something that many people don’t seem to know: In Firefox 10 (yes, the previous release) we stopped marking most add-ons incompatible when you upgrade. That means that a LOT more of your add-ons will continue to work when you upgrade Firefox from here on out. The only add-ons that still require compatibility bumps are those that have binary components, since they need to be recompiled against the current version.
My team is “a riddle, wrapped in a mystery, inside an enigma”. Ok, maybe not *that* mysterious, but we’re definitely involved in a variety of projects. Here’s a roll-up of a week in browser-land.
- Tim landed a patch that adds reporting of leaked windows and documents to our unit test runs. Dao’s work has shown that these often reflect leaks in the platform not just the test, so it’s very valuable to know about them. The next step is to track the leak numbers between runs and report regressions by making the tree color change somehow. Tim also blogged about this.
- Neil has a patch up for adding telemetry that measures how long it takes to open a browser window.
- Mano continued his work on the Safari migrator, which includes various cross-migrator improvements. This work is part of a broader rewrite of the migrators to use Places async APIs, for improved responsiveness.
- Marco landed a major rewrite of Livemarks, converting them to load content on-demand and asynchronously, reducing synchronous IO on the main thread.
- Paolo continued his work on rewriting favicon API consumers to use the new asynchronous API.
- Drew has completed the conversion of the content preferences service to use async mozStorage statements, and is in the process of updating the various JS and C++ consumers to use the new API.
- I talked a bit with Mark Cote about Peptest, a new framework for testing browser responsiveness. Peptest is currently available on the tryserver, but results are highly variable, so there’s more work to do before it can be turned on for all check-ins. Once Peptest is ready to use, we’ll evangelize the crap out of it.
- I continued working on breaking apart the session restore service into more manageable pieces (XPath generator, tab state, form data).
- I put up a WIP patch for reading the session file asynchronously during startup.
- I started working on measuring how much time is taken up by creating about:blank browsers when restoring sessions on demand.
New Tab Page
- Tim worked on Stephen Horlander’s redesign of the new-tab page, and also fixed some bugs.
New Download Manager
- Paolo consolidated existing patches into a single one for final pass and check-in. Marco has been reviewing.
- Paolo and Marco are getting together in Novara on Saturday to sprint on fixing theme bugs and other cleanup required for landing.
Add-on SDK Integration
- Met with Irakli to create plan for landing the core SDK modules in Firefox so that the whole SDK would no longer need to be bundled with every add-on.
- Created a feature page for tracking the project.
- I talked with Brian Warner about syncing Git repos with HG repos.
Web Apps Integration
- I updated the feature page with latest from UX, Apps and PM.
- Felipe Gomes is driving the Firefox side of the integration work, with Tim helping out on UI. They’re working together with Fabrice, Myk, Dan and Tim A.
- David has the window.crypto.getRandomValues patch up for final review and superreview.
- David put up an initial patch for the navigator.id API for Web content.
- David gets to play with NSS+DOM+XPCOM to get DOMCrypt’s internal API working. Lucky guy.
- David is working with Shane Caraveo to get the new Share add-on reviewed and landed.
- Tim and Marco kicked off sponsorship process for Italy’s jsDay conference. They’ll be putting up a booth there, along with Paolo Amadini, representing Mozilla. Thanks to Stormy and Shez for the support from Developer Engagement.
Of course, we all worked on various other things as well, from code review to bug triage to random maintenance fixes. Activity logs and whatnot are listed below.
- David Dahl: Bugzilla activity log, status updates, blog, Twitter
- Drew Willcoxon: Bugzilla activity log, status updates
- Felipe Gomes: Bugzilla activity log, status updates, blog, Twitter
- Asaf Romano: Bugzilla activity log
- Marco Bonardo: Bugzilla activity log, status updates, blog, Twitter
- Neil Deakin: Bugzilla activity log, status updates, blog
- Paolo Amadini: Bugzilla activity log
- Tim Taubert: Bugzilla activity log, status updates, blog, Twitter
- Dietrich Ayala: Bugzilla activity log, status updates, blog, Twitter
We’re at the end of our Performance work-week here in Brussels, and gearing up for a two-day orgy of European open-source culture at FOSDEM. I’ve successfully acquired a cold (and hopefully not worse) due to the temperature being consistently below freezing.
However, the people here in Brussels have made up for their weather shortcomings by welcoming us wherever we go. Between the hackerspaces and co-working spaces, and the restaurants that happily take large groups with little or no notice, I’m very impressed!
The hackerspace in Brussels is located in Schaerbeek, a neighborhood to the north of the city center. The space used to be a vehicle repair garage for the city, but was given up for use by the geeks. They’ve installed serious hardware, and have fully-equipped the place with everything needed for survival. Thanks to Rafael and Patrick, for answering all our questions and helping us make mate and to find food nearby. Lunch on the second day was described by Patrick as a “little French place”, but turned out to be a hall of worship dedicated to Tintin!
Faubourg St Antoine is filled with Tintin toys, art and knick-knacks, including some alternate interpretations and even a clarification for something I’d always wondered about. Sadly, they’ve been issued a legal notice from the current copyright (or EU equivalent) holders requiring them to remove all the Tintin materials from public display :(
Once the temperatures dropped far below freezing, we relocated to BetaGroup Coworking Brussels in Etterbeek, to the southeast of the center. Ramon Suarez, the manager of the space was very accommodating, taking us on short notice. The wi-fi was blazing fast, the coffee was hot, and the ping-pong was a welcome break from heads-down hackery. The space itself was fantastic, with a great combination of quiet co-working areas, public spaces and private meeting offices. With tons of natural light, steel bridges and a meeting space on what looked like a submarine conning tower, it was truly impressive.
We had a wonderful lunch at a very tidy restaurant nearby.
Overall, it’s been a fun and productive week, if a bit chilly. Like, really chilly. Ridiculously so. Why do people even inhabit places that get this cold? Honestly, wtf.
The Performance team and some of the Firefox team are spending the week in Brussels, laying waste to some of the performance issues in the browser.
I’ll be uploading pics to flickr with the tag ‘perfworkweek2012’.
I am needy:
- I want to remember URLs. Bookmarking is too manual and akin to throwing URLs in the sarlacc pit. The user-interface pieces around bookmarking have not changed in a decade. No, the awesomebar is not a good tool for this. I don’t even come close to being able to recall what I want the awesomebar to recall. I need to be ambiently prompted in a way that is visual and has context.
- I need to be able to focus on a given task, project or idea. A single sea of tabs doesn’t help at all. I want blinders. I want an environment. Task immersion.
- I need to be able to categorize URLs into groups, such that the whole group is easily accessible. Trees and menus can go to hell, along with the RSI, eye-strain and visual boredom they provide.
- I need to be able to switch contexts quickly and easily. Eg: From bug triage to perf to dashboards to music, etc.
- I don’t want to leave the browser. Windows are super heavyweight feeling and come along with all kinds of operating system baggage: visual, interaction, performance, etc.
I realized recently that a pattern had emerged in my browser usage that meets a bunch of these needs:
- I set up Firefox to restore my session every time it starts. This way my groups persist, and all the URLs in each group are loaded with their cookies and other session data ready to go when I need them.
- I have “Don’t load tabs until selected” checked, so that Firefox does all this with as little memory as possible – the web pages in all the tabs in all the groups aren’t loaded until I actually use them.
- I restart the browser a couple of times per day to keep memory use slim, which in turn keeps the browser responsive. Restarting is super fast and responsive because I have “Don’t load tabs until selected” (see previous point).
This is the happiest I’ve been with any browser in years. However, there are still a bunch of pain points. I want SO much more.
- I want to tag URLs without bookmarking them. The bookmark concept just gets in the way. It’s an unnecessary unit of psychological weight. It’s a vestigial metaphor of days gone by.
- I want to open a tab group by typing the name of the group in the URL bar.
- I want to add URLs to multiple groups easily, similar to tagging. I’d like to do it via the keyboard.
- I want to send the current tab to a different group (or groups) using only the keyboard.
- I want app tabs that are specific to a given group, and some that are global.
- I want to switch quickly from an app tab back to the last non-app tab I was at. Or be able to peek quickly at an app tab without losing my context in the current set of tabs.
- I want to switch quickly back to the last tab I was at. (Eg: When I open a new tab, and get sent to the end of the current set of tabs). OR be able to have new tabs open immediately to the right of the current tab, with linked parentage.
- I’m tired of sites being browsers inside a browser. And I don’t want “site-specific” browsers – I want a “me-specific” browser, for local or dynamic content.
- Firefox creates the <tab> elements for hidden tabs when restoring the session. It would re/start even faster and use even less memory if the XUL elements for hidden tabs were not created until the group itself was opened.
- As I work, memory use increases and responsiveness decreases, since I keep visiting more and more tabs. If I haven’t visited a tab in a while, Firefox should unload it. If I haven’t visited a group in a while, Firefox should completely unload the whole group, session content *and* XUL elements.
- A downside of the “Don’t load tabs until selected” option is that tab content is not ready and waiting when you select the tab. The web content has to load and the session data for the tab must be restored. Firefox should pre-load tabs that are adjacent to the active tab. This feature, combined with the dormant-izing of tabs described above would result in a decent balance of instant-gratification availability and responsiveness and resource frugality.
One idea I had was a merging of tagging and groups: The groups in Panorama would be comprised of the set of tags that exist. This would result in nice integration bits like search-by-tag in the awesomebar being equivalent to search-in-group. It also might mean that we’ll need to make Panorama “bigger” – maybe allow it to be zoomed, or make it an infinite canvas.
An idea for navigating dynamic content is to merge feeds and groups. Imagine you have a BBC group, which has the BBC feed as it’s source. The set of “tabs” in that group are the items in the feed. If you open the group, all the URLs in the feed are loaded into tabs (but not *really* loaded if you restore-on-demand).
Anyways, it’s interesting to think about how to prototype some of these ideas in an add-on or a collection of them. I’m sure some of the items above already exist as add-ons.
I realize that I’m not a “typical user”. However, after almost 6 years of browser-making, I’m pretty damn sure that there is no such person. I do not believe that the one-size-fits-all browser is the future. When adding a feature or fixing a bug, we shouldn’t have to choose between grandma and the geeks. In order to stay relevant in a highly-personalized future, we should strive to ensure that Firefox is pliable enough that we who make it are not restricted by it, and more importantly we must ensure that add-on developers are free to mash-up and remix and experiment the f*ck out of it.
Several people have asked me about living in Thailand. After spending a year in Chiang Mai, here are a few suggestions, recommendations and things I wished I knew ahead of time.
- Learn to speak Thai, at least a little bit. We took classes over Skype video for 2 months from Brett Whiteside and it really helped. I can speak and understand enough basics for cordial and respectful interactions with people, and to eat wherever I want :). If you’re going to be there for a while learn to read Thai. It’s seriously worth it, if only for reading menus and signs.
- Always print a detailed map of wherever you’re going. Anywhere. All the time. Really.
- Make a custom Google map with the locations of: post office, utility offices (power, water), telco, police station, etc. Just in case.
- Banking: ATMs cost US$5 per transaction, and result in currency conversion fees. Instead, open a Bangkok Bank account and get a debit card. Getting a bank account took some paperwork, about an hour of waiting in the bank, and I think you have to pay a small fee. Bangkok Bank is the only Thai bank with a US branch (in NY). This means that you can make direct transfers from US banks to your Bangkok Bank account, allowing for money access with a minimum of delay and fees.
- 3G: I used AIS, which has unlimited (5GB IIRC) 3G monthly pre-paid option. Absolutely recommended for access to Google Maps while lost ;).
- Thai street food is often safer than food from “western” places. The food at the carts is usually bought fresh that day, and is cooked in front of you. The food at “western” hotels and restaurants is cooked out of sight, and they try to store food, which is risky since there’s not really health department inspections like in the states, to ensure the storage is properly done. You’ll find infinite opinions about this – this is my personal experience. There’s no magic solution. You will have some intestinal discomfort at some point.
- Western toilets are everywhere. At some point however, you’ll need to use a squat toilet. Chiang Mai Rock Climbing Adventures has humorous yet informative instructions for properly using a squat toilet.
- Use Foursquare or some other check-in service. I often couldn’t find a place that I wanted to return to, or had forgotten the street, etc.
- 7-11. Is awesome. In Thailand you will pay your utility bills, refill your phone minutes, purchase concert tickets, and many other things there.
- Temples: Wear long pants and shirts with sleeves. Be quiet and respectful. Donate money to them.
Most importantly: Go anywhere and everywhere. Always take a different route. Pull over often. Walk down unfamiliar streets. See every temple. Explore every market. And always carry your camera.
This year we started talking about ways to improve the methods in which develop Firefox features. This is a snapshot of where we’re at, and what’s coming next.
We began with some conversations about current problems regarding coordination, speed, contribution, regressions, ease of development: my initial dev.planning post, Joe Walker’s post ‘How to Eat an Elephant’, my blog post, dev.planning add-on bundling post.
There was general recognition that the Firefox development model is not agile nor flexible enough to meet the needs of our 2012 goals. We need to be able to ship features that are developed mostly outside of what’s considered the “core” Firefox team. We need to better support the use of external code repositories and bug tracking systems. We need to support features written using the Add-on SDK, both inside and bundled with Firefox.
Here’s where we’re at:
- A group of people formed to start discussing how to make these things happen: the Jetpack Features group. We meet every other Friday (meeting details and notes).
- We have a project branch (Cedar) where we can experiment and do performance testing.
- We corralled the Firefox and Toolkit module owners and the leads of the SDK project in a room to orchestrate the mechanics of landing the SDK inside Firefox itself (notes).
- L10n: The Jetpack team are hard at work on closing this gap with both localized strings in script and localization of HTML content inside add-ons.
- Performance: Jeff Griffiths blogged recently about the performance impact of the core SDK runtime on startup time. More performance tests are coming.
- Feature bundling: The BrowserID feature will likely ship in Firefox as a bundled add-on. There are still open questions though: Is it uninstallable? Does it even show up in the Add-ons Manager? Can it upgrade outside of the Firefox dev cycle? More work to do here.
- Automation: We have the Jetpack unit tests being run with every Mozilla-central check-in (though hidden by default currently). We have the SDK unit tests running on every SDK check-in for both Nightly and Aurora (tree). We still need the Mozilla-central unit tests run on every SDK check-in (that’ll come with integrating the SDK into Firefox), and performance tests run for Mozilla-central and SDK check-ins.
What’s coming up in 2012:
- Completing the picture of addon-as-feature integration into the Moz-ecosystem: l10n, performance and unit test automation.
- Ship BrowserID as a bundled add-on.
- Ship the SDK inside Firefox (the relevant parts, anyway).
- Ship Web Apps as a bundled add-on, if it makes sense to do so.