Unbookmarking the Future of Browsing

I am needy:

  • I want to remember URLs. Bookmarking is too manual and akin to throwing URLs in the sarlacc pit. The user-interface pieces around bookmarking have not changed in a decade. No, the awesomebar is not a good tool for this. I don’t even come close to being able to recall what I want the awesomebar to recall. I need to be ambiently prompted in a way that is visual and has context.
  • I need to be able to focus on a given task, project or idea. A single sea of tabs doesn’t help at all. I want blinders. I want an environment. Task immersion.
  • I need to be able to categorize URLs into groups, such that the whole group is easily accessible. Trees and menus can go to hell, along with the RSI, eye-strain and visual boredom they provide.
  • I need to be able to switch contexts quickly and easily. Eg: From bug triage to perf to dashboards to music, etc.
  • I don’t want to leave the browser. Windows are super heavyweight feeling and come along with all kinds of operating system baggage: visual, interaction, performance, etc.

I realized recently that a pattern had emerged in my browser usage that meets a bunch of these needs:

  • I use Firefox’s Panorama feature to manage groups of tabs. I have groups for a bunch of work areas, and for Food, Music, Design, JavaScript, Health, and many more. This provides task-specific browser contexts, as well as keyboard shortcuts for switching contexts with ease.
  • I set up Firefox to restore my session every time it starts. This way my groups persist, and all the URLs in each group are loaded with their cookies and other session data ready to go when I need them.
  • I have “Don’t load tabs until selected” checked, so that Firefox does all this with as little memory as possible – the web pages in all the tabs in all the groups aren’t loaded until I actually use them.
  • I restart the browser a couple of times per day to keep memory use slim, which in turn keeps the browser responsive. Restarting is super fast and responsive because I have “Don’t load tabs until selected” (see previous point).

This is the happiest I’ve been with any browser in years. However, there are still a bunch of pain points. I want SO much more.

  • I want to tag URLs without bookmarking them. The bookmark concept just gets in the way. It’s an unnecessary unit of psychological weight. It’s a vestigial metaphor of days gone by.
  • I want to open a tab group by typing the name of the group in the URL bar.
  • I want to add URLs to multiple groups easily, similar to tagging. I’d like to do it via the keyboard.
  • I want to send the current tab to a different group (or groups) using only the keyboard.
  • I want app tabs that are specific to a given group, and some that are global.
  • I want to switch quickly from an app tab back to the last non-app tab I was at. Or be able to peek quickly at an app tab without losing my context in the current set of tabs.
  • I want to switch quickly back to the last tab I was at. (Eg: When I open a new tab, and get sent to the end of the current set of tabs). OR be able to have new tabs open immediately to the right of the current tab, with linked parentage.
  • I’m tired of sites being browsers inside a browser. And I don’t want “site-specific” browsers – I want a “me-specific” browser, for local or dynamic content.
  • Firefox creates the <tab> elements for hidden tabs when restoring the session. It would re/start even faster and use even less memory if the XUL elements for hidden tabs were not created until the group itself was opened.
  • As I work, memory use increases and responsiveness decreases, since I keep visiting more and more tabs. If I haven’t visited a tab in a while, Firefox should unload it. If I haven’t visited a group in a while, Firefox should completely unload the whole group, session content *and* XUL elements.
  • A downside of the “Don’t load tabs until selected” option is that tab content is not ready and waiting when you select the tab. The web content has to load and the session data for the tab must be restored. Firefox should pre-load tabs that are adjacent to the active tab. This feature, combined with the dormant-izing of tabs described above would result in a decent balance of instant-gratification availability and responsiveness and resource frugality.

One idea I had was a merging of tagging and groups: The groups in Panorama would be comprised of the set of tags that exist. This would result in nice integration bits like search-by-tag in the awesomebar being equivalent to search-in-group. It also might mean that we’ll need to make Panorama “bigger” – maybe allow it to be zoomed, or make it an infinite canvas.

An idea for navigating dynamic content is to merge feeds and groups. Imagine you have a BBC group, which has the BBC feed as it’s source. The set of “tabs” in that group are the items in the feed. If you open the group, all the URLs in the feed are loaded into tabs (but not *really* loaded if you restore-on-demand).

Anyways, it’s interesting to think about how to prototype some of these ideas in an add-on or a collection of them. I’m sure some of the items above already exist as add-ons.

I realize that I’m not a “typical user”. However, after almost 6 years of browser-making, I’m pretty damn sure that there is no such person. I do not believe that the one-size-fits-all browser is the future. When adding a feature or fixing a bug, we shouldn’t have to choose between grandma and the geeks. In order to stay relevant in a highly-personalized future, we should strive to ensure that Firefox is pliable enough that we who make it are not restricted by it, and more importantly we must ensure that add-on developers are free to mash-up and remix and experiment the f*ck out of it.


Building Firefox in the Post-Browser Age

Ok, so maybe we aren’t in the post-browser age *yet*. But we’re getting there, and quickly. Most of the “apps” I use on my phone are useless without an always-on data connection, and they communicate with their respective motherships via HTTP. We’re staring down a near-future with multiple Web-stack operating systems for both desktop and portable devices. We have server-side application platforms that look startlingly like pieces of a traditional Web client.

All of those places are the Web, so that’s where Mozilla has got to be, when and if it’s possible to do so. And between desktop Firefox, mobile Firefox, Chromeless, B2G, Pancake, Open Web Apps and the various Firefox features developed by the Labs and the Services groups, we’ve got a lot of application logic that needs to exist in various forms across those disparate environments.

Up until recently, even including most of the mobile efforts to date, we’ve had a pretty narrow idea of what constitutes Firefox: Mozilla’s browser application with a front-end built in XUL, and rendering content using Gecko, stored entirely in a single source repository.

This narrow view is insufficient given the needs of internet users and the plans we have to serve those needs in the immediate future. This has been starkly illustrated by the recent move to a native UI for mobile Firefox, projects like Pancake, and the expansion of the development of Firefox features by groups outside of the core team.

A few months ago I dumped a couple of thoughts into a thread on the mozilla.dev.planning newsgroup about these things. More than anything, that thread showed me that the broad spectrum of activity in Mozilla today makes our narrow view of Firefox a huge barrier to future success. Some people didn’t agree that there was a problem at all. Some people railed against Jetpack or Github, while admitting they’d never used either. Some people agreed that developing Firefox is slow and fragile, and pointed at the relative historical success of that approach. Disturbingly, I got a bunch of private emails thanking me for starting the conversation… what does *that* mean?! Overall though, there was a lot of agreement on this point: We need more people to be able to work on Firefox faster, and in a more heterogeneous environment.

There’s a bunch of work towards that end going on right now, both in Firefox team itself and in Mozilla generally, around lowering the barriers to contribution. Specific to Firefox core development though, one experiment in alternate approaches is the attempt to ship the BrowserID feature as a Jetpack-based add-on that is developed on Github and bundled with Firefox. There are a lot of moving parts, but the exercise is helping us figure out the up- and downsides to building features as add-ons, as well as providing performance data about the Add-on SDK. Maybe it’ll work, maybe we’ll have to re-route and patch it against the core. Maybe we’ll land somewhere in-between.

Regardless of that experiment’s outcome, I think we need to be experimenting hard with how we develop Firefox, and asking questions about the longer-term development landscape:

  • Code changes currently have non-deterministic effects in the Firefox ecosystem. We have a jumble of services that stagger into existence at startup, and then race for the exit at shutdown, beating up the file-system at both ends of the application lifecycle. “Async” is a pattern, not a system – without a system, making a bunch of things asynchronous means that the application’s behavior as a whole is generally less predictable. Is there a more systematic way that we can manage the loading and unloading of core browser services?
  • Calcification: Check out the “cleanup page”. There are long-despised-and-untouched pieces of our core infrastructure, such as URL classifier, importers, autocomplete, and parts of Places. Why is it so hard to change these? What are the barriers to making them better?
  • Modularity: Cu.import is great in that it provides some of the benefits that we used XPCOM JS services for, but without the XPCOM. But are we using it enough? Jetpack development puts much more emphasis on modularity via a core built on CommonJS, and I’ve found it to make browser features written in Jetpack far easier to follow, debug, and contribute to. Maybe we should be putting code into modules where we’d normally add it to browser.js, or XBL widgets moreso than we are now? This could reduce our dependence on the XUL window mega-scope that we get in browser.js, which I’d argue leads to code that is easier to developer, debug, test and maintain.
  • Abstracting the application logic away from XUL/XPCOM where possible, allowing for more portable code. This doesn’t make sense in a lot of places in the front-end, but in others such as sync or expiration policies or tab grouping algorithms or frecency generation, it might. These are things which could be useful across a number of different application contexts.

So where from here? There’s general agreement that the Add-on SDK needs to ship in the browser. This might help address some of the questions above. However, it won’t immediately help us share code with other Firefoxes or Mozilla projects, or make core development inherently less-fragile or our application behavior any more deterministic. And there are tools like Cu.import, which we have now, and Harmony modules, which we might have soon (can we use those in chrome?!) that could help with the modularity part.

But only some of this is about the technology – other parts are social. As I said above, some people do not agree that developing Firefox is slow and fraught with peril. Is that plain ol’ resistance to change, or just the lack of a clear alternative? And maybe we code reviewers should be more forward-looking, demanding larger refactorings instead of non-invasive surgeries. But that’s challenging when you’re constrained for time, or the regression cost of refactoring is so high that you become risk averse.

I’d love to hear your thoughts on the future of Firefox application development – especially the core Firefox team, and the people working on Firefox features in other groups or via add-ons. Myk Melez has been corralling a group to talk about feature development with the Add-on SDK specifically, but it quickly spreads into these broader issues. He’s starting a list for it, but until then there are regular meetings, details available in his dev.planning post.


Visualizing web page memory use in Firefox

I remixed about:memory using D3 to make a treemap visualization of how loaded URLs are using memory in Firefox.

Load it by clicking the widget on the add-on bar, or Cmd/Ctrl+Shift+Y

This visualization focuses on which web pages are using large amounts of memory – this is not a complete accounting of all memory being used. This doesn’t yet show how much is being used by some of the Firefox internals.

Add-on installation does not require a restart. Only tested on Nightly builds.

Install

Source code


Dormancy: Freeing up memory from unused tabs

Dormancy ‘retires’ tabs that have gone unused for a while, freeing up that memory. It then restores the tabs to life when accessed.

While Firefox 9 adds restore-on-demand for users that restore their session by default, many users will never benefit from it. This add-on targets users who don’t restore session, but do have long-running instances of Firefox and many tabs. This might land as a core feature in Firefox 9 (https://bugzilla.mozilla.org/show_bug.cgi?id=675539).

NOTE: This is highly experimental, has only been tested on the Nightly builds, and probably will destroy your session. You’ve been warned.

Tabs are considered inactive when they haven’t been selected in longer than 5 minutes. To change that, set this pref to a value in milliseconds:

* extensions.dormancy.TabDormancyAgeMs

Tabs are checked for inactivity every 5 minutes. To change this, set this pref to a value in milliseconds:

* extensions.dormancy.TabCheckIntervalMs

Known bugs:

  • Awesomebar entry for dormant tabs shows data URI
  • Sometimes dormant tabs have no title and no favicon

EXPERIMENTAL. MAY EAT YOUR SESSION OR DO OTHER BAD THINGS.

INSTALL

Source code


Wallflower: Un-Socializing Your Web

I was looking at about:memory and noticed entries for Facebook and Google+ URLs, even though I didn’t have either open. I figured they were probably from the social buttonry that decorates the web these days. No big deal… except they were taking up a bunch of memory! The Facebook button was using over 20mb and the Google+ button was taking over 40mb!

I have never clicked either of these buttons.

So I wrote Wallflower, a simple Firefox add-on (restartless of course) that removes these buttons from any page your browse to, saving your precious memory, CPU and battery life for the content you actually want.

Install Wallflower.

View the source code on Github.


Peek: Apps, App Tabs, and Context.

The new app tab feature in Firefox is great. I use it a lot… which has starkly illustrated how apps and tabs have very different use-cases and usage patterns. Often I will check my Gmail app tab because I see the glowing notification that a new email has arrived, do something (or nothing), and then pop back to where I was browsing… in one of those 78 tabs I have open?!
The windowing model in operating systems allows me switch back and forth between contexts with ease. But app tabs do not:

  • If I’m using my Gmail app tab and I didn’t open any links while there, I can still see the last tab I was at, and click on it. But I have to use the mouse if I want to get directly back there.
  • Out of sheer muscle memory and mouse-averseness, sometimes I can traverse tabs via the next/previous-tab keyboard shortcuts to get back to where I was. Sometimes it’s a *lot* of tabs, so either I’ll hold the arrow key down, speeding past the tab I wanted (and back and forth a few times), or I’ll just hit the arrow key a bunch of times in quick succession. Both options are frustrating, slow and RSI-inducing.
  • Or I could expend mental energy to search in the awesomebar and switch to that tab, which often looks like this: “hm, type ‘bug’ and then try to remember some words in the bug summary, but those words match a bunch of other bugs, and i don’t know the bug number, and also I’m on an attachment page because I’m reviewing a patch on the bug, so the summary won’t be in the page title…” and on and on. Now add the fact that switch-to-tab rarely even shows up in the awesomebar for me, and well, a generally high level of fail with this option.

Then there’s link opening:

  • Links opened in app tabs are put at the beginning of your tabs, and the tab strip is animatedly scrolled there. Boom, instantly lost where I was before checking my email.
  • We tried an experiment where they open at the end of the set of open tabs. I found that to have serious “out of sight, out of mind” problems. That experiment was rolled back. And it doesn’t necessarily solve the context problem anyway.
  • Both approaches cause excess amounts of whizzing animations, either when you want to “go around the horn” to get to the tabs you just opened from app tabs, or when you want to go to them and then get back to where you were.
  • There’s no right answer! Sometimes I see a link to a recipe in my Seesmic app tab that I’d like to open in that series of tabs related to food that I have open somewhere in the middle of my open tabs. The user is not in control of *where* these links are opened. I can’t choose whether to open them at the beginning or end of the tab strip, or in a new tab group, or new window, etc. Part of me thinks that I actually might work best in a world where each app tab is bound to a single tab group, so that tab growth is bound to the source… but that’s a vision for another day (and blog post and add-on).

So I’ve made Peek, an add-on that’s a hybrid solution: Instead of making you go to your app tabs, your app tabs will come to you. Peek allows you to open your app tabs in a floating panel that opens on top of wherever you are in your tabs. Links open to the right of whatever your current active tab is, and in the background, so that when you’re finished peeking, you are exactly where you left off.

To use Peek, first create some app tabs. Then you can peek at them using the keyboard shortcut ALT+SHIFT+1-9 where the number corresponds with the order your app tabs are in. To stop peeking, hit escape (or switch apps or anything else that takes focus away from the panel).

INSTALL. Peek is a Jetpack add-on that does not require a restart of the browser. User beware: This is an experimental add-on – I’ve been using it on Nightly builds, but haven’t done much testing elsewhere.

Benefits:

  • Interact with your apps, and when you’re done, be exactly where you left off browsing.
  • Links are opened in the context of wherever you’re peeking.


Extensibly Awesome: A Jetpack API for the Firefox Location Bar

I tweaked some of Mardak’s code for the Twitter Add-on, and created a Jetpack module that makes it terribly simple to write add-ons that extend the awesomebar with your own suggestions. As an example, here’s an add-on that uses the Google Translate API to translate text into a specified language directly in the location bar. If you type in the keyword ‘translate’, followed by a language code and some text, it will show the translation in the awesomebar results:

Screenshot

The code:

require('awesomebar').add({
  keyword: 'translate',
  onSearch: function(query, suggest) {
    let lang = query.substring(0, query.indexOf(' '));
    let text = query.substring(query.indexOf(' '));
    if (lang.length == 2 && text.length > 2) {
      translate(lang, text, function(translatedText) {
        suggest({
          title: 'Translated text in ' + lang + ': ' + translatedText,
          url: 'http://translate.google.com/?tl=' + lang +
               '&q=' + encodeURIComponent(text),
        }, true);
      });
    }
  }
});

function translate(lang, text, callback) {
  require('request').Request({
    url: 'http://ajax.googleapis.com/ajax/services/language/translate',
    content: {
      v: '1.0',
      q: text,
      langpair: '|' + lang
    },
    headers: {
      Referer: require('tabs').activeTab.location
    },
    onComplete: function() {
      callback(this.response.json.responseData.translatedText);
    }
  }).get();
}

The example needs niceties such as being able to write full language names, but you get the gist.

Here’s the awesomebar.js module.

There’s a bit of documentation in there. The code could use some cleanup, and could probably be much smaller if converted to use the internal Jetpack APIs for things like window-watching, etc.


Follow

Get every new post delivered to your Inbox.