We use Mailman for mailing list management at Mozilla. However, we haven’t been able to control spam on those lists. Not only does spam not get caught before hitting Mailman, the built-in spam rules don’t work – I configured the rules to discard anything above spam level 1, and tons of spam is still held for moderation. But identification *does* work… so we can just ignore the moderation queue forever and move on with life, right? No. Because we have another problem: The newsgroups support means that valid posts might be held for processing because Mailman doesn’t know the sender is valid because they subscribed through Google Groups. So we have to view the lists and mark those messages as valid and add the sender to the valid-senders list. To cap it all off, in the long tradition of open source user interface design, Mailman has a full featured but RSI-inducing UI for managing all of this.
So if you’re an admin for a bunch of Mailman lists, here’s a mitigation that might make your day suck just a little bit less:
I wrote a bookmarklet that will do the following for all messages held for moderation:
- Select the radio button to discard the message
- Check the box to auto-discard from that sender
- Check the box to ban the sender from ever subscribing again
To install, drag this link to your bookmarks Toolbar: Mailman – discard and ban all
Remember to make VERY sure that none of the messages are valid before executing this bookmarklet… not only because valid messages will be discarded and their senders will be banned, but also because the UI for removing addresses from the block-list in Mailman is even worse than the one for moderation.
Tweets. Likes. Check-ins. Steps. Photos. Links. We emit a constant stream of content into the sites we use everyday. Is it ours? Well, it feels like it. And according to the terms of service, it kind of belongs to us. Sometimes. Maybe. I don’t know, honestly. I don’t read those, and likely you don’t either. If you fully parse every TOS you encounter, you can stop reading right now, go straight to the nearest IHOP and have some pancakes.
Groups like IndieWeb are tackling data ownership issues – experimenting with ways of creating and managing content in a way that bakes in data ownership and removes the need for massive corporate services to achieve Web scale interactions. It’s fun and interesting and useful… we meet every two weeks, come check it out! But an IndieWeb world is a long way off. Especially for non-geeks.
All the companies providing you services today have little interest or incentive to make ownership clear or to make your data portable – the fact that you can’t easily move your data around is partially what keeps Google, Facebook, Whatsapp and Pinterest running.
Sure, lots of these services have APIs, through which you could access your data (or at least some of it). But regular users can’t really move data around between competing services through APIs. Some of these companies have data export options, where you can download your data in a zip file, but it’s not like you’re doing that every day. What if today you get banned from Gmail because some process hit a false-positive while enforcing some usage policy? And when was the last time you downloaded all your stuff? Did you ever?
I’ve always wanted to move to a model where I only participate in services through my own website first, and then send a copy of my activity to Twitter, Foursquare, Bit.ly through their APIs (including in each request an extra HTTP header containing the URL to the UELA that my content is covered by!). But that will take a while to build, and won’t work for devices like my Jawbone UP24 which syncs over a proprietary protocol to native apps only, finally letting me access a subset of my data through their API once it’s already on their servers first.
So, back to the point of this post: What can you do now?
I’ve started by attempting to own a copy of as much of my internet output as possible.
Here’s how I do it:
- Create a Google account, and install the Google Drive app. This will keep a local copy of all of your Google Docs on your computer.
- Create an IFTTT account. IFTTT is a fantastic service that allows a large number of apps, websites, services, devices and all kinds of stuff to interact with each other.
- Set up recipes for everything you do online that’s supported by IFTTT which all add to a Google Docs spreadsheet. I have recipes for Tumblr, Twitter, Bit.ly, Soundcloud and a ton more. I was very surprised by how many services have some bits of my daily personal activity.
That’s it. The IFTTT triggers will start firing, the spreadsheets will start filling, and the files will be copied to your local hard drive. You’ll now have both an online copy and a local backup of all the stuff you do online.
Here’s what some of my IFTTT recipes look like:
IFTTT will save it all in Google docs folder hierarchy for each service:
I’ll be honest – it takes a little while. It’s annoying to set up, even with IFTTT’s excellently simple user interface. But in the end I own a copy of any new content I post to the services I use. What do I do with it? Nothing at all, right now. But someday maybe those services will shut down, or I’ll get banned, or poorly implemented IT backup strategies will wipe it all out. I’ll have lost nothing except the time I spent setting this up.
Is it real data portability? Nope. Is it true ownership? No. Hell, “my” content might even be under some crazy license restrictions that I agreed to before putting it up on Pocket or Evernote! But it’s always running, and someday my personal weak-AI agent will be able to do something with it.
PS: Do you use Gmail? Do you only use the web interface or Android/iOS apps? Then you probably don’t have a backup of your email. I run an IMAP mail client (Thunderbird) periodically to get an updated offline copy of my Gmail in an open format.
Although Mozilla feels almost like a household name at this point, it is a relatively small organization – tiny, actually – compared to the companies that ship similar types of software . We must, however, have the impact of a much larger entity in order to ensure that the internet stays an open platform accessible to all.
Producing consumer software which influences the browser and smartphone OS markets in specific ways is how we make that impact. Shipping that software requires teams of people to design, build and test it, and all the countless other aspects of the release process. We can hire some of these people, but remember: we’re relatively tiny. We cannot compete with multi-billion-dollar mega-companies while operating in traditional ways. Mozilla has to be far more than the sum of its paid-employee parts in order to accomplish audaciously ambitious things.
Open source code and open communication allow participation and contribution from people that are not employees. And this is where opportunities lie for both Mozilla and universities.
For universities, undergraduates can earn credit, get real world experience, and ship software to hundreds of millions of people. Graduate researchers can break ground in areas where their findings can be applied to real world problems, and where they can see the impact of their work in the hands of people around the world. And students of any kind can participate before, during and after any involvement that is formally part of their school program.
For Mozilla, we receive contributions that help move our products and projects forward, often in areas that aren’t getting enough attention only because we don’t have the resources to do so. We get an influx of new ideas and new directions. We gain awesome contributors and can educate tomorrow’s technology workers about our mission.
I’ve been working with a few different programs recently to increase student involvement in the Firefox OS:
- Portland State University: The PSU CS Capstone program, run by Prof. Warren Harrison, has teams of students tackling projects for open source groups. The teams are responsible for all parts of the software life-cycle during the project. In the spring of 2013, a group of five students implemented an example messaging app using Persona and Firebase, documenting the challenges of Web platform development and the Firefox OS development/debugging workflow. This year’s group will implement a feature inside Firefox OS itself.
- Facebook Open Academy: This is a program coordinated by Stanford and Facebook, that puts teams from multiple universities together to build something proposed by an existing open source project. The Firefox OS team includes students from Carnegie-Mellon, Purdue, Harvard, Columbia in the US, and Tampere UT in Finland. They’re adding a new feature to Firefox OS which allows you to share apps directly between devices using NFC and Bluetooth. With 14 members across five universities, this team is collaborating via Github, Google Groups, IRC and weekly meetings for both the front-end and back-end parts, providing experience with remote working, group coordination and cross-team collaboration.
- University of Michigan: Prof. Z. Morley Mao’s mobile research group has started looking at device and network performance in Firefox OS. They’ve got a stack of phones and SIM cards, and we’re working with them to find ways to improve battery life and network efficiency on our devices. They’ve started a collection of focus areas and related research on the Mozilla wiki.
If you’re at an academic institution and would like to learn more about how to get your department or your students involved, or if you’re a Mozillian who wants to coordinate a project with your alma mater, email me!
1. Mozilla has ~1000 employees. According to Wikipedia, Google has ~50,000 employees, Apple ~80,000 and Microsoft ~100,000.
Props to Mirko85 for this post which spelled it out for Windows.
STANDARD DEVICE WARNINGS: You’ll violate your warranty probably, and you might brick your phone. You have been warned.
- Install adb and fastboot for Mac
- Download these files from Ubuntu’s server
- Plug the device in via USB
- Open Terminal and navigate to the directory you downloaded the files to
- If you haven’t unlocked the device, execute “fastboot oem unlock” and follow onscreen instructions
- Power down the device
- Enter fastboot mode by pressing volume up, volume down and power buttons at the same time until you feel it vibrate
- Once in fastboot mode, execute the following commands from Terminal
- fastboot flash recovery recovery-quantal-preinstalled-armel+maguro.img
- fastboot flash system quantal-preinstalled-system-armel+maguro.img
- fastboot flash boot quantal-preinstalled-boot-armel+maguro.img
- Using the volume keys, select recovery mode and press power
- In Terminal, execute “adb sideload quantal-preinstalled-armel+maguro.zip”
- Wait for the command to complete (if you see “fixing fs_size in crypto footer”, ignore it, it’s done)
- Use volume keys to navigate to “advanced”, press power, select “reboot recovery” and press power again
- Once back in recovery mode, execute from Terminal: “adb sideload quantal-preinstalled-phablet-armhf.zip”
- Once the command has completed, use the volume keys to select “restart device” and press power button
UPDATE: Scroll down for update on May 26, 2013.
Since beginning work on the Firefox OS project, the number one question I’m asked is “Does it run on my phone?”. Sadly, the answer for almost everyone is “no”. The question itself is interesting though, and shows how people – even geeky technical people – don’t have a good understanding of how mobile devices work, nor the whole business and technical ecosystem that brings these things into the hands of consumers (hm, maybe that’ll be my next blog post). Porting an operating system to a device is tricky work in the best of circumstances and when done without the direct assistance of the various business entities involved in the stack for any single device (OEM, chipset manufacturer, original OS vendor), involves a lot of, well, fiddling around. The kind of fiddling around that voids warranties and turns $600 hardware into a paperweight. The success and hackability of Android simplified things a lot, creating a relatively large community of people doing OS-to-device porting, and enabling a lot of what allowed Firefox OS to bootstrap so quickly. However, it’s still not easy.
I was curious about who is playing around with Firefox OS in the dark corners of the Mos Eisley of the device-porting porting world,the XDA-Developers forums. Below, I’ve listed a number of threads involving efforts to port Firefox OS to various devices. Some have builds, some are aborted attempts, but the totality shows the level of interest in putting a truly open Web operating system on low-powered commodity mobile hardware that is very exciting.
Oh, and if you’re interested in porting Firefox OS to your device, the basic instructions to get started are on the MDN B2G Porting Guide. If you scan any of the threads below or have ever tried doing this kind of work before, you already know: Thar be dragons. You could lose your device and your sanity, and will very likely void the warranty. Consider yourself warned.
- Samsung Epic 4G Touch, Samsung d710 (and some code on Github)
- HTC Wildfire S
- HTC Sensation – Some talk of debugging the porting process, and links to other ports such as Razr, Ascend g300.
- Samsung Galaxy Gio
- HTC Jewel (EVO + LTE)
- Samsung Galaxy Nexus
- LG Optimus 2X
- Samsung Nexus S & Nexus S 4G – thread 1, thread 2
- HTC Dream/G1 (old skool!)
- Samsung Galaxy S III
There are also some efforts at porting to other types of devices, such as Oleg Romashin’s experiments with Firefox on Raspberry Pi, MDN instructions for building for Pandaboard, and a bug for some core changes to Firefox OS to ease porting to basic Linux systems like Beagleboard and Chromebox.
UPDATE May 26, 2013
New devices since this was originally posted, and some fantastic updates:
- Raspberry Pi! Philipp Wagner, a student from Germany, updated Oleg Romashkin’s porting work, and wrote the Raspberry Pi for Firefox OS guide, where you can download builds and read instructions for building it yourself. I installed his build and was up and running on my Raspberry Pi in minutes. Check out his blog post, and buy him something from his Amazon wishlist :)
- As I’m sure you’ve heard by now, Geeksphone has two devices available with Firefox OS pre-installed now available for purchase. The devices keep selling out FAST, so keep a watch on their Twitter account. Also, according to their forums they’re going to make nightly updates available over-the-air soon, so you can stay on the latest versions of Firefox OS.
- At this year’s Mobile World Congress, Sony released ROMs for the Experia E.
- The XDA-Developers blog reported on a Firefox OS port for the HTC HD2.
- Also on the XDA-Developers blog was a port for the HTC Explorer (aka Pico).
- You can find build instructions for running Firefox OS on Pandaboard up on the Mozilla Developer Network.
A couple of other notes:
- XDA-Developers now has a Firefox OS forum, where there are lots of threads on the porting process, individual devices, and app development.
- All XDA-Developers blog posts tagged Firefox OS.
We quietly shipped Firefox 11 with a bunch of performance fixes that both reduce the amount of memory that Firefox uses, and improve the responsiveness of it’s user interface.
These types of changes are not easy to talk about. Often they’re very technical, and meaningless to anyone but the developers involved, which is probably why we usually don’t enumerate them in the the release notes or other public announcements. “Firefox is 74% faster when you open menu X, and twice as fast in some garbage collection scenarios!” Yeah, not an eye-popping headline. We could do a lot better in communicating these improvements in broadly meaningful ways though – nice graphs or some competitive site like arewefastyet would help a lot. But until then, here’s a short summary of the improvements in Firefox 11. And if you know of other performance fixes that don’t fall into the categories below, please add them in the comments!
Memory Use (aka “memshrink”)
The Memshrink project has been going for quite a while now, led by Nicholas Nethercote. He blogs weekly updates on the project’s activity. According to Bugzilla, there were 29 memshrink bugs marked fixed during the Firefox 11 development cycle – four of which were P1, or very high priority. Some of the fixes were related to tools and detection methods, but many are actual reductions in memory use. The changes that made it into Firefox 11 include fixes for detected leaks, removing of zombie compartments, lazy-loading data, reducing the size of some caches, reducing memory used while scrolling, and many more.
UI Responsiveness (aka “snappy”)
The Snappy project started last December, and is run by Taras Glek. Its aim is to improve the responsiveness of the Firefox UI. Taras has been posting weekly updates on Snappy activity on his blog. Bugzilla shows 15 snappy bugs marked fixed during the Firefox 11 development cycle. The project had just started, but there are still some significant wins in this release! Firefox 11 includes reductions in queries in the bookmarks system, reduced preference lookups, faster data collection for session restore, and various improvements in the DOM code.
While it’s not related to performance, I do want to call attention to something that many people don’t seem to know: In Firefox 10 (yes, the previous release) we stopped marking most add-ons incompatible when you upgrade. That means that a LOT more of your add-ons will continue to work when you upgrade Firefox from here on out. The only add-ons that still require compatibility bumps are those that have binary components, since they need to be recompiled against the current version.
My team is “a riddle, wrapped in a mystery, inside an enigma”. Ok, maybe not *that* mysterious, but we’re definitely involved in a variety of projects. Here’s a roll-up of a week in browser-land.
- Tim landed a patch that adds reporting of leaked windows and documents to our unit test runs. Dao’s work has shown that these often reflect leaks in the platform not just the test, so it’s very valuable to know about them. The next step is to track the leak numbers between runs and report regressions by making the tree color change somehow. Tim also blogged about this.
- Neil has a patch up for adding telemetry that measures how long it takes to open a browser window.
- Mano continued his work on the Safari migrator, which includes various cross-migrator improvements. This work is part of a broader rewrite of the migrators to use Places async APIs, for improved responsiveness.
- Marco landed a major rewrite of Livemarks, converting them to load content on-demand and asynchronously, reducing synchronous IO on the main thread.
- Paolo continued his work on rewriting favicon API consumers to use the new asynchronous API.
- Drew has completed the conversion of the content preferences service to use async mozStorage statements, and is in the process of updating the various JS and C++ consumers to use the new API.
- I talked a bit with Mark Cote about Peptest, a new framework for testing browser responsiveness. Peptest is currently available on the tryserver, but results are highly variable, so there’s more work to do before it can be turned on for all check-ins. Once Peptest is ready to use, we’ll evangelize the crap out of it.
- I continued working on breaking apart the session restore service into more manageable pieces (XPath generator, tab state, form data).
- I put up a WIP patch for reading the session file asynchronously during startup.
- I started working on measuring how much time is taken up by creating about:blank browsers when restoring sessions on demand.
New Tab Page
- Tim worked on Stephen Horlander’s redesign of the new-tab page, and also fixed some bugs.
New Download Manager
- Paolo consolidated existing patches into a single one for final pass and check-in. Marco has been reviewing.
- Paolo and Marco are getting together in Novara on Saturday to sprint on fixing theme bugs and other cleanup required for landing.
Add-on SDK Integration
- Met with Irakli to create plan for landing the core SDK modules in Firefox so that the whole SDK would no longer need to be bundled with every add-on.
- Created a feature page for tracking the project.
- I talked with Brian Warner about syncing Git repos with HG repos.
Web Apps Integration
- I updated the feature page with latest from UX, Apps and PM.
- Felipe Gomes is driving the Firefox side of the integration work, with Tim helping out on UI. They’re working together with Fabrice, Myk, Dan and Tim A.
- David has the window.crypto.getRandomValues patch up for final review and superreview.
- David put up an initial patch for the navigator.id API for Web content.
- David gets to play with NSS+DOM+XPCOM to get DOMCrypt’s internal API working. Lucky guy.
- David is working with Shane Caraveo to get the new Share add-on reviewed and landed.
- Tim and Marco kicked off sponsorship process for Italy’s jsDay conference. They’ll be putting up a booth there, along with Paolo Amadini, representing Mozilla. Thanks to Stormy and Shez for the support from Developer Engagement.
Of course, we all worked on various other things as well, from code review to bug triage to random maintenance fixes. Activity logs and whatnot are listed below.
- David Dahl: Bugzilla activity log, status updates, blog, Twitter
- Drew Willcoxon: Bugzilla activity log, status updates
- Felipe Gomes: Bugzilla activity log, status updates, blog, Twitter
- Asaf Romano: Bugzilla activity log
- Marco Bonardo: Bugzilla activity log, status updates, blog, Twitter
- Neil Deakin: Bugzilla activity log, status updates, blog
- Paolo Amadini: Bugzilla activity log
- Tim Taubert: Bugzilla activity log, status updates, blog, Twitter
- Dietrich Ayala: Bugzilla activity log, status updates, blog, Twitter