Harvesting Air Quality Data with a NodeMCU, SensorWeb and IFTTT

Project SensorWeb is an experiment from the Connected Devices group at Mozilla in open publishing of environmental data. I am excited about this experiment because we’ve had some serious air quality discoveries in Portland recently – our air is possibly the worst in the USA, and bad enough that mega-activists like Erin Brockovich are getting involved.

Screen Shot 2016-07-13 at 5.21.01 PM

A couple of weeks ago, Eddie and Evan from Project SensorWeb helped me put together a NodeMCU board and a PM2.5 sensor so I could set up an air quality sensor in Portland to report to their network. They’re still setting up the project so I haven’t gotten the configuration info from them yet…

But you don’t need the SensorWeb server to get your sensor up and running and pushing data to your own server! I want a copy of the data for myself anyway, to be able to do my own visualizations and notifications. I can then forward the data on to SensorWeb.

So I started by flashing the current version of the SensorWeb code to the device, which is a Nodemcu 0.9 board with an ESP8266 wifi chip on board, and a PM2.5 sensor attached to it.

2016-07-13 17.05.59

I used Kumar Rishav’s excellent step-by-step post to get through the process.

Some things I learned along the way:

  • On Mac OS X you need a serial port driver in order for the Arduino IDE to detect the board.
  • After much gnashing of teeth, I discovered that you can’t have the PM2.5 sensor plugged into the board when you flash it.

After getting the regular version flashed correctly, I tested with Kumar’s API key and device id, and confirmed it was reporting the data correctly to the SensorWeb server.

Now for the changes.

I flashed the device and viola, it is publishing data to my spreadsheet.

 

Screen Shot 2016-07-13 at 4.48.28 PM

And now once SensorWeb is ready to take new devices, I can set up a new IFTTT recipe to forward the posts to them, allowing me to own my own data and also publish to the project!


Bubble and Tweak – IoT at the Ends of the Earth

Screen Shot 2016-07-12 at 3.19.07 PM

Last month I spent a week working on IoT projects with a group of 40 researchers, designers and coders… in Anstruther, a small fishing village in Scotland. Not a high-tech hub, but that was the point. We immersed ourselves in a small community with limited connectivity and interesting weather (and fantastic F&C) in order to explore how they use technology and how ubiquitous physical computing might be woven into their lives.

The ideamonsters behind this event were Michelle Thorne and Jon Rogers, who are putting on a series of these exploratory events around the world this year as part of the Mozilla Foundation’s Open IoT Studio. The two previous editions of this event were a train caravan in India and a fablab sprint in Berlin (which I also attended, and will write up as well. I SWEAR.).

Michelle and Jon will be writing a proper summary of the week as a whole, so I’m going to focus on the project my group built: Bubble.

Screen Shot 2016-07-12 at 3.18.39 PM

Context

From research conducted with local fishing folk, farmers from a bit inland, and a group of teenagers from the local school, we figured out a few things:

  • Mobile connectivity is sparse and unreliable throughout the whole region. In this particular town, only about half the town had any service.
  • The important information, places and things aren’t immediately obvious unless you know a local.
  • Just like I was, growing up in a small town: The kids are just looking for something to do.

Initially we focused on the teens… fun things like virtual secret messaging at the red telephone boxes. Imagine you connect to the wifi at the phone box, and the captive portal is a web UI for leaving and receiving secret messages. Perhaps they’re only read once before dying, like a hyperlocal Snapchat. Perhaps the system is user-less, mediated only by secret combinations of emojis as keys. The street corner becomes the hangout, physically and digitally.

We meandered to public messaging from there, thinking about how there’s so much to learn and share about the physical space. What’s the story behind the messages to fairies that are being left in that phone box? I can see the island off the coast from here – what’s it called and what’s out there? Who the hell is Urquardt and what’s a “wynd”? Maybe we make a public message board as well – disconnected from the internet but connected to anything within view.

We kept going back to the physical space. We talked about a virtual graffiti wall, and then started exploring AR and ways of marking up the surroundings – the people, the history, the local pro-tip on which fish and chips shop is the best. But all of this available only to people in close physical proximity.

Screen Shot 2016-07-12 at 3.40.49 PM

Implementation

Given the context and the constraints, as well as watching direction some of the other groups were going in, we started designing a general approach to bringing digital interactivity to disconnected spaces.

The first cut is Bubble: A wi-fi access point with a captive portal that opens a web page that displays an augmented reality view of your immediate surroundings, with messages overlaid on what you’re seeing:

Screen Shot 2016-07-12 at 2.36.26 PM

A few implementation notes:

  • We used a Raspberry Pi 3, running as a wi-fi access point.
  • It ran a node.js script that served up the captive portal web UI.
  • The web UI used getUserMedia to access the device camera awe.js for the AR bits and A-Frame for a VR backup view on iOS.
  • We designed a logo and descriptive text and then lasercut some plaques to put up where hotspots are.

Designs, board, battery and boxes:

Screen Shot 2016-07-12 at 3.58.10 PM

Connected to the front-end:
Screen Shot 2016-07-12 at 3.57.57 PM
In the final box:
Screen Shot 2016-07-12 at 3.58.33 PM

Challenges:

  • Captive portals are hobbled web pages. You can’t do things like use getUserMedia to get access to the camera.
  • iOS doesn’t have *any* way to let web pages access the camera.
  • Power can be hard. We talked about solar and other ways of powering these.
  • Gotta hope they don’t get nicked.

Bubble was an experimental prototype. There are no plans to work further on it at this time. If you’re interested, everything is on Github here. You can read more about the design here (PDF).

Thanks to fellow team members Julia Gratsova, Katie Caldwell, Vladan Joler. (Sadly, no Julia in the phonebox!)

Screen Shot 2016-07-12 at 3.19.51 PM

Remixable Quilts for the Web

Atul Varma set up a quilt for MozCamp Asia 2012 which I thought was a fantastic tool for that type of event. It provided an engaging visualization, was collaboratively created, and allowed a quick and easy way to dive further into the details about the participating groups.

Screenshot 2014-12-18 15.23.19I wanted to use it for a couple of projects, but the code was tied pretty closely to that specific content and layout.

This week I finally got around to moving the code over to Mozilla Webmaker, so it could be easily copied and remixed. I made a couple of changes:

  • Update font to Open Sans
  • Make it easy and clear how to re-theme the colors
  • Allow arbitrary content in squares

The JS code is still a bit too complex for what’s needed, but it works on Webmaker now!

Screenshot 2014-12-18 15.09.42

View my demo quilt. Hit the “remix” button to clone it and make your own.

The source for the core JS and CSS is at https://github.com/autonome/quilt.


Firefox OS and Academic Programs

Although Mozilla feels almost like a household name at this point, it is a relatively small organization – tiny, actually – compared to the companies that ship similar types of software [1]. We must, however, have the impact of a much larger entity in order to ensure that the internet stays an open platform accessible to all.

Producing consumer software which influences the browser and smartphone OS markets in specific ways is how we make that impact. Shipping that software requires teams of people to design, build and test it, and all the countless other aspects of the release process. We can hire some of these people, but remember: we’re relatively tiny. We cannot compete with multi-billion-dollar mega-companies while operating in traditional ways. Mozilla has to be far more than the sum of its paid-employee parts in order to accomplish audaciously ambitious things.

Open source code and open communication allow participation and contribution from people that are not employees. And this is where opportunities lie for both Mozilla and universities.

For universities, undergraduates can earn credit, get real world experience, and ship software to hundreds of millions of people. Graduate researchers can break ground in areas where their findings can be applied to real world problems, and where they can see the impact of their work in the hands of people around the world. And students of any kind can participate before, during and after any involvement that is formally part of their school program.

For Mozilla, we receive contributions that help move our products and projects forward, often in areas that aren’t getting enough attention only because we don’t have the resources to do so. We get an influx of new ideas and new directions. We gain awesome contributors and can educate tomorrow’s technology workers about our mission.

I’ve been working with a few different programs recently to increase student involvement in the Firefox OS:

  • Portland State University:  The PSU CS Capstone program, run by Prof. Warren Harrison, has teams of students tackling projects for open source groups. The teams are responsible for all parts of the software life-cycle during the project. In the spring of 2013, a group of five students implemented an example messaging app using Persona and Firebase, documenting the challenges of Web platform development and the Firefox OS  development/debugging workflow. This year’s group will implement a feature inside Firefox OS itself.
  • Facebook Open Academy: This is a program coordinated by Stanford and Facebook, that puts teams from multiple universities together to build something proposed by an existing open source project. The Firefox OS team includes students from Carnegie-Mellon, Purdue, Harvard, Columbia in the US, and Tampere UT in Finland. They’re adding a new feature to Firefox OS which allows you to share apps directly between devices using NFC and Bluetooth. With 14 members across five universities, this team is collaborating via Github, Google Groups, IRC and weekly meetings for both the front-end and back-end parts, providing experience with remote working, group coordination and cross-team collaboration.
  • University of MichiganProf. Z. Morley Mao’s mobile research group has started looking at device and network performance in Firefox OS. They’ve got a stack of phones and SIM cards, and we’re working with them to find ways to improve battery life and network efficiency on our devices. They’ve started a collection of focus areas and related research on the Mozilla wiki.

If you’re at an academic institution and would like to learn more about how to get your department or your students involved, or if you’re a Mozillian who wants to coordinate a project with your alma mater, email me!

 

1. Mozilla has ~1000 employees. According to Wikipedia, Google has ~50,000 employees, Apple ~80,000 and Microsoft ~100,000.


Firefox OS: Devices and Dark Matter

UPDATE: Scroll down for update on May 26, 2013.

Since beginning work on the Firefox OS project, the number one question I’m asked is “Does it run on my phone?”. Sadly, the answer for almost everyone is “no”. The question itself is interesting though, and shows how people – even geeky technical people – don’t have a good understanding of how mobile devices work, nor the whole business and technical ecosystem that brings these things into the hands of consumers (hm, maybe that’ll be my next blog post). Porting an operating system to a device is tricky work in the best of circumstances and when done without the direct assistance of the various business entities involved in the stack for any single device (OEM, chipset manufacturer, original OS vendor), involves a lot of, well, fiddling around. The kind of fiddling around that voids warranties and turns $600 hardware into a paperweight. The success and hackability of Android simplified things a lot, creating a relatively large community of people doing OS-to-device porting, and enabling a lot of what allowed Firefox OS to bootstrap so quickly. However, it’s still not easy.

I was curious about who is playing around with Firefox OS in the dark corners of the Mos Eisley of the device-porting porting world,the XDA-Developers forums. Below, I’ve listed a number of threads involving efforts to port Firefox OS to various devices. Some have builds, some are aborted attempts, but the totality shows the level of interest in putting a truly open Web operating system on low-powered commodity mobile hardware that is very exciting.

Oh, and if you’re interested in porting Firefox OS to your device, the basic instructions to get started are on the MDN B2G Porting Guide. If you scan any of the threads below or have ever tried doing this kind of work before, you already know: Thar be dragons. You could lose your device and your sanity, and will very likely void the warranty. Consider yourself warned.

There are also some efforts at porting to other types of devices, such as Oleg Romashin’s experiments with Firefox on Raspberry Pi, MDN instructions for building for Pandaboard, and a bug for some core changes to Firefox OS to ease porting to basic Linux systems like Beagleboard and Chromebox.

 

UPDATE May 26, 2013

New devices since this was originally posted, and some fantastic updates:

A couple of other notes:


Firefox 11 is Smaller and Faster

We quietly shipped Firefox 11 with a bunch of performance fixes that both reduce the amount of memory that Firefox uses, and improve the responsiveness of it’s user interface.

These types of changes are not easy to talk about. Often they’re very technical, and meaningless to anyone but the developers involved, which is probably why we usually don’t enumerate them in the the release notes or other public announcements. “Firefox is 74% faster when you open menu X, and twice as fast in some garbage collection scenarios!” Yeah, not an eye-popping headline. We could do a lot better in communicating these improvements in broadly meaningful ways though – nice graphs or some competitive site like arewefastyet would help a lot. But until then, here’s a short summary of the improvements in Firefox 11. And if you know of other performance fixes that don’t fall into the categories below, please add them in the comments!

Memory Use (aka “memshrink”)

The Memshrink project has been going for quite a while now, led by Nicholas Nethercote. He blogs weekly updates on the project’s activity. According to Bugzilla, there were 29 memshrink bugs marked fixed during the Firefox 11 development cycle – four of which were P1, or very high priority. Some of the fixes were related to tools and detection methods, but many are actual reductions in memory use. The changes that made it into Firefox 11 include fixes for detected leaks, removing of zombie compartments, lazy-loading data, reducing the size of some caches, reducing memory used while scrolling, and many more.

UI Responsiveness (aka “snappy”)

The Snappy project started last December, and is run by Taras Glek. Its aim is to improve the responsiveness of the Firefox UI. Taras has been posting weekly updates on Snappy activity on his blog. Bugzilla shows 15 snappy bugs marked fixed during the Firefox 11 development cycle. The project had just started, but there are still some significant wins in this release! Firefox 11 includes reductions in queries in the bookmarks system, reduced preference lookups, faster data collection for session restore, and various improvements in the DOM code.

Add-on Compatibility

While it’s not related to performance, I do want to call attention to something that many people don’t seem to know: In Firefox 10 (yes, the previous release) we stopped marking most add-ons incompatible when you upgrade. That means that a LOT more of your add-ons will continue to work when you upgrade Firefox from here on out. The only add-ons that still require compatibility bumps are those that have binary components, since they need to be recompiled against the current version.

Download Firefox 11.


Browser Services Update (TGIF)

My team is “a riddle, wrapped in a mystery, inside an enigma”. Ok, maybe not *that* mysterious, but we’re definitely involved in a variety of projects. Here’s a roll-up of a week in browser-land.

Performance

New Tab Page

New Download Manager

  • Paolo consolidated existing patches into a single one for final pass and check-in. Marco has been reviewing.
  • Paolo and Marco are getting together in Novara on Saturday to sprint on fixing theme bugs and other cleanup required for landing.

Add-on SDK Integration

  • Met with Irakli to create plan for landing the core SDK modules in Firefox so that the whole SDK would no longer need to be bundled with every add-on.
  • Created a feature page for tracking the project.
  • I talked with Brian Warner about syncing Git repos with HG repos.

Web Apps Integration

  • I updated the feature page with latest from UX, Apps and PM.
  • Felipe Gomes is driving the Firefox side of the integration work, with Tim helping out on UI. They’re working together with Fabrice, Myk, Dan and Tim A.

Identity Integration

Share Integration

Evangelism/Engagement

  • Tim and Marco kicked off sponsorship process for Italy’s jsDay conference. They’ll be putting up a booth there, along with Paolo Amadini, representing Mozilla. Thanks to Stormy and Shez for the support from Developer Engagement.

Of course, we all worked on various other things as well, from code review to bug triage to random maintenance fixes. Activity logs and whatnot are listed below.


Follow

Get every new post delivered to your Inbox.

Join 26 other followers