Harvesting Air Quality Data with a NodeMCU, SensorWeb and IFTTT

Project SensorWeb is an experiment from the Connected Devices group at Mozilla in open publishing of environmental data. I am excited about this experiment because we’ve had some serious air quality discoveries in Portland recently – our air is possibly the worst in the USA, and bad enough that mega-activists like Erin Brockovich are getting involved.

Screen Shot 2016-07-13 at 5.21.01 PM

A couple of weeks ago, Eddie and Evan from Project SensorWeb helped me put together a NodeMCU board and a PM2.5 sensor so I could set up an air quality sensor in Portland to report to their network. They’re still setting up the project so I haven’t gotten the configuration info from them yet…

But you don’t need the SensorWeb server to get your sensor up and running and pushing data to your own server! I want a copy of the data for myself anyway, to be able to do my own visualizations and notifications. I can then forward the data on to SensorWeb.

So I started by flashing the current version of the SensorWeb code to the device, which is a Nodemcu 0.9 board with an ESP8266 wifi chip on board, and a PM2.5 sensor attached to it.

2016-07-13 17.05.59

I used Kumar Rishav’s excellent step-by-step post to get through the process.

Some things I learned along the way:

  • On Mac OS X you need a serial port driver in order for the Arduino IDE to detect the board.
  • After much gnashing of teeth, I discovered that you can’t have the PM2.5 sensor plugged into the board when you flash it.

After getting the regular version flashed correctly, I tested with Kumar’s API key and device id, and confirmed it was reporting the data correctly to the SensorWeb server.

Now for the changes.

I flashed the device and viola, it is publishing data to my spreadsheet.

 

Screen Shot 2016-07-13 at 4.48.28 PM

And now once SensorWeb is ready to take new devices, I can set up a new IFTTT recipe to forward the posts to them, allowing me to own my own data and also publish to the project!


Bubble and Tweak – IoT at the Ends of the Earth

Screen Shot 2016-07-12 at 3.19.07 PM

Last month I spent a week working on IoT projects with a group of 40 researchers, designers and coders… in Anstruther, a small fishing village in Scotland. Not a high-tech hub, but that was the point. We immersed ourselves in a small community with limited connectivity and interesting weather (and fantastic F&C) in order to explore how they use technology and how ubiquitous physical computing might be woven into their lives.

The ideamonsters behind this event were Michelle Thorne and Jon Rogers, who are putting on a series of these exploratory events around the world this year as part of the Mozilla Foundation’s Open IoT Studio. The two previous editions of this event were a train caravan in India and a fablab sprint in Berlin (which I also attended, and will write up as well. I SWEAR.).

Michelle and Jon will be writing a proper summary of the week as a whole, so I’m going to focus on the project my group built: Bubble.

Screen Shot 2016-07-12 at 3.18.39 PM

Context

From research conducted with local fishing folk, farmers from a bit inland, and a group of teenagers from the local school, we figured out a few things:

  • Mobile connectivity is sparse and unreliable throughout the whole region. In this particular town, only about half the town had any service.
  • The important information, places and things aren’t immediately obvious unless you know a local.
  • Just like I was, growing up in a small town: The kids are just looking for something to do.

Initially we focused on the teens… fun things like virtual secret messaging at the red telephone boxes. Imagine you connect to the wifi at the phone box, and the captive portal is a web UI for leaving and receiving secret messages. Perhaps they’re only read once before dying, like a hyperlocal Snapchat. Perhaps the system is user-less, mediated only by secret combinations of emojis as keys. The street corner becomes the hangout, physically and digitally.

We meandered to public messaging from there, thinking about how there’s so much to learn and share about the physical space. What’s the story behind the messages to fairies that are being left in that phone box? I can see the island off the coast from here – what’s it called and what’s out there? Who the hell is Urquardt and what’s a “wynd”? Maybe we make a public message board as well – disconnected from the internet but connected to anything within view.

We kept going back to the physical space. We talked about a virtual graffiti wall, and then started exploring AR and ways of marking up the surroundings – the people, the history, the local pro-tip on which fish and chips shop is the best. But all of this available only to people in close physical proximity.

Screen Shot 2016-07-12 at 3.40.49 PM

Implementation

Given the context and the constraints, as well as watching direction some of the other groups were going in, we started designing a general approach to bringing digital interactivity to disconnected spaces.

The first cut is Bubble: A wi-fi access point with a captive portal that opens a web page that displays an augmented reality view of your immediate surroundings, with messages overlaid on what you’re seeing:

Screen Shot 2016-07-12 at 2.36.26 PM

A few implementation notes:

  • We used a Raspberry Pi 3, running as a wi-fi access point.
  • It ran a node.js script that served up the captive portal web UI.
  • The web UI used getUserMedia to access the device camera awe.js for the AR bits and A-Frame for a VR backup view on iOS.
  • We designed a logo and descriptive text and then lasercut some plaques to put up where hotspots are.

Designs, board, battery and boxes:

Screen Shot 2016-07-12 at 3.58.10 PM

Connected to the front-end:
Screen Shot 2016-07-12 at 3.57.57 PM
In the final box:
Screen Shot 2016-07-12 at 3.58.33 PM

Challenges:

  • Captive portals are hobbled web pages. You can’t do things like use getUserMedia to get access to the camera.
  • iOS doesn’t have *any* way to let web pages access the camera.
  • Power can be hard. We talked about solar and other ways of powering these.
  • Gotta hope they don’t get nicked.

Bubble was an experimental prototype. There are no plans to work further on it at this time. If you’re interested, everything is on Github here. You can read more about the design here (PDF).

Thanks to fellow team members Julia Gratsova, Katie Caldwell, Vladan Joler. (Sadly, no Julia in the phonebox!)

Screen Shot 2016-07-12 at 3.19.51 PM

Remixable Quilts for the Web

Atul Varma set up a quilt for MozCamp Asia 2012 which I thought was a fantastic tool for that type of event. It provided an engaging visualization, was collaboratively created, and allowed a quick and easy way to dive further into the details about the participating groups.

Screenshot 2014-12-18 15.23.19I wanted to use it for a couple of projects, but the code was tied pretty closely to that specific content and layout.

This week I finally got around to moving the code over to Mozilla Webmaker, so it could be easily copied and remixed. I made a couple of changes:

  • Update font to Open Sans
  • Make it easy and clear how to re-theme the colors
  • Allow arbitrary content in squares

The JS code is still a bit too complex for what’s needed, but it works on Webmaker now!

Screenshot 2014-12-18 15.09.42

View my demo quilt. Hit the “remix” button to clone it and make your own.

The source for the core JS and CSS is at https://github.com/autonome/quilt.


Mozlandia – A Portland mini-guide for Mozillians

Around 1,100 Mozillians will be in Portland next week. WELCOME!

Here is a list of things I recommend that you see, do, eat and drink and some handy tips. I’ve focused on places that are in or near downtown, though included some spots worth going out of your way for.

That said, downtown is not Portland. Well, obviously it’s part of it, duh. But it’s not overly representative of the Portland you’ve heard so much about. Get out of downtown for an evening or afternoon – and check out one of the neighborhoods. Nothing listed here is very far from downtown – Portland is pretty small.

I’ve left a LOT out. And forgot the rest. If you have recommendations, leave them in the comments.

UPDATE: Blue Star donuts, which are far more delicious but far less penis-shaped than Voodoo donuts, will be giving away free beignets on Friday Dec 5.

GENERAL TIPS

  • Portland has no Uber or Uber-like services. If you have Car2Go in your city you can use that here too. Curb is the app for local taxi service through a couple of providers. You can get a bus to pretty much anywhere without too much of a wait. Ride the MAX and streetcar if you’re headed to areas that they service.
  • Oregon has no sales tax. Do your Christmas shopping while you’re here.
  • It gets dark around 16:30 – 17:00, so factor that into any plans for hikes, etc.
  • If you go into the mountains, check the road conditions to see if you need chains for the car, or if the area is accessible at all.
  • Go to Powell’s books. Go a few times. Then go back again. Check out the Science Fiction pillar there – how many have you read?
  • Get high… for a good view of the city: Have a drink at Departures, dinner at Portland City Grill or visit Pittock Mansion for great views of the city. Even driving over the Fremont bridge (Highway 405) provides a spectacular view of downtown, the Willamette River and our many bridges.
  • There aren’t a ton of restaurants/bars/coffeehouses that can hold large numbers of people. If you break up into groups of four (max), you’ll have a much better time and the servers, chefs, baristas and bartenders will like you for it.
  • Be patient. Oregon is home to the slow-food movement in America. Regardless if this explains why your coffee took 20 f***ing minutes to make, it’ll make you feel better to think about that. See also the above note about small group sizes to mitigate the waits.
  • Tipping: I tip a dollar for a good coffee. I generally tip ~20% for any kind of sit-down meal. But I also worked as a barista, server and chef before getting into tech, so I’m biased towards tipping😉

CARTS – Just the 9th & Alder pod for the purposes of this post. If you’re from California – up here they are not “food trucks”, they are “food carts”… even when they are trucks.

  • Hoang’s – Get soup #17.
  • Nong’s khao man gai – Simple and good and oddly famous.
  • Wolf & Bear – Israeli street food that is amazing. And vegan.
  • Güero – Excellent tortas. Add an egg.
  • La Jarochita 2 – Great Mexican cart. The only Mexican cart (aside from Güero) worth going to at the 9th and Alder cart pod.
  • Brunch Box – A cart that graduated to a proper storefront, but is right by the 9th and Alder carts. Great egg sandwiches for breakfast (try the Black and Blue) and terribly good burgers the rest of time. Very fast.
  • Kargi Gogo – Georgian food. The country, not the American state. Irakli says it’s legit too.

DRINK

  • Multnomah Whiskey Library
  • Clyde Common
  • Pepe le Moko
  • Teardrop
  • Bailey’s Taproom

COFFEE

  • Courier – They also make fresh canneles daily.
  • Heart – Excellent coffee.
  • Barista
  • Coffeehouse Northwest
  • Stumptown
  • Case Study
  • Coava – Not downtown but my favorite coffee roaster in town.
  • Ristretto – Not downtown, but fantastic coffee in beautiful spaces. The Nicolai location is especially great – check out the custom metalwork and crazy woodwork, and visit Schoolhouse Electric in the same space.

EAT – I’ve only noted a few of my go-to places, or those that are near where Mozillians are staying, or open late, etc. I’ve got a somewhat-up-to-date list here with more recommendations, but there’s just really too much to cover. Go out and explore.

  • Clyde Common – Always interesting menu, good whiskey selection, great cocktails. Fries + harissa and a cocktail is a great way to end your day and start your evening.
  • Tasty and Alder (downtown) – Shared plates, everything is good.
  • Luc Lac – Vietnamese + cocktails, serves food VERY late, and closest place worth going to that’s also near the Waterfront Marriot.
  • Bunk – Excellent sandwiches.
  • Lardo – Excellent sandwiches as well.
  • Racion – Amazing cocktails and fun Spanish-tinged molecular gastronomy.
  • Little Big Burger – Fast and good late night burger.
  • Not downtown, but places I always want to go back to…
    • Biwa – Izakaya, open late, always amazing. Burger served only after 11pm.
    • Tarad – Northern Thai food, open late. Pok Pok gets all the attention but after living in northern Thailand, this is where we go for the real thing.
    • Mee Sen – Another good Northern Thai food spot we go to.
    • Podnah’s BBQ – Worth heading up into NE Portland for.
    • La Taq – Next door to Podnah’s, fantastic Mexican bar food.
    • Sweedeedee – Most likely to feel like you’re in Portlandia at this spot. Amazing house-baked breads. Tiny. Expect a long wait but good people watching and excellent food.
    • Por Que No (Mississippi location) – Great tacos.
    • Ha & VL – Get the special soup of the day, whatever it is.

NEIGHBORHOODS – Places to walk, eat, drink, shop and experience parts of Portland that are far more interesting than downtown.

EXCURSIONS

  • Columbia Gorge – See Multnomah Falls, the view from Vista House, climb Beacon Rock, stop at the Bonneville Dam and see Herman the Sturgeon, drive across the Bridge of the Gods (and back), then go for lunch at one of the Hood River breweries. Some easy/moderate 1-2 hour hikes I like: Angel’s Rest, the Multnomah/Wahkeena Falls loop, Triple Falls,  Horsetail Falls, Punchbowl Falls, all listed here.
  • Mount Hood – Drive out Highway 26 to the historic Timberline Lodge with amazing views of Mt Hood (if it’s clear), then down to Hood River and head home through the Columbia Gorge on Hwy 84.
  • Mount St Helens – A couple of hours north, but the view of the blast zone and crater from the visitor center on the NW side is well worth it if the view is clear. On the SE side you can hike one mile underground through Ape Cave, and check out the amazing rock formations and brave the cable bridge at Lava Canyon.
  • Oregon Coast – About two hours drive. Walk the promenade at Seaside, get ice cream at Tillamook, watch storms from the tip of Oregon at the mouth of the Columbia river.

Firefox OS: Devices and Dark Matter

UPDATE: Scroll down for update on May 26, 2013.

Since beginning work on the Firefox OS project, the number one question I’m asked is “Does it run on my phone?”. Sadly, the answer for almost everyone is “no”. The question itself is interesting though, and shows how people – even geeky technical people – don’t have a good understanding of how mobile devices work, nor the whole business and technical ecosystem that brings these things into the hands of consumers (hm, maybe that’ll be my next blog post). Porting an operating system to a device is tricky work in the best of circumstances and when done without the direct assistance of the various business entities involved in the stack for any single device (OEM, chipset manufacturer, original OS vendor), involves a lot of, well, fiddling around. The kind of fiddling around that voids warranties and turns $600 hardware into a paperweight. The success and hackability of Android simplified things a lot, creating a relatively large community of people doing OS-to-device porting, and enabling a lot of what allowed Firefox OS to bootstrap so quickly. However, it’s still not easy.

I was curious about who is playing around with Firefox OS in the dark corners of the Mos Eisley of the device-porting porting world,the XDA-Developers forums. Below, I’ve listed a number of threads involving efforts to port Firefox OS to various devices. Some have builds, some are aborted attempts, but the totality shows the level of interest in putting a truly open Web operating system on low-powered commodity mobile hardware that is very exciting.

Oh, and if you’re interested in porting Firefox OS to your device, the basic instructions to get started are on the MDN B2G Porting Guide. If you scan any of the threads below or have ever tried doing this kind of work before, you already know: Thar be dragons. You could lose your device and your sanity, and will very likely void the warranty. Consider yourself warned.

There are also some efforts at porting to other types of devices, such as Oleg Romashin’s experiments with Firefox on Raspberry Pi, MDN instructions for building for Pandaboard, and a bug for some core changes to Firefox OS to ease porting to basic Linux systems like Beagleboard and Chromebox.

 

UPDATE May 26, 2013

New devices since this was originally posted, and some fantastic updates:

  • Raspberry Pi! Philipp Wagner, a student from Germany, updated Oleg Romashkin’s porting work, and wrote the Raspberry Pi for Firefox OS guide, where you can download builds and read instructions for building it yourself. I installed his build and was up and running on my Raspberry Pi in minutes. Check out his blog post, and buy him something from his Amazon wishlist🙂
  • As I’m sure you’ve heard by now, Geeksphone has two devices available with Firefox OS pre-installed now available for purchase. The devices keep selling out FAST, so keep a watch on their Twitter account. Also, according to their forums they’re going to make nightly updates available over-the-air soon, so you can stay on the latest versions of Firefox OS.
  • At this year’s Mobile World Congress, Sony released ROMs for the Experia E.
  • The XDA-Developers blog reported on a Firefox OS port for the HTC HD2.
  • Also on the XDA-Developers blog was a port for the HTC Explorer (aka Pico).
  • You can find build instructions for running Firefox OS on Pandaboard up on the Mozilla Developer Network.

A couple of other notes:


Firefox 11 is Smaller and Faster

We quietly shipped Firefox 11 with a bunch of performance fixes that both reduce the amount of memory that Firefox uses, and improve the responsiveness of it’s user interface.

These types of changes are not easy to talk about. Often they’re very technical, and meaningless to anyone but the developers involved, which is probably why we usually don’t enumerate them in the the release notes or other public announcements. “Firefox is 74% faster when you open menu X, and twice as fast in some garbage collection scenarios!” Yeah, not an eye-popping headline. We could do a lot better in communicating these improvements in broadly meaningful ways though – nice graphs or some competitive site like arewefastyet would help a lot. But until then, here’s a short summary of the improvements in Firefox 11. And if you know of other performance fixes that don’t fall into the categories below, please add them in the comments!

Memory Use (aka “memshrink”)

The Memshrink project has been going for quite a while now, led by Nicholas Nethercote. He blogs weekly updates on the project’s activity. According to Bugzilla, there were 29 memshrink bugs marked fixed during the Firefox 11 development cycle – four of which were P1, or very high priority. Some of the fixes were related to tools and detection methods, but many are actual reductions in memory use. The changes that made it into Firefox 11 include fixes for detected leaks, removing of zombie compartments, lazy-loading data, reducing the size of some caches, reducing memory used while scrolling, and many more.

UI Responsiveness (aka “snappy”)

The Snappy project started last December, and is run by Taras Glek. Its aim is to improve the responsiveness of the Firefox UI. Taras has been posting weekly updates on Snappy activity on his blog. Bugzilla shows 15 snappy bugs marked fixed during the Firefox 11 development cycle. The project had just started, but there are still some significant wins in this release! Firefox 11 includes reductions in queries in the bookmarks system, reduced preference lookups, faster data collection for session restore, and various improvements in the DOM code.

Add-on Compatibility

While it’s not related to performance, I do want to call attention to something that many people don’t seem to know: In Firefox 10 (yes, the previous release) we stopped marking most add-ons incompatible when you upgrade. That means that a LOT more of your add-ons will continue to work when you upgrade Firefox from here on out. The only add-ons that still require compatibility bumps are those that have binary components, since they need to be recompiled against the current version.

Download Firefox 11.


Browser Services Update (TGIF)

My team is “a riddle, wrapped in a mystery, inside an enigma”. Ok, maybe not *that* mysterious, but we’re definitely involved in a variety of projects. Here’s a roll-up of a week in browser-land.

Performance

New Tab Page

New Download Manager

  • Paolo consolidated existing patches into a single one for final pass and check-in. Marco has been reviewing.
  • Paolo and Marco are getting together in Novara on Saturday to sprint on fixing theme bugs and other cleanup required for landing.

Add-on SDK Integration

  • Met with Irakli to create plan for landing the core SDK modules in Firefox so that the whole SDK would no longer need to be bundled with every add-on.
  • Created a feature page for tracking the project.
  • I talked with Brian Warner about syncing Git repos with HG repos.

Web Apps Integration

  • I updated the feature page with latest from UX, Apps and PM.
  • Felipe Gomes is driving the Firefox side of the integration work, with Tim helping out on UI. They’re working together with Fabrice, Myk, Dan and Tim A.

Identity Integration

Share Integration

Evangelism/Engagement

  • Tim and Marco kicked off sponsorship process for Italy’s jsDay conference. They’ll be putting up a booth there, along with Paolo Amadini, representing Mozilla. Thanks to Stormy and Shez for the support from Developer Engagement.

Of course, we all worked on various other things as well, from code review to bug triage to random maintenance fixes. Activity logs and whatnot are listed below.