Menu 1

Archive | openstreetmap

A Digital Globe

“Energy Flux,” data source: National Geospatial-Intelligence Agency, September 2000.

Crowdsourcing, as a term, has been around for something like 12 years according to Wikipedia. OpenStreetMap is a little older and the idea stretches back fairly arbitrarily. Wikipedia thinks it goes back to the 1714 Longitude Prize competition. That seems like a stretch too far, but in any case, it’s been around a while.

The ability to use many distributed people to solve a problem has had some obvious recent wins like Wikipedia itself, OpenStreetMap and others. Yet, to some large degree these projects require skill. You need to know how to edit the text or the map. In the case of Linux, you need to be able to write and debug software.

Where crowdsourcing is in some ways more interesting is where that barrier to entry is much lower. The simplest way you can contribute to a project is by answering a binary question – something with a ‘yes’ or ‘no’ answer. If we could ask every one of the ~7 billion people in the world if they were in an urban area right this second, we’d end up with a fair representation of a map of the (urban) world. In fact, just the locations of all 7 billion people would mimic the same map.

Tomnod is DigitalGlobe’s crowdsourcing platform and today it’s running a yes/no campaign to find all the Weddell seals in their parts of the Antarctic.

The premise is simple and effective; repeatedly look for seals in a box. If there seals, press 1. If not, press 2. After processing tens of thousands of boxes you get a map of seals, parallelizing the problem across many volunteers.

Of course, it helps if you have a lot of data to analyze, with more coming in the door every day. There aren’t that many places in the world where that’s the case and DigitalGlobe is one of them, which is why I’m excited to be joining them to work on crowdsourcing.

Crowdsourcing today is pretty effective yet there are major challenges to be solved. For example:

  • How can we use machine learning to help users focus on the most important crowd tasks?
  • How can crowds more effectively give feedback to shape how machine learning works?
  • Why do crowds sometimes fail, and can we fix it? OpenStreetMap is a beautiful display map yet still lacks basic data like addresses. How can we counter that?

These feedback loops between tools, crowds and machine learning to produce actionable information is still in its infancy. Today, the way crowds help ML algorithms is still relatively stilted, as is how ML makes tools better and so on.

Today, much of this is kind of like batch processing of computer data in the 1960’s. You’d build some code and data on punch cards, ship them off to the “priests” who ran the computer and get some results back in a few days. Crowdsourcing in most contexts isn’t dissimilar. We make a simple campaign, ship it to a Mechanical Turk-like service and then get our data back.

I think one of the things that really separates us from the high primates is that we’re tool builders. I read a study that measured the efficiency of locomotion for various species on the planet. The condor used the least energy to move a kilometer. And, humans came in with a rather unimpressive showing, about a third of the way down the list. It was not too proud a showing for the crown of creation. So, that didn’t look so good. But, then somebody at Scientific American had the insight to test the efficiency of locomotion for a man on a bicycle. And, a man on a bicycle, a human on a bicycle, blew the condor away, completely off the top of the charts.
And that’s what a computer is to me. What a computer is to me is it’s the most remarkable tool that we’ve ever come up with, and it’s the equivalent of a bicycle for our minds. ~ Steve Jobs

In the future, the one I’m interested in helping build, the links between all these things is going to be a lot more fluid. Computers should serve us, like a bicycle for the mind, to enhance and extend our cognition. To do that, the tools have to learn from the people using them and the tools have to help make the users more efficient.

This is above and beyond the use of a hammer, to efficiently hit nails in to a piece of wood. It’s about the tool itself learning, and you can’t do it without a lot of data.

This is all sounding a lot like clippy, a tool to help people use computers better. But clippy was a child of the internet before it was the internet it is today. Clippy wasn’t broken because of a lack of trying, or a lack of ideas. It was broken from a lack of feedback. What’s the difference between clippy and Siri or “ok, Google”? It’s feedback. Siri gets feedback in the billions of internet-connected uses every day where clippy had almost no feedback to improve at all.

Siri’s feedback is predicated upon text. Lots and lots of input and output of text. What’s interesting about DigitalGlobe’s primary asset for crowd sourcing is all the imagery, of a planet that’s changing every day. Crowdsourcing across imagery is already helping in disasters and scientific research and 1,001 other fields with some simple tools on websites.

What happens when we add mobile, machine learning and feedback? It’ll be fun to find out.


Kickstarter almost funded

It’s very humbling to look at this graph of funding over the last few days for the OpenStreetMap Stats Kickstarter:

I had expected the whole thing to fail, now it looks like it’ll succeed. I was asked once in a job interview about how much failure I’ve recently had. The idea was that if you’re not failing you’re not really trying – if everything is a success then you can’t be pushing the envelope.

I figured asking for $1k for a statistics site that’s relevant to a minority of a minority in the world was going to be too much to ask for. In the grand scheme of things it’s not a whole lot of cash, but still. And yet, here we are.

Speaking of failure, “failure” itself is the wrong way to model how these things work. Scott Adams has called it “having a system” instead of “goals”. Other people have called it “failing forward”. Either way – the basic idea is that whatever happens you want to win. Adams wrote a whole book about this:

In this case, if the Kickstarter fails then I can shut the project down. This for me is a clear win. I get more time and one less distraction. I don’t have to pay for the hosting any more. I also learn that tiny kickstarters aren’t going to work and not to bother trying them again in a similar context.

On the other hand, if it succeeds that’s great too. I can dedicate the time to fix the site, the hosting is paid for and it proves that there are people out there who care about it.

Setting up situations like this can be enormously beneficial – where you win either way. But, it’s still hard since my lizard brain wants to avoid anything that looks like failure and being judged by those who see it in that way.

There are plenty of smart, educated people out there who think Amazon’s lack of profit is a “failure” for example. I think it’s beautiful. For a start, the definition of “profit” is “we have no idea what to do with the money so we’ll give it to you”. Amazon isn’t running out of ideas worth funding. Second, if they spend all the notional profit then they don’t have to pay tax on it and get some percentage advantage via that. Reinvesting in this way for a few decades leads to some spectacular growth.

This all leads to an idea that’s almost too tantalizing to verbalize: Maybe it’s possible to live by doing Kickstarter after Kickstarter? The idea is insanely fun and the implications profound. If it’s possible to raise $1k in a week then that would lead to a $52k/year revenue, supposing you had 52 great ideas. Perhaps more likely are $10k kickstarters every 2-4 weeks, or $100k kickstarters every month or two. With some number of them failing, plus costs, it should still be possible to live using this method.

OpenStreetMap Stats Kickstarter

I’m attempting to raise $1k in a week via Kickstarter to fix the OpenStreetMap Stats site.

The site lets you explore OSM data by country, time and data type:

Sadly it’s suffered bit rot and some countries are broken and not updating. The $1k goes toward fixing, open sourcing and hosting it for a year or two. Else, it gets canned.

So far it’s raised $163 with 6 days to go.

Explore OpenStreetMap Statistics

OSM Stats for Namibia

Ever wanted to explore OSM statistics over time and in depth? OSM Stats is for you. Notice the site asks for your location – this is just to show you your country automagically by default.

The site lets you explore by country, over time, major types of OSM data. The left-hand graph shows you the aggregate count over time, the right-hand graph shows the difference (delta) over the same time period. You can click different data types on the left, change country at the top, and change the time range just above the graphs.

You can find some interesting things. Here’s the default view for the United Kingdom:

What it shows is data growing over time. We like graphs that go up-and-to-the-right. The right-hand graph shows, as expected, the amount of data being added declining over time. This is because there’s less and less to map in the UK as I started the project there.

Compare that to Haiti:

Can you guess what the spikes in data addition are?

Now look at residential roads only in the United States:

Things are declining over time! Where are all those residential roads going? Well a small part of the answer (notice the vertical axis is 2 orders of magnitude less than above) is the growth of living streets in the US:

That’s a small taste of the things you can learn – have fun exploring the site and email me any comments.

OpenGeoCodes iOS and Android Apps – Collect Open Address Data

Open Address data from OpenGeoCodes in Durango, CO. Green pins are manually verified, red are awaiting verification.

Open Address data from OpenGeoCodes in Durango, CO. Green pins are manually verified, red are awaiting verification.

screen696x696OpenGeoCodes now has iOS and Android apps to optimize the hand collection of addresses.


Addresses are the primary limiting factor of OpenStreetMap – there just isn’t much out there that’s easily licensed and OSM itself for a variety of reasons lacks address data. OSM looks pretty – it’s a great display map. It’s also routable with a lot of work. But, you can’t find addresses on it.

OpenGeoCodes has data in the US and some starter data in Canada and the UK to try to fix this.

So what do the apps do?

The apps let you walk around and collect data. Say you’re standing outside 100 Main Street – just tap it, the app records the location and you’re done. Normally the app tries to guess where you are based on location.

But wait, there’s more! As you walk along, the app will optimize what addresses to show you. For example if you’re walking on the even side of a street going north, the app will figure this out and present you ascending even numbers. So if you enter 100 and 102, and the app knows 104 is nearby it will focus on this.

This makes it easy to walk along and just tap, tap, tap to collect data. We collect this data together and then make it freely downloadable. There’s also a mailing list if you want to get involved.

Where to from here? The feature list includes a more human design, notifications for when near places with no data, OSM upload and fixing and more. Drop me an email if you run in to any issues.


Connecting WordPress to OpenStreetMap with Auth0

If you log in to this blog you can now do so using your OSM account. If I say so myself, that’s pretty awesome. Here’s how I did it:

First we need to install the Auth0 WordPress Plugin which you can find by opening up WP, going to plugins and searching for “Auth0.”

Auth0 is just fantastically awesome. It’s a service which allows you to login to anything using anything. For example, you have a rails project and want to log in using facebook. Or you have a Node project and you want to log in using Active Directory. Or in our case, WordPress and log in using OSM. Auth0 is very extensible and developer friendly. For example, there are arbitrary JS events that fire through the login process. You can have twilio send you an SMS when someone logs in as an example.

Another way I like to think about it is like this: It used to be that you’d write a PHP app with JS front end and SQL backend. What Ruby on Rails did is meld together SQL and Ruby so now you only need to learn two things (Ruby and JS). You didn’t have to write SQL as well. Now things like meteor are removing even that so you just write JS everywhere. In a similar way – the first thing you do when you make an app is usually go build your login system. No more! Just use Auth0 and avoid all that pain. Auth0 is going to be more secure than anything you do, and immediately extensible.

Once you’ve installed your WP plugin, head over to the connection API Explorer.  Most things in your Auth0 dashboard are trivial you just turn them on and off like this:Capture

But OSMs authentication API isn’t this easy for a number of reasons. First, it uses OAuth 1.0(a) and Auth0 support version 2 out of the box as a thing on the dashboard. Second, OSM still uses XML where the rest of the world has moved on to JSON.

But it still works! We just need to use the Auth0 connection API to create a connection.

Next you need to log in to OSM. Click on your user at the top right, click settings, click OAuth settings, and create a OAuth Client. More docs are here. Once you have a OAuth app set up on OSM, you will get your client id and secret you can use with the code below.

On the Auth0 dashboard, create an application (applications -> create) and note down the client ID and secret.

On the API page, create a token with connections:create with the token generator at the top left – this lets you create new connections straight from the browser:

Capture2You also want to create a token with all the permissions around the following: clients (create and update), connections (create, read and update), rules (create and delete) and users (create, read and update). Note down this token, as the Auth0 WP plugin will need it.

Next we need to paste some code in to the box marked “body”:

  "name": "OpenStreetMap",
  "strategy": "oauth1",
  "options": {
    "client_id": "YOUR OSM CLIENT ID HERE",
    "client_secret": "YOUR OSM SECRET HERE",
    "requestTokenURL": "",
    "accessTokenURL": "",
    "userAuthorizationURL": "",
    "scripts": {
      "fetchUserProfile": "function (token, tokenSecret, ctx, cb) {var parseString = require('xml2js').parseString;var OAuth = new require('oauth').OAuth;var oauth = new OAuth(ctx.requestTokenURL,ctx.accessTokenURL,ctx.client_id,ctx.client_secret,'1.0',null,'HMAC-SHA1');oauth.get('',token,tokenSecret,function(e, b, r) {if (e) return cb(e);if (r.statusCode !== 200) return cb(new Error('StatusCode: ' + r.statusCode));parseString(b, function (err, result) {if (err) return cb(err);var profile = {user_id: result.osm.user[0]['$']['id'],given_name: result.osm.user[0]['$']['display_name'],picture: result.osm.user[0].img[0]['$'].href};cb(null, profile);});});}"
"enabled_clients": [

Remember to enter your ID and secret. Now click the “Try” button and you should get a “201” response that the connection was created.

So now we need to hook this up to the Auth0 login box (called the “lock”). Back on your wordpress site, go to the Auth0 plugin, settings, basic. Here enter your client ID and secret from when you made the application in the Auth0 dashboard. Also add the API token you created with all those permissions and click save. Next go to the advanced settings and enter the following in the CSS box:

#a0-lock.a0-theme-default .a0-zocial.a0-osm {
    color: #fff;
    width: 40px;
    height: 40px;
    background: #ffffff url('') center center no-repeat no-repeat;
    background-size: 21px;
    border: 1px solid #5C666F;

And then this in the JS box:

lock.once('signin ready', function() {
    var link = jQuery('<div title="Login with Open Streen Maps" class="a0-zocial a0-icon a0-osm " dir="ltr"><span>Login with Open Streen Maps</span></div>');
    link.on('click', function () {
            connection: 'OpenStreetMap' });

    var iconList = jQuery('.a0-iconlist');

We need to do this to add the button on the login box for OSM. Lastly, go back to your Auth0 dashboard and the app you created. You’ll need to add a line in “Allowed Callback URLs ” which will be unique for your site, but for my blog it’s this:

That should be it, now users can log in using OSM:

Capture3Notice that I’ve turned on facebook, GitHub and so on… and a the end there’s the OSM logo. Click it and you can login with OSM!

Of course, now you’ve done this it means you can connect an iOS application, your webapp or anything else to OSM in the same way. Magic.

New Year, New Roles

It’s been a wild two-and-a-half years at Telenav helping bring OpenStreetMap to the consumer. We shipped consumer turn-by-turn navigation in the US with Scout which was for me a big first – a turning point of showing OSMs true potential.

As the OSM project at Telenav has grown the need for the visionary founder has shifted and I’m stepping back from full-time work at Telenav. I’ll be still helping part-time and helping with new projects going forward. 2016 is going to be a fun year with a great team at Telenav (including all the bright folks we brought in from Skobbler) and I know they’ll continue to push out more OSM goodness.

New Kickstarter: Every Road


I have a new kickstarter live now: Every Road! Every Road is a unique poster print of every road leading away from your house (or any other point). Above you see the bay area, driving away from the Ferry Building at the NE of San Francisco. Here’s the same thing driving away from a point in the Sunset:


The roads get thinner as you go and lead to a tree-like structure. Each print is totally unique to you. Here’s driving away from Buckingham Palace in London:


Here’s walking everywhere from Wall Street in Manhattan. Notice most route on Manhattan end up walking North and branch out across each side:


Here’s the same thing, but driving:


Notice how it leads to a totally different map because driving leads to quicker routes along the edge of Manhattan and then driving inward to each point. As opposed to walking, where your maximum speed doesn’t change depending on what road you’re walking on.

The data of course comes from OpenStreetMap, more details are at the Every Road kickstarter!

Powered by WordPress. Designed by WooThemes