Menu 1

How Amazon is Winning

This week is re:invent, Amazon’s AWS conference in Las Vegas. They’ve been shipping new products about once every 15-30 minutes it seems this week. Everything from magic AI cameras to container management to new databases.

As the announcements keep coming it’s easy to feel disoriented, which may be exactly the point.

John Boyd

I’ve been re-reading Certain to Win, Chet Richards book about Boyd’s OODA loop applied to business. The essential idea is to create disorientation, confusion and withdrawal in your enemy while promoting harmony, speed and effectiveness in your own organization.

It takes a book or three to more fully understand how to do this, but at a high level:

  • The faster you can execute, the less able an enemy is able to predict your movements and counter them.
  • In order to know what to execute you need to:
    • Observe the situation,
    • Orient yourself to it using your knowledge (and possibly just act based on instinct, skipping the Decide step) and,
    • Decide what to do.

To get there as a large organization you need decentralized command. There’s simply too much going on to keep track of everything and plan. To get decentralization, you need a bunch of trust between people.

It looks like Amazon gets its trust from their principles: A fairly concise set of deciders on how to act. It can be used to some extent to resolve situations without referring to management (and therefore politics, I guess). For example, number one is customer obsession. You can use this to decide on something, a feature say, by asking if it’s good for the customer. You’d be surprised how often product managers might want to do things that work badly for customers.

This acts as the implicit guidance & control that goes from Orient to Act, skipping Decide. Or at least, it makes many decisions speedier and easier.

Speaking of speed, there’s another principle titled “Bias for Action”.

Certain to Win includes an anecdote about Yamaha trying to compete with Honda in the motorcycle market, the “Honda-Yamaha War” as it became known. Yamaha declared they’d build a huge bike factory presumably to reduce per-unit costs. Honda countered by producing about ten new bike designs for sale per month.

This huge iterative speed and innovation (trying things at random very fast) beat out Yamaha’s economic strategy of reducing costs and flooding the market. It’s the difference that Thiel tries to get across in Zero to One; the difference between building something new the first time and then copying it. Yamaha, I posit, was trying to out-copy while Honda was trying to create the new thing. Out-copying doesn’t work.

And thus with AWS and the whole of Amazon.

The bewildering speed of new products doesn’t make sense if you’re a competitor trying to keep up by copying. It’s too difficult to keep up and the environment is changing too rapidly. This is how a fast OODA loop works, by setting the timing of the battlefield. The timing, the cadence, is set here by Amazon at a rapid rate. In war, you want to be in the same place with your assets: Your tanks should be moving around and changing so fast that the speed hinders your enemies very understanding of what is happening. It just doesn’t make sense what is happening.

I can’t think of a better analogy for what Amazon is achieving here, whether or not OODA has consciously been deployed internally. If you look, Amazon quietly launches products all the time like this, re:invent isn’t a special case. The only special cases which come to mind are the Kindle and Kindle Phone launches, which if I had to bet, probably act as negative datapoints internally at Amazon anyway. Why do a big launch? My bet is they figured out it wasn’t necessary, they have their amazon.com homepage and it works just as well.

Amazon will continue to win for as long as their cadence outstrips their rivals. The more rivals try to copy or model it in to simple narratives like having a great website, warehouse robots or other things which are the result of the cadence, they’lll continue to lose. As soon as they start focusing on copying Amazons DNA instead of Amazons products, they’ll win. Much like various cities try to copy having big buildings and airports, just like New York, instead of copying the free market or the US Constitution.

Incidentally, this is why I think Microsoft is doing well right now. The Windows Insider Program is testing out tons of things all the time:

0

Streetview with Synchronized iPhones

Can you make Google Streetview-like images quickly and cheaply? That’s an R&D question I worked on while at Telenav after putting together the original OpenStreetView pitch.

Producing street view images isn’t trivial. Typically these are captured with dedicated hardware on dedicated vehicles driven around by paid employees. 

I remember years ago talking to people building things like this. You couldn’t use standard DSLRs because the shutters were only rated for something like 100,000 exposures. This is fine for your typical prosumer but street view vehicles would be burning through a camera every week or something. Then, the car needs a bucket-load of data storage and you put a lot of miles on the vehicle very quickly. It gets expensive quick!

OpenStreetView’s solution, and Mapillary’s, is to put a phone on the dashboard pointing forward. This gets a lot of useful information but nowhere near the 360-degree view we’re used to in street view. But, the hardware is readily available (everyone has a phone) and the people using it are working for free. So it’s a great tradeoff, really.

How to get from there to 360-degree views?

At the time, dedicated spherical hardware cameras were expensive and hard to use. Think $500+ and you couldn’t talk to them. Most had a built-in SD card and could do a few preset recording modes without GPS (because, why would you need GPS?) For a half-decent camera the costs were more like $1k+

These prices were too high for even pro volunteers to spend. How could we drop the cost so that anybody could start taking  360-degree photos?

The obvious place to start is phones since they contain everything you need: cameras, compass, processing and a variety of radios. So, of course, I took an old iPhone and taped it to the roof of my truck, with a panoramic lens on it:

Old iPhone 4 devices can be found in bulk for ~$10 each which pulls the cost down. Taking photos as you drive or walk around resulted in images like this:

Notice that most of the image space is unused. If you unroll the donut you get a 360 strip, like this:

One of the few advantages of this approach is that “real” street view needs to blur things like faces and license plates. Since this strip is so low resolution, it comes pre-blurred!

You could in theory drive a car around like this and the phone could take photos and GPS points, unroll the image and upload it all in one. But… the images are pretty low resolution.

The answer is to use more than one phone:

We can use many phones in a mount. If they all take a photo at the same time then we can stitch them together and build a panorama. There turns out to be quite a lot of subtlety in the timing, capture, upload and stitching. The fundamental limit is the lens geometry of the camera. iPhones, like other devices, vary around 40ish degrees field of view. Since you need lots of overlap for a good panorama, you start to need something like 9 phones.

You can get to less phones by using wide angle lenses and changing the geometry a little:

Because of the CCD layout you get more pixels and a wider PoV in landscape.

The mounts were built with OpenSCAD. You write snippets of code (on the left) which outputs 3D shapes on the right. Here, we make some boxes and then subtract out another box to make a phone holder. Then we rotate and build many copies of them. To hold it together, there’s a thin cylinder (in blue) at the bottom. This will output a 3D file for printing.

Actually printing this in to a piece of plastic turns out to be surprisingly painful. Simplify3D helps a lot. The 3D model needs to be turned in to a set of commands for the printer to execute (move here, print a little bit of plastic, move over here…). Every printer is different. It takes a long time. We’re a long way from “just print this file” as we’re used to with printing on paper.

Measurements in the 3D model don’t quite come out in real life, either. The plastic oozes and has a set of material properties, so that it doesn’t print exactly what you send it but may be a few millimeters off. If you print walls that are too thin they will snap. You need to print a “raft” which is a layer of plastic on the print bed to print on top of, that you later snap off.

The cycle time is pretty long. Printing something can take 5-10 hours. Then you fix something, wait another 5-10 hours and so on.

The whole process is entertaining and educational, and reminds me yet again that manufacturing physical things is hard.

The resulting panoramas aren’t too bad, as you can see above. Each phone gets different lighting conditions and the photos are projected on the inside of a sphere. What you see above is just 5 phones, or about half a pano.

The software does some magic to try and sync timing. Initially I’d hoped that since the phones are (probably, hopefully) running ntpd they’d have pretty synchronous clocks. Wrong! Instead, a server (laptop) running a thin client is running the wifi network all the phones are connected to. Each phone runs an app which wakes up and connects. The server says something like “lets take a photo in 4 seconds” and the cameras all sync to this and take a photo at the same time.

They then connect again and upload their picture and a GPS point. This is nice as you get, say, 9 GPS readings per pano. Then they start again to take another set of photos.

The server software would then (and this is where it’s incomplete) take all these photos, build a pano and upload it somewhere. The panos I built were using autopano SIFT to find overlaps in the images but we could have taken compass readings too and used those alone or in conjunction to build the panoramas.

The finished image doesn’t look bad, as you can see. But it’s long and thin and has to crop the top and bottom off the images. The full pano would be much longer and thinner.

As the project progressed, two things happened.

  1. We started getting further from our goal (cheap, simple panos) not closer. Long thin pano image strips aren’t 360 views; you can’t look up and down. The cost and complexity kept going up with 3D printing, (old iOS version since it’s an iPhone 4) software to hang everything together, car mounts, charging 9 phones at once…
  2. Readily available commercial solutions came down in price and complexity. Moto and Essential phones now have cheap panorama attachments, for example. They tend to use two fisheye lenses back-to-back in a small consumer package.

So, while this was an interesting R&D experiment and a lot was learned it ultimately didn’t work out. You can find all the code for the server, iOS client and 3D files here.

0

A Digital Globe

“Energy Flux,” data source: National Geospatial-Intelligence Agency, September 2000.

Crowdsourcing, as a term, has been around for something like 12 years according to Wikipedia. OpenStreetMap is a little older and the idea stretches back fairly arbitrarily. Wikipedia thinks it goes back to the 1714 Longitude Prize competition. That seems like a stretch too far, but in any case, it’s been around a while.

The ability to use many distributed people to solve a problem has had some obvious recent wins like Wikipedia itself, OpenStreetMap and others. Yet, to some large degree these projects require skill. You need to know how to edit the text or the map. In the case of Linux, you need to be able to write and debug software.

Where crowdsourcing is in some ways more interesting is where that barrier to entry is much lower. The simplest way you can contribute to a project is by answering a binary question – something with a ‘yes’ or ‘no’ answer. If we could ask every one of the ~7 billion people in the world if they were in an urban area right this second, we’d end up with a fair representation of a map of the (urban) world. In fact, just the locations of all 7 billion people would mimic the same map.

Tomnod is DigitalGlobe’s crowdsourcing platform and today it’s running a yes/no campaign to find all the Weddell seals in their parts of the Antarctic.

The premise is simple and effective; repeatedly look for seals in a box. If there seals, press 1. If not, press 2. After processing tens of thousands of boxes you get a map of seals, parallelizing the problem across many volunteers.

Of course, it helps if you have a lot of data to analyze, with more coming in the door every day. There aren’t that many places in the world where that’s the case and DigitalGlobe is one of them, which is why I’m excited to be joining them to work on crowdsourcing.

Crowdsourcing today is pretty effective yet there are major challenges to be solved. For example:

  • How can we use machine learning to help users focus on the most important crowd tasks?
  • How can crowds more effectively give feedback to shape how machine learning works?
  • Why do crowds sometimes fail, and can we fix it? OpenStreetMap is a beautiful display map yet still lacks basic data like addresses. How can we counter that?

These feedback loops between tools, crowds and machine learning to produce actionable information is still in its infancy. Today, the way crowds help ML algorithms is still relatively stilted, as is how ML makes tools better and so on.

Today, much of this is kind of like batch processing of computer data in the 1960’s. You’d build some code and data on punch cards, ship them off to the “priests” who ran the computer and get some results back in a few days. Crowdsourcing in most contexts isn’t dissimilar. We make a simple campaign, ship it to a Mechanical Turk-like service and then get our data back.

I think one of the things that really separates us from the high primates is that we’re tool builders. I read a study that measured the efficiency of locomotion for various species on the planet. The condor used the least energy to move a kilometer. And, humans came in with a rather unimpressive showing, about a third of the way down the list. It was not too proud a showing for the crown of creation. So, that didn’t look so good. But, then somebody at Scientific American had the insight to test the efficiency of locomotion for a man on a bicycle. And, a man on a bicycle, a human on a bicycle, blew the condor away, completely off the top of the charts.
And that’s what a computer is to me. What a computer is to me is it’s the most remarkable tool that we’ve ever come up with, and it’s the equivalent of a bicycle for our minds. ~ Steve Jobs

In the future, the one I’m interested in helping build, the links between all these things is going to be a lot more fluid. Computers should serve us, like a bicycle for the mind, to enhance and extend our cognition. To do that, the tools have to learn from the people using them and the tools have to help make the users more efficient.

This is above and beyond the use of a hammer, to efficiently hit nails in to a piece of wood. It’s about the tool itself learning, and you can’t do it without a lot of data.

This is all sounding a lot like clippy, a tool to help people use computers better. But clippy was a child of the internet before it was the internet it is today. Clippy wasn’t broken because of a lack of trying, or a lack of ideas. It was broken from a lack of feedback. What’s the difference between clippy and Siri or “ok, Google”? It’s feedback. Siri gets feedback in the billions of internet-connected uses every day where clippy had almost no feedback to improve at all.

Siri’s feedback is predicated upon text. Lots and lots of input and output of text. What’s interesting about DigitalGlobe’s primary asset for crowd sourcing is all the imagery, of a planet that’s changing every day. Crowdsourcing across imagery is already helping in disasters and scientific research and 1,001 other fields with some simple tools on websites.

What happens when we add mobile, machine learning and feedback? It’ll be fun to find out.

OpenLocate

Your phone knows where it is thanks to a suite of sensors that basically try to measure everything they possibly can about their environment. Where does the GPS think I am? What orientation is the device in? What WiFi networks can I see? What are the nearby Bluetooth devices? Have I been moving around a lot lately, accelerometer? What cell phone networks am I connected to?

Unless you’re standing in a field in Kansas with a clear view of the sky for ten minutes (so your GPS has lots of time to settle), your location will be questionable.

The original iPhone used WiFi network data to figure out where it was, because a GPS wasn’t included. Skyhook (I think it was…) drove cars around major cities sniffing for networks while recording their geolocation. Then an iPhone could look up its location by comparing what networks it could see to the database of network locations. Then, it could start adding networks not in the database it could also see at the same place.

As phones added all kinds of sensors, these databases grew and became free-floating associations of place information. We can now correlate almost anything with where you are so that if the GPS doesn’t work (because you’re inside a building, say), devices fail-over to what other clues they have to figure out where you are.

Integrating all this information is still a challenge, especially if you’re driving around a major city. The reliability of all the location signals are questionable as Pete Tenereillo outlined in a recent LinkedIn post. Driving around San Francisco, you’re still subjected to the map jumping all over the place even with high end phones and the latest software.

How users experience this can happen at the other end too, when you see your uber or delivery driver jumping around the map on their way to you:

As well as finding your location, many apps want to store it too. There’s 1,001 ways to do that. Different amounts of data, different formats, different places to send it. What ends up happening, quite reasonably, is that various location-based app developers both capture and store location data in many different ways, and there are paid-for APIs and SDKs to help with pieces of the puzzle.

What’s changed over time is the value of this data. Aggregating vast amounts of anonymized location data can help with use-cases such as building base maps for example. If you take all the GPS traces of everyone every day, you can figure out where all the roads are and their speed limits and so on. This data is equally valuable for other uses; advertising and predicting stock prices as two examples. If you know how many people went to WalMart this week you have an indication of their stock value. Things like this appear to have driven the new $164M round for Mapbox – “Mapbox collects more than 200 million miles of anonymized sensor data per day”.

What’s lacking is an open and standardized way to capture and store this data. Enter OpenLocate, an open iOS and Android SDK to simplify capture and storage of location data.

It’s supported by a long list of backers and it should remove a bunch of work when developing anything location-based, much as Auth0 removes having to set up custom authentication. For more, see the announcement blog post here!

Social Media

Scrolling social media, I’m reminded of this scene in The Hunger Games: Catching Fire:

She’s engaged. Make everything about that. What kind of dress is she gonna wear? – floggings. What’s the cake gonna look like? – executions. Whose gonna be there? – fear. Blanket coverage. Shove it in their faces.”

It’s startlingly accurate. As I scroll today, this is what I see:

  • Smiling faces at dinner
  • Government is going to kill you by taking away healthcare
  • Someone graduating college
  • Debt is at an all-time high
  • A beautiful picture of a morning sky
  • Buy this Bluetooth echocardiogram chest strap in case you have a heart attack

 

How to Ride a Bike

Twenty-five years ago I was riding bikes down flights of stairs, and I still do this today. If you ever want to do anything interesting on a bike, you probably want your weight at the back of the bike. Consider the alternative, which is to ride your bike like you ride on a flat surface. This is what happens:

Notice how the weight is all on the seat of the bike. When the bike does something you don’t want it to do, like stop, you still continue going forward. If your weight is on the seat then you have two problems. First, you have no leeway since your weight can only go up to the handlebars before you come off. Second, your arms are at a super high angle to horizontal so they can’t stop you from flying forward while the bike stops.

Now, look at this:

Notice that the weight is pretty far back and the arms are almost parallel to the surface of the ground (which is downhill). This is how you want to ride a bike, if you’re ever drunk and think you want to do something interesting.

Notice here – weight in the wrong place, arms in the wrong place:

Similarly, more stairs with weight and arms all wrong:

So – remember. If you’re drunk and want to do something interesting on a bike, put your weight at the back. Your arms will naturally have to go to the right place (horizontal with the ground).

Eclipse Poster Kickstarter

I just wrapped up the last kickstarter (details here) when the fine people at NASA put out this super accurate map of the eclipse in August. I put together a small kickstarter to print as many as possible (they’re done at cost!), learn more about it out here.

The map is incredibly accurate, right down to using a topographic model of the moon… will be interesting to see if it succeeds!

 

Kickstarter almost funded

It’s very humbling to look at this graph of funding over the last few days for the OpenStreetMap Stats Kickstarter:

I had expected the whole thing to fail, now it looks like it’ll succeed. I was asked once in a job interview about how much failure I’ve recently had. The idea was that if you’re not failing you’re not really trying – if everything is a success then you can’t be pushing the envelope.

I figured asking for $1k for a statistics site that’s relevant to a minority of a minority in the world was going to be too much to ask for. In the grand scheme of things it’s not a whole lot of cash, but still. And yet, here we are.

Speaking of failure, “failure” itself is the wrong way to model how these things work. Scott Adams has called it “having a system” instead of “goals”. Other people have called it “failing forward”. Either way – the basic idea is that whatever happens you want to win. Adams wrote a whole book about this:

In this case, if the Kickstarter fails then I can shut the project down. This for me is a clear win. I get more time and one less distraction. I don’t have to pay for the hosting any more. I also learn that tiny kickstarters aren’t going to work and not to bother trying them again in a similar context.

On the other hand, if it succeeds that’s great too. I can dedicate the time to fix the site, the hosting is paid for and it proves that there are people out there who care about it.

Setting up situations like this can be enormously beneficial – where you win either way. But, it’s still hard since my lizard brain wants to avoid anything that looks like failure and being judged by those who see it in that way.

There are plenty of smart, educated people out there who think Amazon’s lack of profit is a “failure” for example. I think it’s beautiful. For a start, the definition of “profit” is “we have no idea what to do with the money so we’ll give it to you”. Amazon isn’t running out of ideas worth funding. Second, if they spend all the notional profit then they don’t have to pay tax on it and get some percentage advantage via that. Reinvesting in this way for a few decades leads to some spectacular growth.

This all leads to an idea that’s almost too tantalizing to verbalize: Maybe it’s possible to live by doing Kickstarter after Kickstarter? The idea is insanely fun and the implications profound. If it’s possible to raise $1k in a week then that would lead to a $52k/year revenue, supposing you had 52 great ideas. Perhaps more likely are $10k kickstarters every 2-4 weeks, or $100k kickstarters every month or two. With some number of them failing, plus costs, it should still be possible to live using this method.

OpenStreetMap Stats Kickstarter

I’m attempting to raise $1k in a week via Kickstarter to fix the OpenStreetMap Stats site.

The site lets you explore OSM data by country, time and data type:

Sadly it’s suffered bit rot and some countries are broken and not updating. The $1k goes toward fixing, open sourcing and hosting it for a year or two. Else, it gets canned.

So far it’s raised $163 with 6 days to go.

How to Outsource

I shipped a short book on kindle and paperback about outsourcing – how to find people, work with them, divide things up, communicate and so on. This is based on spending years with UpWork, oDesk, elance and other tools. Quite a few people asked me to summarize the best way to use these tools and so I wrote it all up. From the amazon page:

People like you would like to outsource work to those around the globe who can do it faster, better or cheaper than you can.

This book gets you started quickly and teaches you how to think about the work, how to find the right people and how to hire them using tools like UpWork, 99Designs, CrowdSpring and Mechanical Turk.

Along the way you’ll learn best practices and tips so you don’t waste time and money outsourcing the wrong things to the wrong people.

Powered by WordPress. Designed by WooThemes