Maps to KML?

People often ask me, can I take a Google Maps mashup, and make a KML file out of it (or in some other data format)? This question never comes from a Maps developer, only from people who see a cool mashup and want to overlay the data on their own map.

The short answer is: no, you can't.

The long answer is: no, you can't, and that would be wrong.

The longest answer is: no, you can't, unless the developer chooses to make a KML file available to you. It's possible, in fact, that they are using the KML file loaded onto their map using GeoXml or some other method. Otherwise, no, and it would be wrong.

I can understand the desire to do so. After all, a mashup is a combination of data from different sources, and a Google Maps API mashup is a mashup of data with Google's mapping API, in JavaScript, Flash, or just plain image URLs using the Static Maps API. And people often use public sources, or sources that are about things that are interesting or impacting lots of people, like Sports or the volcanic eruption in Iceland. Why wouldn't that be available to you?

First, from a moral point of view, the construction of that site belongs to the developer of that site. Usually, the data isn't presented to them raw, a certain amount of development has to happen to transform the data into a format usable for the app. In fact, they may have agreements with the data provider that they don't, or some license in the download from a public site restricts redistribution.

Of course, it's possible they just didn't think of it. Ping them, maybe they would do it for you just to be nice. Depends on the data of course.

Second, think about it from a technical point of view. Flash of course is it's own beast, and can write files. If the developer so chose, they could put a download data link or something. But as far as the JavaScript APIs go, JavaScript deliberately doesn't give you file access for security reasons. So a data file can't be taken from a JavaScript page. Nor has Google hidden an API inside the Google Maps API to allow other people to pull data out programatically. Again, that would be wrong.

I'm not saying that people can't look at your code and figure out your data sources, if you're the developer. If you want to protect that data source, you should design accordingly.

More thoughts on ARML

Sean Gillies commented on my last post on ARML:

Is there anything in the Wikitude namespace that isn't already provided by KML or Atom (KML uses a bunch of Atom elements already and should probably use more)?

It got me thinking a bit more about the subject. First, yes, he's right, if KML supported more Atom links, then everything could be more readily handled by Atom. Or by their KML equivalents. I'm not sure why there's an ar:description, which could be handled by a KML description, or a wikitude:providerUrl, which could be handled by an Atom link element.

I'm concerned too about the extensions. I'd rather have a larger initial set of tags that are optional than put too much on provider specific extensions. If ARML is to really thrive, it'll be because killer browsers come along early, and standard ARML is robust enough to do what most of them will want to do. And, if developers come up with really awesome ways in which ARML can be used that aren't conceived of by the creators of ARML. I'm not sure that Augmented Reality will thrive yet, but if cross-browser AR apps are going to work, the initial standard needs to be robust enough and flexible enough without extensions. I'm not saying they're can't be extensions, just that it has to be able to stand on it's own. Browsers can choose what to implement.

BTW, despite being a KML fan, I'm not saying that KML is necessarily the only way to specify the location. It's great to see it used this way, and it has a lot of potential for cross-platform integration that way. One option might be to actually wrap things in Atom like the Maps Data API does and have additional AR specific elements surface in the Atom.

Hopefully, we can talk more about this at WhereCamp tomorrow.

Initial thoughts on ARML

Yesterday at Where 2.0, I went to a great talk by Derek Smith of SimpleGeo and Martin Lechner of Mobilizy on Augmented Reality.. It was a great talk, but readers of my blog will probably not be surprised that I latched on to document formats as an issue. Here's an example from the ARML Specification:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
...

I love seeing KML used in other markups. However, I'd prefer to see something more like this:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ExtendedData>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
</ExtendedData>
...

It may seem like a small change, but ExtendedData was explicitly designed to be a bucket of tags that are carried around with any KML Feature. While the current specification "works" in Earth, that's because it is forgiving, and other KML implementers might not parse it as well. Using ExtendedData would stick closer to the KML specification.

My Slides from Where 2.0 GeoSpatial Cloud

My Where 2.0 Slides Going Mobile

I Hate Slides

OK, I admit it, I hate slides. Maybe it dates to when I was a kid and had to sit through many slide shows by parents, teachers, and other adults in my life. Oh, I should say for those of you who are not old enough to remember this, slides used to be bits of film, with a special projectors. That may have shaped my early dislike of a static medium.

Now, today's presentations are much more lively, you can do lots of fun stuff with them. Play videos, animate text, blink tags (nooooo!!!!!) and much more. My company even makes a product where you can collaboratively develop slides.

But, fundamentally slides are a data sink. How do you get data out of a slide? Copy and paste. It promotes a style of presentation that is artfully designed, and final. Sure, you can copy slides back and forth from one deck to another. Sure, you can change how they look, and copy and paste the text into other documents. But try to get the data out of the slides programatically. Or better yet, auto generate some slides from data. Wouldn't it be great to have all your slides tagged for content, or have it in an XML format that allowed you to XQuery for specific content?

The other thing I hate is it encourages people to think they can present a topic if they have the slides. That they don't need to know the actual content, just be able to read the slides and ad lib a bit. Slides should be an end point, a representation of an accumulated knowledge.

I prefer to show code or run demos, but there's usually not a great way to share that kind of presentation off a conference web site. But there's a well established tradition of sharing slides.

I actually developed a really not great version of a slides database back in 2005. It ran on Berkeley DB and had a Java interface. I realize how hard it is to conceptualize this kind of app.

I'm not saying I have the answer either. But I'm really tired of apps that are just about presenting data and don't actually allow any access to that data. How did we get to a point where the static visualization of content was the final point of how to communicate it? Why is our model for content sharing an outmoded form of film production?

Slides from WebMapSocial on Crisis Mapping

My slides from WebMapSocial tonight. The whole subject was crisis response, and it was a great crowd!

My Slides from DevFest 2010 Tokyo

Mobile Mapping with Google APIs

Client-Side Geocoding Rocks!

Part of my job is to help partners with their code, making sure they are successfully implementing on Google Earth and Maps. The most common problem people come to me with is something like "Can you give me more quota, my http geocoding limit is reached." This is for the free Google Maps API. There's a simple solution. Don't do it!

The way the quota for the Google Maps API geocoder works is by IP address. If you put the geocoder in your JavaScript code (or Flash) it's rendering in the browser. And therefore counts against the IP address quota of the browser, not your site. If you have a single server side script that does an IP address lookup, it's going to be against the single IP address of the server. In other words, if your site is popular, you're going to run out quickly.

So, geocode in the browser and then send the geocode to do your lookup. We even allow for caching, as long as you're going to display on a Google Map or Google Earth. Caches can happen in the browser or server side.

A few additional things to think about:

1) Cloud computing services often share a range of IP addresses. If you're running on AWS or Google App Engine or any other cloud computing service, you particularly want to do client side geocoding, as you may be running quotas parallel to other apps.

2) Some mobile networks share IP addresses as well. That can cause problems for your client side software, as lots of people on their smart phones are looking at your map. If you anticipate heavy mobile use, you might consider having a server-side back-up as a fail-over. Try to geocode and if that doesn't work, send the address to your server for http geocoding.

3) If you're still running into problems with quota, you may consider a Maps API Premier license. Check out http://maps.google.com/getmaps for more information on the differences and to get started.

Slides from my upcoming talk at Berkeley

These are still rough and un-pretty. Feel free to offer suggestions, additions, etc.