Basic Time Slider in Closure

Earlier this month I was playing with a the Google Closure library and using the slider. I wanted to play with it for developing a time slider too. Since I'm going to History Hack Day at the end of January in London, I thought I'd show off some samples there and this was a perfect hack for that.

Anyway, it's pretty simple. You set minimum and maximum values on the slider that correspond to the millisecond values used by JavaScript to represent time. Then use the JS Date object to convert those to readable date and time for display under the slider. You can see the sample here. I set the slider to between Wed Dec 31 2003 23:59:59 and Fri Jan 01 2010 00:00:00 since I knew all dates in the underlying table fell between them. I used another Wikileaks table, the Afghan War Diary, 2004-2010 table.

The tricky part is that Fusion Tables doesn't document the date formats that it accepts for queries. We just haven't gotten around to changing the docs, but I happen to know that we accept these formats:

MM/dd/yy
MM-dd-yy
MMM-dd-yy
yyyy.MM.dd
dd-MMM-yy
MMM/yy
MMM yy
dd/MMM/yy
yyyy

AGU 2010: Data Visualization

So, this was the first year without a digital globes track. I think two things happened:

1. Organizers got tired of doing it every year
2. More important: Digital globes are now all over the American Geophysical Union's annual meeting. There's really no need to separately call it out.

So this year, I presented to Visualization Aided Data Analysis: Tools and Techniques for the Geophysical Sciences. Smaller than my other years at AGU, this was still a vital record of the different tools needed for data analysis. I was particularly impressed by Hank Childs' presentation on VisIt: A Tool for Visualizing and Analyzing Very Large Data, which really seemed to be a tool for visualizing anything in any possible way. Wow, awesome tool. I was also really impressed by Vis Research: Advances in Visualization Research by Ken Joy, U.C. Davis. He had really impressive visualizations of flow patterns.

My own discussions of Fusion Tables, Earth, and Earth Engine seemed a bit more basic, but perhaps that's the point I've been trying to make, that visualization tools can be powerful and hard to use, putting incredible versatility and detail in the hands or professionals, or easy to use and basic for everyone. And when they are easy and basic, everyone can use them effectively.
 Anyway, here's my slides:

Freebase Meetup and JSONP

So, I was invited to speak at a Freebase meetup by Kirrily Roberts. I thought "Cool, Freebase. Oh, I should learn that!" Especially now that Freebase is part of Google! So apologies if you're looking for my intended next sample which was closure as a time slider, that will come.

I did a couple of maps mashups to try and learn the query language. But at the same time, I wanted to learn more about Google Closure. So, here's a couple of samples that I did. Let me explain in further detail:

<link href="http://code.google.com/apis/maps/documentation/javascript/examples/standard.css" rel="stylesheet" type="text/css" />
<script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>
<script src="goog/base.js"></script>
<script>
   
goog.require('goog.net.Jsonp');
var obj;
var map;
 
function initialize() {
  var latlng = new google.maps.LatLng(0,0);
  var mapOptions = {
        zoom: 2,
        center: latlng,
        mapTypeId: google.maps.MapTypeId.ROADMAP
  };
 
 
  map = new google.maps.Map(document.getElementById("map_canvas"), mapOptions);
  getEvents(); 
} 

function getEvents() {
  var envelope = 'http://www.freebase.com/api/service/mqlread?query={%20%22query%22:%20[{%20
        %22type%22:%20%22/time/event%22,%20%22id%22:%20null,%20%22name%22:%20null,
        %20%22included_in_event%22:%20[{%20%22name%22:%20%22world%20war%20ii%22%20}],
        %20%22locations%22:%20[{%20%22geolocation%22:%20{%20%22latitude%22:%20null,
        %20%22longitude%22:%20null%20}%20}]%20}]%20}';
  getData(envelope,'');
} 

function getData(dataUrl,data) {
  var jsonp=new goog.net.Jsonp(dataUrl);
  jsonp.setRequestTimeout(100000);
  jsonp.send('',setObject);
}

function setObject(response){
  obj = response['result'];
  addMarkers();
}
function addMarkers(){
  for(var i in obj){
    for(var j in obj[i].locations){
      var latLng = new google.maps.LatLng(obj[i].locations[j].
            geolocation.latitude,obj[i].locations[j].geolocation.longitude);
      var marker = new google.maps.Marker({
          map: map,
          position: latLng,
          title: obj[i].name
      });}
      console.log(obj[i].name);
  }
}

So what I did there was use goog.net.jsonp from Closure to load the JSON from the query results and then create markers. The first sample finds events during World War II and find their geolocation. Since some events have more than one geolocation, it gives each location a marker with the same title, which will show up as a tool tip on hovering over the marker. In this case, the event contains a /location/location node which has /location/location/geolocation nodes. Someone comment if I'm not getting that quite right.

For those that don't know, I used the JSONP utility to get around the same domain policy of browsers, which prevents you from loading scripts directly from domains other than the one you are loading from. Server side scripting would have solved that issue for me easily, but I decided to set myself the additional challenge of finding that workaround. JSONP takes advantage of the caveat to the same domain policy that allows you to load scripts into <script> tags in HTML. In the case of the Closure library, it creates a temporary <script> tag, uses it to get the JSON object, and then gets rid of the script tag. One must exercise caution when using this approach, of course, that the source of your JSON is trusted to prevent attacks on visitors to your site.

My second sample took a /location/location, in this case Berkeley California (where I grew up) and found all /location/location nodes that had a containedby relationship to Berkeley California. And then mapped them. Then I added in animation to, so if you click on one place it bounces. Just for fun.

It was pretty basic. Figuring out the query language is not easy. Actually, before I spoke at the meetup, Jamie Taylor did a great talk about the schema which really cleared up some things for me. Unfortunately, we experienced a complete A/V failure in the room the meetup was in> Jaime did his talk with a white board, which I think made him happy. I jokingly referred to my talk as interpretive dance. I described my approach, why I was using JSONP, what tools were available, and then we went and had a beer.

Update

I should have credited this article on JQuery and JSONP with part of my inspiration.

Playing with Closure UI Library: Slider

For Google Developer Day in Saõ Paulo, my colleague Ossama Alami created this sample, which allows you to select between three different Google Fusion Tables layers and play with queries against them. I looked at that and decided that I would use it to play with the Closure Library. Specifically, I wanted to play with the UI library. So for the GDD events in Munich, Moscow, and Prague, I decided to add a slider. The results were fun, and instructional for me. The code to add a slider was pretty simple. Here's my sample, and here's the slider code:


<script>

var el = document.getElementById('s1');
s = new goog.ui.Slider;
s.decorate(el);
s.addEventListener(goog.ui.Component.EventType.CHANGE, function() {
document.getElementById('out1').innerHTML = s.getValue();
});

goog.events.listen(
s.getContentElement(),
[goog.events.EventType.MOUSEOUT,
goog.events.EventType.KEYUP,
goog.events.EventType.MOUSEUP],
function() {
var preset = document.getElementById("preset").selectedIndex;
var query = presets[preset].sampleQuery + s.getValue();
document.getElementById('query').value = query;
layer.setTableId(parseInt(document.getElementById('preset').value));
layer.setQuery(query);
});
</script>


I just took that from the documentation. Of course, there's a an HTML element for the slider, and some CSS to style it, and loading the library itself. Well, you can view source. The instructional bit was that of course as soon as you move the slider, events start firing. If you listen for a CHANGE event, as you slide the slider, it'll fire too fast. Each tick, it'll try to grab a new layer from Fusion Tables. That'll consume lots of bandwidth as FT tries to return a layer with each tick. So instead, I listen for MOUSEOUT, KEYUP, and MOUSEUP events, which fire when the user is done moving the slider, either moving off it or releasing the mouse.

Next up, I'm going to try to use it to simulate a timeslider.

Google Developer Days in Europe

I just got back from Google Developer Days in Munich, Moscow and Prague. OK, I got back on Thursday, but I'm just coming out of my jetlagged state. I presented similar talks in all three places, though in Munich I talked about Latitude, and in Prague Jarda Bengl joined me and talked about Street View. My demos are all linked in the slides. The theme was New Features in Google Geo. Here's the Munich slides:

Here's the Prague slides, with Street View at the end:

In Moscow, I went a little slower, cut out the Latitude and Street View slides because of the simultaneous translation. I did have a separate slide deck, but it adds nothing over the other ones.

Fusion Tables was the killer, though, very popular in all three cities, especially with the new spatial queries. Turns out, people love one click visualizations and easy Maps API integration. Especially fun was the demo I did using a Closure slider to dynamically change the FusionTableLayer query. I'm going to refine that some more, play with it and then do a write-up. I'm trying to take the time to learn Closure finally.

Finally, here's some pictures I took from GDD Moscow and the GTUG hackathon the next day. Unfortunately, people were pretty wiped by the next day, as there was a Android/HTML5 hackathon the day before. Some folks had traveled 16 hours on the train each way to go to GDD. Only about 20 people showed to the Geo hackathon the next day, but the energy level was great and people seemed to have a lot of fun.

MoscowGDDAndGTUG2010

Presentation at NACIS

I presented today at the North American Cartographic Information Society annual meeting, on new features in Google Earth Pro, Fusion Tables, and the Google Maps API. NACIS focuses a lot more on design, on cartography, and therefore it was a very interesting conference to be at, different from the usual GIS and developer conferences I present at. I sat in a session on rethinking the bike map afterward. Unfortunately, I had to leave after that, so didn't get to participate in much of the conference.

Here's my slides. There were a lot of questions on all aspects, but particularly on imagery.

My Trip to University of North Texas

I had a great two days at the University of North Texas, organized by Andrew Torget, professor of History. I met with historians, engineers, folks from the College of Information, geographers, and more. Most of the meetings were unstructured, free form meetings. Those are sometimes the most fun. Today I did a presentation that was mostly demos from the new features in Google Earth Pro 5.2 and Google Fusion Tables. There was a lot of interest in the different features, and there's a link there in a getting start tutorial I threw together.

Plus I'm really starting to like goo.gl, especially now that it's open and you can track the click.

Slides from Google Developer Day Tokyo: Mapping on your phone

Different from previous mobile presentations I've done, focuses more on optimizations like Fusion Tables layers, KML layers, styled maps and a few other things. I also presented the following demos:

Styled Maps

Geolocation Chrome didn't do so well at geolocation, but the Samsung Galaxy did.

Washington Post Marathon site

Fusion Tables in V3 sample

Post Google Developer Day Fusion Tables Tech Talk

Today, I presented a tech talk on Google Fusion Tables. It was really the first time I did one. The slides (below) are pretty minimal, because Fusion Tables is an amazing presentation platform. I mostly presented from a boundaries KML that I had uploaded to Fusion Tables. I got the data originally from Natural Earth, which is an AWESOME site, with tons of great data. I also used this simple page to show Maps API V3 integration as well as embedding. Fusion Tables is officially the new hotness.

My Slides from Last Night's Ignite Spatial NoCo

So, Google failed me. The first link for Ignite Spatial NoCo was a previous event, and I didn't check the date. I ended up going to Fort Collins, and then back to Windsor. Unfortunately, I missed half the other great speakers, but the event was fun. This is my 5 minute talk from last night.

My Slides from TimesOpen 2.0 Event on Mobile/Geo

There were about 100 people there, and it was great fun. The developers were all really engaged. When I asked how many had used the Google Maps API, almost everyone raised their hands. I almost went home at that point. And lots of interest in Fusion Tables, which I only did a demo of, didn't put in the slides, but check it out if you haven't already.

Accounting for GIS

 I've been thinking a lot lately about what is missing from the Geoweb. As I am 41, I naturally look back to the early days of mass computing for helpful comparisons. Thinking about the 80s and 90s, I realized just recently what I was looking for. I was looking for everyone.

Stay with me here.

Before the advent of mass computing, accountants were a specialized profession. In order to balance your books, generally you would have a bookkeeper or accountant who went through reams of paper, and did what most people thought was an arcane specialized job. Today, there are still accountants, they still do an arcane specialized job. The difference is that there is a standard type of application, the spreadsheet, that opened up a tool of their job to pretty much anyone.

So a portion of their job, playing with numbers, became open to millions of users. Spreadsheets, and the analysis of data, have become an essential tool for business. People can publish and analyze data in ways they were never able to do before, and have come up with many unique ways to use spreadsheets that accountants never would have. Arguably, Microsoft Excel is the most used database in the world, and the vast majority of users are not accountants.

That's what I want to see happen with GIS. I want to see applications where people are able to manipulate and visualize Geo data even without programming experience or GIS training. I think we've come a short ways toward that. Applications like Google Earth and Google Maps let you visualize data through a standard GUI interface. Maps APIs allow people with some programming skills to put a map on their site.

But nothing makes it easy. People come to me all the time and say "Here's a spreadsheet, how do I put all my stores on the web? Add in driving directions? OK now I want to add in other things..." Data driven apps are still harder to do. I think Fusion Tables is getting there, but needs more. It needs basic spatial queries (point in polygon for instance). And it needs the ability to upload other GIS data formats, not just KML.

But wait, I hear some GIS Pros cry, you'll take jobs from us! If people can visualize their own data, why would they need us? And besides, we do a much better job than they do.

It's true, good GIS pros will produce better analysis than the average person on their own. But I doubt that GIS Pros will lose jobs. What happened with spreadsheets was that whole new uses for data were developed. People created spreadsheets to track and chart all sorts of things. Sure, probably a few accountants lost jobs, but there are still almost 1.3 million accountants and auditors in the US, with their prospects growing. I think putting the ability to visualize geographic data into the hands of everyday users will increase the demand for specialized knowledge, increase the general awareness of geographic data opportunities, and only be good for the profession.

Thoughts on GeoWeb 2010

I went to GeoWeb in Vancouver this year, and ended up presenting two workshops.

New Features in Google Earth
https://docs.google.com/present/view?id=df795pd2_204d6g5n7d2

Going Mobile with Google Geo APIs
https://docs.google.com/present/view?id=df795pd2_207hpdznv5n 

GeoWeb is an interesting conference. It has roots back to at least 2002, where it was a GML conference. It's origins as a GIS conference have colored participation for a number of years. But this year it was more than that. It's clear that most GIS professionals, vendors, etc. have gotten it, the web is important.

Though attendance was sparse, like most conferences this year, the approximately 200 people in Vancouver were enthusiastic and really engaged. I had a bunch of GIS types making their first Google Maps API app in my second session.

And there's spontaneity as well, with new sessions added during the conference.

Google's had a historic commitment to GeoWeb, and I don't expect that to change. There are some key decision makers there, and a fair number of developers as well.

One thing we have to all figure out, though, as a community is how to bridge the final gap between the web and GIS. There's a lot of efforts to do it, but nothing easy. That's the fundamental thing I think the GIS community misses out about the web, it's supposed to be easy. Any truly disruptive technology starts out simple. Maybe that's why Wave didn't work out. And anything that requires an Arc* license isn't easy either.

I'm not saying Google's figured it either, though we're moving there with Fusion Tables and the next conversion features in Google Earth Pro etc. But fundamentally, we need more features and tools that are easy to use. And cheap. Without that, we're not going to make GIS pros into mashup developers.

Ruminations on the 5th Birthday of the Maps API

A few weeks ago, we celebrated the 5th birthday of the Google Maps API. We celebrated again the next week. In a transpacific video conference, the Geo teams in Mountain View and Sydney ate cake and drank champagne. Speeches were made, memories recounted.

In particular, Paul Rademacher gave a brief history of Housingmaps, the first known Google Maps mashup, created initially before there was even an official API. It was thanks to the work of Paul, and several other mashup creators, that Google saw the potential to create something really special.

I often say that Google had two choices at the time. We could either sue Paul and others who reverse engineered the API and created mashups. Or, we could go with it. We went with it. I say "we" btw, as if I had anything to do with it, but it wasn't until a year later that I started at Google.

As I recounted that story at WhereCamp Socal on Sunday, I asked people what our choices were, and Tim Craig shouted out "Kill him or hire him!" Maybe that's the difference between ESRI and Google :-). (Tim is actually a great, gentle guy. As far as I know...)

It's hard to remember a time before mashups now. It's not that Google Maps was the first mashup ever, but it was the first monster mashup platform, and still the biggest. And it is having repercussions beyond people putting maps on their sites. As soon as people figured out that they could mashup maps with data, they started looking for data to do it with. And putting pressure on governments and companies to make that data available. Or simply going out and generating their own.

OK, those of you old enough to think back that far, try to remember the web before all these mashups. Still a cool thing, but not nearly as exciting. I'm old enough to remember a time before the web, of course, but that's beside the point. The fact that we can put a map with high resolution satellite imagery on our web site for free is amazing in retrospect.

The impact on the mapping community was also pretty profound. I think we're still figuring out what the implications of that are. Neogeography, the geoweb and all the other things many of us hold dear, they take off with the Google Maps API.* It is hard to imagine this, but 5 years ago who would have thought this would happen? I know it's a cliche to say that about the web in general, especially for those of us old enough to remember before the web, but 5 years. 5! Can you imagine going to a site for a retail store and not seeing a map to their location?

But there's a lot more here than putting dots on your map, showing where your store is. That's important, revolutionary indeed, but think beyond that.

Maps mashups have been used to map election violence in Kenya, report potholes in the UK, show reports of people in need after the Haiti Earthquake, convey comprehensive data about Africa, and much more. Creating advocacy maps, informative maps, or fun maps no longer requires a professional cartographer. Nothing against cartographers, but put the power in the hands of the people.

So, what happens when you start mashing up data? You start looking for more data. The confluence of the Open Source movement with the mashup community produced a call for more open data. And because of the power of these mashups to put data in front of the people's faces quickly and easily, governments had to respond. In the US, in the UK and around the world, more and more data is being opened up by governments, NGOs, and to a lesser extent corporations. Bringing data to the public is a democratization. Knowledgeable, informed citizenry can respond to their governments, and help out as we saw in the case of the CrisisCamps that sprang up after the Haiti earthquake.

We still have a long way to go. Good tools are developing, but most people aren't programmers, to take advantage of the API, or know much about geographic data. That's why I'm excited about GeoCommons and Fusion Tables, because they allow people to use tabular data (say, in a Excel), probably the most used "databases" in the world.

But all those involved in mapping on the web, pat yourselves on the back. Look where we are compared to 5 years ago. I'm guessing most of those reading this blog are involved in Geo in some way, and so I'm saying to you: Thank you, you've done a great thing for thing for the world. You're bring democracy to the world. I know that sounds hyperbolic, and I know it's not the only thing driving increased access to information. But mashups, driven by mapping mashups (Google and otherwise) are helping change the world.

Map Styles and Usability: Please Help

I've been away for a couple of weeks, and I'm now catching up with what's been going in the Geo blogging community. I saw a couple of posts on the Google Maps API new styling features:

Thoughts about Google’s Styled Maps announcement

Google Map Styles

There were more of course, but Steven Romalewski and Richard Treves raise some good points about usability. We've basically provided no guidance on usability of the new styles. The examples that we provide are designed to show extremes of styling to get the point across.

The truth is, we're mostly engineers, not cartographers. I'd love to see some great guides to how to style your map. Anyone want to give it a go? Anything good out there, I will make sure we link to it and talk about it.

To be fair, Steven is also concerned that it'll actually drive more people to use Google Maps API. His concern, our hope of course :-). He is concerned that this will reduce the commitment to other mapping platforms and perpetuate a mono-culture of maps. I don't see any danger of that right now, but I do appreciate that concern. Competition is good for us, it does help drive us to better things. So Cloudmade, Bing, OSM, everyone else, please make your mapping better. It helps us too.

Techniques for protecting your data

I was asked in the comments in this post: Maps to KML?, to talk about techniques for protecting your data in Google Maps applications. I'm finally getting around to that.

First off, let's just say that like any part of the web, it is probably impossible to totally protect your data that is published on a Google Map. People can always get access to it at very least by just viewing the data, which you want, and making notes. Screenshots, viewing source, intercepts, and other techniques can be used by the truly ambitious. True data privacy in a completely public page is an oxymoron. Note, I'm not saying privacy on the web is bad or not possible, but we're talking here about displaying data. As long as it's displayed, someone can get at it.

That being said, there are steps you can take to make it harder to get at, to make people work at it.

The most important thing you can do is avoid hard coding any information into the page. That should be fairly obvious, but many people fall into the trap. That means:

1. Don't have any code in your JavaScript that uses a specific Latitude/Longitude pair, an address, or anything of that nature.
2. Use calls to server resources to plot only the data necessary to display at that moment. Try to verify the origin of the requests to prevent people from scraping. Generating your data on the fly prevents someone from getting all your data at once.
3. Obfuscate/compile your Javascript code to make it harder to read.

You can also rasterize your data, or turn it into image overlays. There's a lot of techniques for doing this. This talk by John Coryat is a couple of years old, and was oriented to Maps API V2. However, in it he discusses many techniques that are applicable to V3.

Rasterization makes it difficult to extract the data directly. It can also increase your performance in some cases where you have lots of data.

If you're working with KML, you can distribute the KML to only trusted people to load in their Google Earth instances, but this is subject to trusting them. Any KML used in a Maps API application will be easily findable by someone who can get passed any obfuscation you have.

That's all I've got. Feel free to post any additional techniques in the comments.

My slides from iHub Nairobi's Mobile Monday

Slides from BarCamp Nairobi/ WhereCampAfrica

So, it was pretty free-form, and I didn't stick to the slides, but there's still good links and resources here.

And, I hate this, but OpenOffice on the Mac corrupts PowerPoint export somehow. This prevented me from converting to Google Presentations, so I created both a PDF version and an OpenOffice version.

The images in the Eye Candy section are clickable.

BarCamp/WhereCamp Nairobi

Over the weekend, I attended BarCamp Nairobi, which was combined with WhereCampAfrica. It was a great event, which filled both the iHub and Nailab spaces. Despite a fist fight that developed between myself, Stefan Magdalinski, and Mikel Marron, the event was otherwise very friendly and cooperative.

I went to great talks on:

• Shika (a local ICT4D organization working in the slums in Kenya. If anyone has a link, please pass it on in comments)
  
  
• AkiraChix
  
  
• OSM
  
  
• Seven Seas Technologies and entrepreneurship
  
  
• Map Kibera
  
  

And too much more to post on. But like all Barcamps, the most interesting stuff is in the halls. I learned a lot about the emerging tech community here, and the difficulties of getting work in the face of a small percentage of the population being online. Like most of the developing world, they are jumping straight to mobile, largely skipping a large scale computer market. And mobile devices are where most people get their net access.

Of course, the event had to break each day for World Cup games, which were put up on the projection screen at the iHub.

It looks like the iHub is the place to be. My only regret on this trip is that I won't be here on the 26th, when the iHub hosts a big LAN party for gamers.

For those of you in Nairobi, I'll be presenting tonight at the iHub for Mobile Monday, about 20 minutes on using Google Mapping technologies on mobile devices.

Geo Highlights from Day 2 of Google I/O

Wow, Day 1 at I/O was such a big day Geo, it would be hard to top it. But there were some amazing gems. Check out these highlights:

1. Styled Maps! Probably the biggest news of day 2. Maps API V3 now gives you the option to style your maps. Don't like golden highways and green forests? Change it! Check out our announcement for more details and links.
2. Matt Lowrie gave a great talk on the SketchUp API and using SketchUp.
3. Josh Livni and I previewed a whole bunch of additions to the Earth API and a new KML extension. In particular, Earth API now has control over the time slider, and better balloon handling. Now, you can preserve your JS and Flash in the balloons. In KML, we previewed the Track extension, which will allow you to assign multiple way points to a model or point and move it around, rather than recreating them with multiple Timestamps.
4. Finally, and this was actually announced on Wednesday, you can now add a FusionTable layer to a Maps API V3 app, right from the API.

It's really exciting to see all this. We're closing the loop on a lot of developer requested features, and we're really happy. Thanks for those of you who came or watch it on video.

Geo Highlights from Day 1 at Google I/O

Boy, are my feet sore!

Wow, what an amazing day! Geo rocked Day 1 at Google I/O.

First, Daniels Lee announced that the Google Maps API V3 has graduated from labs and now is the recommended version of the Maps API to use. It also means that V3 is part of Google Maps API Premier, which is something people have been asking me about.

That also means that V2 is deprecated. We'll continue to support it, and fix bugs, for at least the next 3 years. Check out the deprecation policy in the terms of service. We're also deprecating Mapplets.

We also announced Street View in the V3 API, Flash-less so you can use it on mobile browsers. See my talk for more details. Videos and slides should post soon.

We announced a Directions web service as well, allowing us to close by far the single most requested feature in the issue tracker.

And finally, we previewed a Places widget, allowing you to show Places nearby your current location. It's built on the Places web service, now in Developer Preview.

There was also a fireside chat with Geo engineers and Product Managers, a Developer Sandbox with lots of great stuff on display, and a talk on Maps Data API by Tom Manshrek.

Plus, there were tons of Geo developers all over the conference. I think I talked to half of them. If you're in the other half, come and talk to me tomorrow!

Googe I/O Excitement

At this point, I could just list every single Geo session at I/O and tell you I was excited to see them. Particularly the two that I'm presenting in. And that would be true. I could also rave about all the sandbox partners, and tell you how they were going to be fabulous. And of course they are. But the truth is, what I'm most interested in for Google I/O is all the great people who I'll meet. Honestly, developers are some of the most interesting people in the world, and I'm really happy that I'm going to be spending 2.5 days meeting those of you who are coming. So, come say hi, introduce yourself, I'd love to spend some time learning about what you're doing.

Maps to KML?

People often ask me, can I take a Google Maps mashup, and make a KML file out of it (or in some other data format)? This question never comes from a Maps developer, only from people who see a cool mashup and want to overlay the data on their own map.

The short answer is: no, you can't.

The long answer is: no, you can't, and that would be wrong.

The longest answer is: no, you can't, unless the developer chooses to make a KML file available to you. It's possible, in fact, that they are using the KML file loaded onto their map using GeoXml or some other method. Otherwise, no, and it would be wrong.

I can understand the desire to do so. After all, a mashup is a combination of data from different sources, and a Google Maps API mashup is a mashup of data with Google's mapping API, in JavaScript, Flash, or just plain image URLs using the Static Maps API. And people often use public sources, or sources that are about things that are interesting or impacting lots of people, like Sports or the volcanic eruption in Iceland. Why wouldn't that be available to you?

First, from a moral point of view, the construction of that site belongs to the developer of that site. Usually, the data isn't presented to them raw, a certain amount of development has to happen to transform the data into a format usable for the app. In fact, they may have agreements with the data provider that they don't, or some license in the download from a public site restricts redistribution.

Of course, it's possible they just didn't think of it. Ping them, maybe they would do it for you just to be nice. Depends on the data of course.

Second, think about it from a technical point of view. Flash of course is it's own beast, and can write files. If the developer so chose, they could put a download data link or something. But as far as the JavaScript APIs go, JavaScript deliberately doesn't give you file access for security reasons. So a data file can't be taken from a JavaScript page. Nor has Google hidden an API inside the Google Maps API to allow other people to pull data out programatically. Again, that would be wrong.

I'm not saying that people can't look at your code and figure out your data sources, if you're the developer. If you want to protect that data source, you should design accordingly.

More thoughts on ARML

Sean Gillies commented on my last post on ARML:

Is there anything in the Wikitude namespace that isn't already provided by KML or Atom (KML uses a bunch of Atom elements already and should probably use more)?

It got me thinking a bit more about the subject. First, yes, he's right, if KML supported more Atom links, then everything could be more readily handled by Atom. Or by their KML equivalents. I'm not sure why there's an ar:description, which could be handled by a KML description, or a wikitude:providerUrl, which could be handled by an Atom link element.

I'm concerned too about the extensions. I'd rather have a larger initial set of tags that are optional than put too much on provider specific extensions. If ARML is to really thrive, it'll be because killer browsers come along early, and standard ARML is robust enough to do what most of them will want to do. And, if developers come up with really awesome ways in which ARML can be used that aren't conceived of by the creators of ARML. I'm not sure that Augmented Reality will thrive yet, but if cross-browser AR apps are going to work, the initial standard needs to be robust enough and flexible enough without extensions. I'm not saying they're can't be extensions, just that it has to be able to stand on it's own. Browsers can choose what to implement.

BTW, despite being a KML fan, I'm not saying that KML is necessarily the only way to specify the location. It's great to see it used this way, and it has a lot of potential for cross-platform integration that way. One option might be to actually wrap things in Atom like the Maps Data API does and have additional AR specific elements surface in the Atom.

Hopefully, we can talk more about this at WhereCamp tomorrow.

Initial thoughts on ARML

Yesterday at Where 2.0, I went to a great talk by Derek Smith of SimpleGeo and Martin Lechner of Mobilizy on Augmented Reality.. It was a great talk, but readers of my blog will probably not be surprised that I latched on to document formats as an issue. Here's an example from the ARML Specification:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
...

I love seeing KML used in other markups. However, I'd prefer to see something more like this:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ExtendedData>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
</ExtendedData>
...

It may seem like a small change, but ExtendedData was explicitly designed to be a bucket of tags that are carried around with any KML Feature. While the current specification "works" in Earth, that's because it is forgiving, and other KML implementers might not parse it as well. Using ExtendedData would stick closer to the KML specification.

My Slides from Where 2.0 GeoSpatial Cloud

My Where 2.0 Slides Going Mobile

I Hate Slides

OK, I admit it, I hate slides. Maybe it dates to when I was a kid and had to sit through many slide shows by parents, teachers, and other adults in my life. Oh, I should say for those of you who are not old enough to remember this, slides used to be bits of film, with a special projectors. That may have shaped my early dislike of a static medium.

Now, today's presentations are much more lively, you can do lots of fun stuff with them. Play videos, animate text, blink tags (nooooo!!!!!) and much more. My company even makes a product where you can collaboratively develop slides.

But, fundamentally slides are a data sink. How do you get data out of a slide? Copy and paste. It promotes a style of presentation that is artfully designed, and final. Sure, you can copy slides back and forth from one deck to another. Sure, you can change how they look, and copy and paste the text into other documents. But try to get the data out of the slides programatically. Or better yet, auto generate some slides from data. Wouldn't it be great to have all your slides tagged for content, or have it in an XML format that allowed you to XQuery for specific content?

The other thing I hate is it encourages people to think they can present a topic if they have the slides. That they don't need to know the actual content, just be able to read the slides and ad lib a bit. Slides should be an end point, a representation of an accumulated knowledge.

I prefer to show code or run demos, but there's usually not a great way to share that kind of presentation off a conference web site. But there's a well established tradition of sharing slides.

I actually developed a really not great version of a slides database back in 2005. It ran on Berkeley DB and had a Java interface. I realize how hard it is to conceptualize this kind of app.

I'm not saying I have the answer either. But I'm really tired of apps that are just about presenting data and don't actually allow any access to that data. How did we get to a point where the static visualization of content was the final point of how to communicate it? Why is our model for content sharing an outmoded form of film production?

Slides from WebMapSocial on Crisis Mapping

My slides from WebMapSocial tonight. The whole subject was crisis response, and it was a great crowd!

My Slides from DevFest 2010 Tokyo

Mobile Mapping with Google APIs

Client-Side Geocoding Rocks!

Part of my job is to help partners with their code, making sure they are successfully implementing on Google Earth and Maps. The most common problem people come to me with is something like "Can you give me more quota, my http geocoding limit is reached." This is for the free Google Maps API. There's a simple solution. Don't do it!

The way the quota for the Google Maps API geocoder works is by IP address. If you put the geocoder in your JavaScript code (or Flash) it's rendering in the browser. And therefore counts against the IP address quota of the browser, not your site. If you have a single server side script that does an IP address lookup, it's going to be against the single IP address of the server. In other words, if your site is popular, you're going to run out quickly.

So, geocode in the browser and then send the geocode to do your lookup. We even allow for caching, as long as you're going to display on a Google Map or Google Earth. Caches can happen in the browser or server side.

A few additional things to think about:

1) Cloud computing services often share a range of IP addresses. If you're running on AWS or Google App Engine or any other cloud computing service, you particularly want to do client side geocoding, as you may be running quotas parallel to other apps.

2) Some mobile networks share IP addresses as well. That can cause problems for your client side software, as lots of people on their smart phones are looking at your map. If you anticipate heavy mobile use, you might consider having a server-side back-up as a fail-over. Try to geocode and if that doesn't work, send the address to your server for http geocoding.

3) If you're still running into problems with quota, you may consider a Maps API Premier license. Check out http://maps.google.com/getmaps for more information on the differences and to get started.

Slides from my upcoming talk at Berkeley

These are still rough and un-pretty. Feel free to offer suggestions, additions, etc.

Ways to Help and Resources in Haiti

This is a sort of dump of things I know about going on, piling on what others have written:

Google's Official Response Page

This page contains information on how to donate, satellite imagery, news links, and links to hospitals. We'll probably update this with more resources as they become available.

Google MapMaker

Google MapMaker is a providing data directly to the UN for relief operations, so data you provide will go directly to them.

OpenStreetMap Extracts for Haiti

Get involved with OpenStreetMaps and contribute data.

Contribute health alerts to HealthMap

Either through #healthmap or directly through their site.

Go to CrisisCamp

There's a special CrisisCamp on Saturday in Mountain View, getting volunteers together to work on technical projects, including data, maps, and technical assistance to NGOs working in the region.

GeoCommons Map of the News

Well, the title says it all doesn't it?

Get Involved with CrisisCommons

"Inspired by and started CrisisCamp DC in June, 2009. CrisisCommons is meant to capture knowledge, information, best practices, and tools that support crisis preparedness, prevention, response, and rebuilding"

Check Out James Fee's Post

He's put together a long list of resources and ways to get involved.

Workshop: Working with Geospatial Data Using Open Source Tools

I'm giving the following workshop. Any suggestions for content are welcome. And you are welcome to attend, whether or not you're a UC student.

Title: Working with Geospatial Data Using Open Source Tools

Location: 110 South Hall, UC Berkeley,

Date/Time: Monday, January 25, 2-5pm

There are hundreds of millions of geospatial data files available on the web. Many of these files are in a format that makes it hard to access the data, or combine it with other data sets. This workshop will introduce the basics of geospatial data formats, and using open source tools to work directly with geospatial data. In particular, you will learn about shapefiles, KML, GeoRSS, GeoJSON and other standard formats. And you will learn about using the GDAL/OGR packages, GeoServer, and other tools to work with the data. And finally, you'll learn about some options for displaying the data on a map.