Random Markers

The Hazards of Magic

2011-12-08T12:33:00.000-08:00

Designers and people who admire designers have been talking for awhile about this concept of "magic". That is, making a product that feels so magical it doesn't matter that it wasn't the first in it's field or even necessarily the most powerful, but that it brings things together in a way that seem really intuitive, beautiful and magical. This has become so common that I no longer know who generated this theory. Apple is widely held to generally produce magical devices, devices that work really well, are seamlessly integrated, and that are beautiful. The iPod wasn't the first digital music player, but don't tell that to legions of people who think it is the only music player. The iPhone wasn't the first smart phone, or even the most powerful, but it just worked and felt intuitive. Or at least that's what people tell me. To be honest I've always been partial to Android and not just because I'm tribal. But that's not important here.

Google also has that magic in web search or Google Earth. People look at it and just marvel. Old time GIS people are often frustrated by Google Earth, for instance, that they were doing maps and looking at satellite imagery, and if you just downloaded ArcView, and installed....and at that point they've lost the magic. No offense to ESRI products, they make some great stuff and for analysis there is nothing that beats them, but there isn't that easy, intuitive use to it that you find with Google's mapping products.

"Magic" often distracts people from the limitations of a device or site. Apple's products tie you into using iTunes for instance, and lock you into a platform. But people who love iPhones will line up to buy a new white version of the iPhone with little or no feature updates. Just because it's white. I'm not blaming them, it's an amazing job that Apple has done, creating that desire for a white iPhone you never knew that you had.

Netflix had the magic too, right? They have a really amazing recommendation engine, far better at surfacing movies and shows you wanted to see than any other similar site. Amazon, in my view, comes closest and their recommendations for movies and their interface are just terrible compared with Netflix. No one cared that Silverlight was being kept alive by Netflix, and we paid a nice premium to have it on a variety of devices. Ooops, can't view it in Australia, well there's always VPN right? The number of movies available, and free, and in the early days their ability to get you movies sometimes the day after you returned yours was truly magical.

Well, that's gone right? The hazard of magic, I think, is that when you lose it you really lose it. The vehemence that people reacted with was really incredible. There's been a lot of analysis of why this happened. There's an interesting piece on O'Reilly today about What publishers can learn from Netflix's problems. One of the singular moments in the whole thing was, however, when Netflix reassured us that DVDs from Qwikster would still be in the little red envelopes. The Oatmeal had a great comic on that. That was the moment the magic was gone, when people started screaming about that. The distraction, the attempt to get users to focus on things that don't add functionality, to create a mystique around a small aspect of the brand, had failed miserably.

I'm not sure United Airlines ever really had the magic. I fly a lot, and ended up on United flights frequently in the last year. They merged with Continental, and at the beginning of every flight in the last 6 months, they would put up a video with the president of United extolling the virtues of the merger. What were those virtues? Well, they were repainting planes. That's right. Continental planes were being repainted to United planes. By the third time I saw that video, which I couldn't get away from, I wanted to scream. Untied has an aging fleet of planes. Mostly uncomfortable, with TV screens placed above our heads in the ailes instead of in the seat backs, uncomfortable chairs, and food that is usually pretty bad and that you have to pay for. But they are a low cost airline that goes everywhere and has a great frequent flyer program with lots of partners. What I wanted to hear about was their plans for modernizing their fleet, or providing more leg room, or something like that. No magic at all.

OK, so what's the lesson of this? If you've got magic, you can get people excited by surface changes and get them to look away from your problems. If you lose that magic, or never had it, don't focus on the surface, focus on what people care the most about. In Netflix it was the price and then integrated app, DVDs and streaming together, working on fast, easy, and nice delivery of your product, video, to every platform. In United's case, focus on what's working for you and don't keep talking about appearance. People don't care about that as much as you think.

The School of Information's Alumni Day: What I said and wish I'd said

2011-12-07T09:54:00.000-08:00

In 2006 I graduated from UC Berkeley's School of Information. One of my professors, Bob Glushko, who runs 202, the Information Organization and Retrieval class, invites alumni to come back every year and discuss what they've done with their education. I thought I'd write it up as a blog post incorporating what I said, my responses to some of the questions, and a few bits I wished I'd said:

Bob wants me to talk about how this class helps me with my work, and I'll certainly be doing that. But I also want to talk about how your two years here will inform the work that you do, since these are probably two of the most important years of your life.

I'm Mano Marks, a Developer Advocate at Google. What that means is I travel the world helping Developers put Google Geographic technologies on the sites, and advocating on their behalf with the engineering teams. People who graduate from the School of Information have gone on to be user experience researchers, designers, coders, policy wonks, product managers, directors, developer advocates, academics, lawyers, and much more. They work at Google, Yahoo!, Microsoft, startups, design firms, consulting firms, nonprofits, think tanks, for the government, and many other places I haven't thought of.

Back in 2004 when I started at what was then the School of Information Management and Systems, or SIMS, we were trying to figure out what the program was. One of my classmates, Benjamin Hill, who taught 290TA the Information Organization Laboratory this semester, ran a survey asking people what their elevator pitch for SIMS would be. The answer that won was something like "Mumble it's interdisciplinary and walk away." SIMS, now the I School, was in search of itself. No one knew exactly what we were doing. But at the same time the professors, and hence the students, were passionate about it. I hope that hasn't changed. The dynamic tension inherent in the exercise of what-are-we-doing drove innovative new approaches and discussions that I still value today.

What does 202 do for you? What it did for me give me a greater sense of the whole picture of organizational practices. We can get lost in the details of a particular system, which database we use, whether to use Python or Ruby, all the fascinating and fun details for geeks. But what 202 got me to do is look at an organization as a system built on top of information flows. How information flows internally but also how it flows in relation to external partners and customers.

This helps me in my daily work, working with external partners who are using Google's mapping products. It helps me see, and hopefully communicate to them, that their maps are not just a map slapped up on a site, but rather a way of communicating information to their partners and customers, and gathering information from them. It is a part of an entire ecosystem of data that their company/organization/loose network uses, consumes, and produces. Our maps (or substitute your own application or document format) don't just have to be artifacts or dead things, but living interaction devices.

But beyond 202, beyond the underlying data and information flows in an organization is a whole context that the organization exists in, a legal and sociological framework that shapes and guides it. That context is what the rest of the I School experience is about. Understanding that context, and how it relates to the core concepts of 202 is what the School of Information is all about. Or at least, that's my proposal. Please challenge that, after all I'd hate for there to be a last word on what the I School is or does.

Some quick links

2011-12-01T11:04:00.000-08:00

Experimenting with different ways of distributing links. Mostly I do it using Google+ but I get concerned I post too much there, so here's some stuff I'm reading today:

WebGL Playground: a great tool for playing with WebGL code, allows you to change things live and see the results.

Hangout-3d: Demo code for using WebGL in a Google+ hangout.

Porting 3D graphics to the web: Opera tutorial. Shows techniques for porting 3D graphics from commonly used 3D graphics programs to JSON to use in WebGL.

These were pulled from Learning WebGL.

Latest imagery updates in Google Earth

NORAD Santa Tracker Coming

Thoughts on The Paperless Map

2011-11-30T08:52:00.000-08:00

I was reading The paperless book by Todd Sattersten over at O'Reilly, a similar discussion around maps occurred to me.

Sattersten references Stacey Madden who argues that, no offense to e-books, but they aren't "books" and we should reserve that word for a physical medium. Every time I've read that kind of argument on the Internet, as in we should reserve x word to mean only an older meaning of x, the person arguing that has already lost. Hacker/cracker anyone?

I am sure there are those who argued, when maps first came to the web, that they weren't really maps, they were some other medium. However, if they did, you don't hear those voices anymore. Perhaps that's because of the strong history of GIS before the web, or because maps are primarily used for a vehicle to convey information, not generally as a completed composition. Those who know more of the history of cartography, please point me to any debates I may be unaware of.

Sattersten describes an experiment where readers are invited to participate in the compilation of a book by purchasing content as it's developed and adding their comments and feedback. It's actually hard for me to imagine bothering doing that with a book, especially if it's one of fiction, but I can easily imagine contributing to a map. I just actually don't see the need to put an end point on it, as in "this map is done."

Of course, I have had many discussions on the superiority of paper maps over digital maps in certain circumstances, say out in the field where your device may not have a reliable source of power, or where the large format may make it easier to share with others. I wonder if tablets will start to fill the latter use case. And I've admired the bridging technology of the digital pen with GIS.

I think we made the transition with maps a long time ago, thank you CGIS. So while the GIS world got a surprise when we started moving to the web, it didn't immediately dismiss the medium as not a map. In fact, one of the things I've heard from GIS folks that are surprising to non-GIS people is the question "Why would I care about web maps? They're only about making maps."

Easy Panorama creation with Ice Cream Sandwich

2011-11-23T12:18:00.000-08:00

I was hanging out with +Dan Galpin in China, and he was showing me an Ice Cream Sandwich phone. ICS comes with a panorama capability in it's phone. He took a few photos and sent them to me. I spent a few minutes and pulled them up into custom Street View panoramas.

It was pretty easy to do, but there were a few caveats. The panoramas didn't contain any location EXIF headers. Not sure if that will be part of the final release. My demo didn't require it since I was just using a free floating panorama viewer. Also, the panorama camera didn't always go a full 360 degrees, so the seam between the two ends was a little uneven. The only other issue was figuring out where the center of the panorama was and coding that into the heading of each panorama. Still, it was a nice and easy way to display these panoramas. Oh, and take a look at the full version, it shows the copyright information for Dan that I coded into the panorama.

Google Developer Day Berlin, start of day

2011-11-19T00:04:00.000-08:00

I'm excited to be here in Berlin, at the last Google Developer Day of 2011. It's hard to believe we're on the cusp of 2012, and getting ready for a whole new season of conferences. At Google, we're a little worn out from 2011:

(Photos from Tel Aviv)

But we're amazed by the turn out here in Berlin, over two thousand. The most of any GDD this year, possibly the biggest single event we've ever thrown, outside of the US. The event is being held at the massive ICC Berlin. It's so big that I'm not going to the keynote, because I'm afraid I wouldn't have enough time to make it to my session which only starts 15 minutes afterwards. Seriously, this place is like the Death Star. We keep expecting to see little robots racing down the halls, and given that it's GDD that's an actual possibility.

One of my complaints about GDDs is that there is so much great content, but I never get to go to the sessions because I'm working. In a tradition going back to the first German GDD, we've got a track devoted to speakers from universities in Berlin speaking about some cutting edge research. I can't wait till the videos go up. But the truth is, I'd rather be talking to developers anyway, I learn so much from you. So if you're here, see you at my session, or in the GTUG area, or exploring some of the sandbox events. For the rest of you, see you on Google+.

Thankfulness

2011-11-17T03:46:00.000-08:00

I recently took a break from blogging, mostly due to travel. I've been on the road for almost 4 weeks, traveling around the world. Literally, around the world: San Francisco, Beijing, Shanghai, Sydney, Tel Aviv, Bern, Berlin, and on Sunday I go home to San Francisco. Excited about that actually.

But I'm back to blogging, and feeling reflective. I thought I'd share a couple of things I'm thankful for before next weeks' Thanksgiving festivities in the U.S.

I am thankful for my work. I get to travel around the world, meeting really amazing and interesting developers who are doing great projects such as the iOnRoad app developers, or the folks from PointGrab who demoed using Google Earth:

I am thankful for the Internet, or rather for the fast connection I enjoy most of the time. While I was in China for about one and a half weeks, the Internet connection was terrible. I didn't really get it, the Great Firewall of China, before. I thought certain sites were blocked, OK, but I could go on with my life normally. Nope. It seems all sites, at least all non-Chinese sites, are filtered, even if they are not blocked. They appear to be checking each site and each bit of content as it loads. Sites of my favorite web comics, for instance, such as PVP, loaded slowly. Yes, PVP is filtered in China. The image of the comic would slowly, pixel by pixel, appear. It was torturous because I didn't know which pages would load and which wouldn't. Most people I talked to used a proxy, a VPN connection to outside of China, but even that was slow because of course you're adding on layers of connection and other servers routing your content. And yet they also told me there is lively debate within China on the net about the GFWC.

So I am thankful for these things. I hope to be writing more about what I'm thankful for in the next week before Thanksgiving.

Memories of Prague

2011-10-21T09:00:00.000-07:00

When you go to Prague, people ask you a lot "Is this your first time in Prague?"

In fact, my trip for Google Developer Day in Prague on Tuesday was my fourth, and my third in the last year. My first though was in 1990.

My first visit was in 1990. I was in the midst of my junior year abroad, living in Budapest. I didn't have much money, so I took the train up to Prague, a 9 hour trip if I remember correctly, stayed 16 hours in Prague, and went back that evening. It was a magical place back then, just emerging from the fog of communism. Spared destruction during the Second World War, Prague retains a medieval core, one of the most beautiful and unblemished in Europe, in my opinion. It became the cool center for expats in the early 90s, when English speaking 20-somethings would go to Prague to teach English and drink inexpensive but delicious beer. So many went there, in fact, that a 2002 novel about expats in Budapest in 1990-91 was entitled Prague, as in they missed the boat and went to the wrong Eastern European capital. (I don't regret Budapest, BTW, but Prague did have the cachet back then).

I left my nice camera at home, so took pictures with my Nexus S. I'm particularly happy with this one:

There are plenty more pictures of Prague from last year at GDD in my Picasa albums. Some day I'll scan in all the photos from 1989 and 1990 and post those.

Thoughts on vector based mapping

2011-10-19T09:51:00.000-07:00

Traditional...wait, I have to put that in quotes..."Traditional" web mapping, as you see in pretty much any web mapping application, relies of raster tiles to convey the base map. The reason for this approach is pretty simple: You can convey a ton of information in a raster image simply by changing the colors of pixels. And it's highly performant. As many pixels in your image is as many points of information you can portray.

But there's been a lot of buzz for awhile about the possibility of vector based mapping. Vector mapping is essentially pushing the underlying data used to build a map to the client, instead of the fully assembled map. By pushing vectors to a client (browser or mobile application) you can push the assembly of the map into the client machine, saving quite a bit of work on the server side. But it also gives you a lot of capabilities that are absent from raster maps. For instance:

Reprojection and restyling on the fly: If you are assembling a map based on vectors, you're just providing the base map data and allowing the local app handle constructing the map. That allows you to do interesting things like reproject and restyle a map in the client and on the fly.
Graduated map drawing: You can draw the map while the user watches, instead of loading it tile by tile.
Perspective changes: You can tilt maps, create 2.5D or 3D maps, do all sorts of things based with the data.
Fluid transitions: Instead of jumping between zoom levels, you can have smoother transitions as you navigate a map.

This comes at a cost to the client of course, you have to have a device of sufficient power to assemble the map. Fortunately, a variety of new technologies in the browser are making that happen. Here's the basic approaches developers are taking:

Google Maps for Android: Google Maps for Android uses vectors to draw the maps. It uses a custom vector format and pushes it down to the application where it runs on the mobile device. If you have an Android phone, chances are you have vector mapping already on that device:

Scalable Vector Graphics (SVG): SVG has been around for awhile, and has even been used by Google for rendering overlays on Google Maps API maps. But it was always held back by the performance in the browser and lack of support by IE. IE9 finally gives support for SVG, and most modern browsers are now performant enough to use it. Polymaps, a JavaScript library, was the first major implementation of vector mapping for the web that I saw. It allows developers to draw maps using SVG and style them using CSS styling. I think SVG is still a little slow, but it is now widely supported and the CSS styling is a great way for developers to style maps.

Canvas: There's been a lot of hall-way talk at conferences like WhereCamps and FOSS4G about using HTML5 Canvas to render vectors, pushed down in a standard format. I haven't seen any great implementations, but Canvas is also widely supported in the browser and generally has good performance.

WebGL: WebGL gives the browser access to a machine's graphics card to do rendering, giving it some tremendous power. Browsers traditionally only had that kind of access through plugins like Flash. Google launched an experiment last week called MapsGL. It's pretty awesome. We've got buildings, smooth transitions, and nice animations of 45 degree imagery. The downside of WebGL is that it isn't widely supported yet in the browser. Currently only Chrome 14 and later versions of Firefox have good enough support for MapsGL. And currently Microsoft has no plans to support it. My hope is that WebGL will take off because of the power that graphics processor gives maps developers.

This isn't meant to be an exhaustive list, feel free to add other approaches you've seen or implemented. I'm really excited by vector mapping and want to see it succeed. I really feel like it's "The Future"

How I got to be where I am: part one of many

2011-10-05T07:37:00.000-07:00

So yesterday I received an invitation to speak at the The Apps Against Abuse Challenge: San Francisco event, which I'll be at tomorrow. Interestingly, I don't think that the inviter has any idea that I once worked at a rape crisis center and that was how I got into tech. Or, as people not in tech used to say, "into computers." I'm telling the story now because people often ask about it.

I am 42, graduated college in 1992, with a BA in History. In 93, I got an MA in History, and took a year off from the Ph.D. program. I went and worked in Budapest for a year, and never went back to school. That's a story for another post though. Bottom line, I went back home to the SF Bay Area, and got the first job that came along, doing data entry at a non-profit. Back then, it was called the Rape Crisis Center of Contra Costa and Marin Counties, but now it's Community Violence Solutions. It was a caused I believed in, and still do, and I eventually ended up doing a term on the board of the now-defunct National Coalition Against Sexual Assault. Another story for another day.

One day, while working at RCC, one of my colleagues was exasperated, she couldn't make the computer work. I walked over and hit the 'on' switch on the monitor. You have to realize this is 1994, and most people, particularly those who worked in non-profits, didn't have much experience with computers. A couple years later one of the office staff in our Marin office refused to unplug a computer because she was afraid damaging it, necessitating me driving over there to unplug the computer from the power strip and plug it into another power strip, thereby "fixing" it.

So anyway, with that acting of turning on the monitor, I became the computer expert in the building. A few years later, someone would coin the term "accidental techie" to describe what happened to me. I'd always had a computer, fiddled around with it, but never gotten too deep into it. Until then. When you're the computer person at a non-profit in the 90s, you were the computer person. I did everything from database work to hardware troubleshooting, to networking. Eventually we hired a data entry person for me to supervise. I went on to a series of non-profit jobs, eventually back to school, and then to Google. But I will never forget that first moment of turning on the monitor and having people literally turn to me with new found respect in their eyes.

Why do we do it?

2011-09-30T20:24:00.000-07:00

Here's a question I get asked a lot: Why does Google do it? Why create things and give them away for free? Why is the Google Maps API free? What does Google get out of Google Earth?

Because it's the right thing to do. The Google Maps API was the world's first monster web mashup platform and is still the most used web API in the world. When we started Google Maps, we thought "Oh great, people can search on maps for directions, and look at satellite imagery." We though we were making a really cool app that people could visit. And we did.

But external developers showed us how it could be used to create mashup, data visualizations that really gave meaning to data. And they could do it without having a geography degree or a certification in GIS software. And they could do it with that then unique slippy map feel that we now expect from Google Maps. So we released the Maps API.

Because we could get sites like HealthMap, or the Ushahidi project. We did it because people wanted to see real estate on maps. We did it because mapping crime alerts was meaningful to people. We did it because people want to share KML files, mark up the earth, and have now created over 1 billion of them. We did it because in a crisis, people can turn to our maps and yours.

Sure, there's a business model behind it, Google makes money on advertising, and the more sites out there, the more people are search. So, yeah. And we do make money from people who want to make the maps private, or offline.

But fundamentally, we do it because it's the right thing to do, we do it because we believe that people want to, that they benefit from using maps as a platform for understanding data.

Technology: Do you get it?

2011-09-24T20:12:00.000-07:00

I give a lot of talks. I talk to developers, engineers, gis people, students, professors, geologists, NGO workers, and many more. And there's one statement that comes up quite often, a statement people make to me all the time. Usually not engineers or developers, but just about anyone else. It is some variation of:

Oh, I'm just not good with technology.

This statement puzzled me for a long time, even though I think I understand what they are trying to say. The truth is, even the remote stone age tribe that hasn't made contact with the rest of the world uses technology. The light switch you flipped on? Technology. The paper you write on? Technology. The radio, car, dish washer, technology. The fork you eat with. Even the fact that you wear clothes. Technology. Anyone who has played Civilization or some other strategy game with a tech tree knows this.

What they are trying to say is that they are not good with computers. But chances are, they email regularly, post on [insert favorite sns], or use some IM client.

So what do they mean? I'm going guess they mean something like this:

I am intimidated by how much you know about geo/programming/mobile phones etc. And I feel I could never learn that much.

This is a much more reasonable proposition. Still likely untrue, but much more reasonable. I generally find that if I work with them, they can learn a surprising amount in a short period of time. My personal theory is that it is fear of failing more than anything that prevents people from actually being able to do something. Once they get past that, they tend to see how easy something can be, at least at the start. I know there are things I've been scared to try for fear of failing or looking bad. That's natural. I think obsession with certain terms, like "technology" in our cultural make it harder for people to start, create a class of things they don't understand.

I am not saying that everyone can be good at anything, or that some people aren't better at things than others. But one of our jobs as technologists and developers is to make it easier for people to understand what they are already doing, and to learn new things by relating them to things they've already learned. I've taught rudimentary JavaScript programming to people who didn't know what a text editor was, and some of them have gone on to make really amazing maps visualizations, by showing them the basics and relating it to what they already know.

OK related note, some of you have heard this before, but a short story:

A friend of mine, a UNIX systems administrator, told me that he didn't think he was a geek. Oh, he's also a SciFi fan. And I mean he organizes SciFi conventions in his spare time fan. I should say Fan with a capital F. So he says to me, he doesn't think he's a geek.

After a moment of stunned silence on my part, I said "OK. So, let me ask you a question. What's your favorite text editor?"

"Well..." he began and I said "You're a geek." He looked at me and I explained "If your response was anything beyond 'Huh?' or 'You mean like Word?' then you're a geek." He then went on to explain to me why it used to be EMACS because of the capabilities, but was now vi because it existed on any machine he cared about without installing anything extra. My point was made.

Highlights from FOSS4G

2011-09-17T13:31:00.000-07:00

I went to FOSS4G this week, and had a great time. Believe it or not, this was the first time I went to a FOSS4G, and now of course I am hooked.

First let me say that everyone I talked to and every talk I went to really was great. Such a friendly and smart crowd of people. And everyone was working on really interesting problems. I missed several talks because I just got into a conversation with someone and we didn't want to stop!

There were a few things I wanted to call out in particular that I found really compelling.

CartoSet and CartoDB: Awesome early stage project that draws inspiration from Fusion Tables. Nice functionality, great designers working on it. The folks at Vizzuality are doing great work.

WebGL Earth: A nice 3D globe written entirely in JavaScript. Especially compelling in the developer release of Chrome. From Petr Klokan, of GDAL2Tiles and MapTiler fame.

PostGIS Raster (pg Raster): This year a lot of buzz was around PostGIS Raster, which would as much as possible treat queries on raster data as SQL queries. It is still pretty experimental, but shows a lot of promise, especially attached to the mature PostGIS project. But the other thing that shows is there is a lot of desire for treating rasters as data, not just overlays. Of course, we believe that at Google, which is why we're working on Google Earth Engine. But I'm also glad to see so much desire for it in the FOSS community.

Here's some photos I took at the conference

FOSS4G2011

Playing with the HTML5 Drag and Drop API

2011-08-25T17:03:00.000-07:00

So another thing that inspired me during Sean Maday's talk at Google's Geospatial Awareness Day was his demonstration of using HTML5's drag and drop API to load a KML file into a Google Earth API site. In that sample, he loaded the KML file. The JS then parsed the KML and read it into the plugin. I immediately thought about using it with Maps, and adding in a GroundOverlay. So I wrote a quick sample to do just that. Of course, it doesn't work on older browsers, but if you're using later versions of Chrome, FF, or Safari it works just fine. I haven't tested it on IE9.

The code owes a lot to Riyad Kalla's HTML5 Drag and Drop Upload and File API Tutorial. He does a good job of explaining the API, so I'll leave that part to his post. All mine does is put any image in a GroundOverlay on a map. Drag any image from your computer into the little space at the top and it'll show up in a pre-defined place on the map. Next-up I'll work on a way to stretch and position the image. I'm also thinking I'll try to pull out any positioning in the EXIF headers to make a guess about location.

I'm thinking this would be useful for image upload and positioning sites. Say I want users to place images of old maps or more recent satellite imagery. They could position it on the map and then upload it to my server. Of course this doesn't take care of proper georeferencing or projections. Thoughts for another project.

Here's my code for creating the GroundOverlay:

function handleReaderLoadEnd(evt) {



  var imageBounds = new google.maps.LatLngBounds(

    new google.maps.LatLng(40.716216,-74.213393),

    new google.maps.LatLng(40.765641,-74.139235));

  var oldmap = new google.maps.GroundOverlay(evt.target.result,

    imageBounds);

oldmap.setMap(map);

}

HTML5 Input Range for a slider

2011-08-24T20:40:00.000-07:00

I was at Google's Geospatial Awareness Day today, and Sean Maday did a demo of the new HTML5 input type of range, and I had a palm on forehead moment. Regular readers of the blog know I'm slightly obsessed with sliders, so I came home tonight and did a quick sample using a demo I did in Mexico City at the Esto es Google event. The example uses a FusionTablesLayer to show polygons from Natural Earth Data showing the boundaries of the Mexican states, and colors them using the populations, as found on Wikipedia.

Here's the sample, and here's the code for this sample.

function initialize() {

 

    var center = new google.maps.LatLng(23.99170847335287, -102.6702973539042);

 

    map = new google.maps.Map(document.getElementById('map_canvas'), {

      center: center,

      zoom: 5,

      mapTypeId: 'roadmap'

    });

 

    layer = new google.maps.FusionTablesLayer({

      query: {

        select: 'Shape',

        from: '1231298'

      }

    });

    layer.setMap(map);

  }

    function showValue(newValue)

{

  document.getElementById("range").innerHTML=newValue;

   layer.setOptions({query:{

      select: 'Shape',

      from: '1231298',

      where:"'2010'>" + newValue

    }}

      );

}

  </script> 

</head> 

<body onload="initialize()"> 

  <div id="map_canvas" style="height:75%">

  </div> 

  <input type="range" min="600000" max="15000000" value="600000" step="100000" onchange="showValue(this.value)" />

<span id="range">600000</span>

Using gx:altitudeOffset to raise polygons during a KML Tour

2011-08-12T12:12:00.000-07:00

During the keynote Tuesday at Esto es Google, I showed a demo in Google Earth demo. In the demo, Polygons showing the boundaries of the states of Mexico were raised to relative heights based on their population. The KML is uploaded here if you want to take a look. Basically, I got the boundaries from Natural Earth Data, and then wrote a quick Python script (it's not worth posting here) that helped me create a KML which updated, over 10.5 seconds, the altitudes of the polygons based on the population. I got the populations from Wikipedia. is a Google extension to KML that allows you to say "Hey, change the altitude of this whole polygon by X meters" without having to update each coordinate in the element. By specifying that the takes place over a period of time, you get a nice raising effect. And since they are extruded to the ground, you get a 3D bar chart effect. I also move the camera around so you can see different perspectives and polygons that might otherwise be hidden. I have 3% battery life, so I won't post a screenshot, but you can see if you open up the KML file and play the tour.

Thoughts on Social Media: Early Days with Google+

2011-07-22T14:44:00.001-07:00

OK, because everyone else is doing it, I thought I'd write a post on my experiences using Google+.

First, I'd like to say for the record, I do work for Google, but not on Google+. I work on Google Geo, and specifically helping external developers use Google Geo APIs. I don't know why, but a variety of people have said to me "Now that you're working on Google+..." or something like that. I think I might have shared a post by someone who does and that's why people think that. Or something. And as always, these are my thoughts, not necessarily those of my employer.

Anyway, here's some thoughts:

I Heart Circles
I do feel like they solve an issue that I've had with social media in the last several years. I have a job where I represent a company, Google, and therefore don't want to mix-up my public sharing of photos and link with my private sharing with friends and family. I haven't really had that since getting on Facebook and Twitter and having thousands of people following me. Which basically meant I didn't have a private life online. Many people my age wonder why that's an issue, but I have nieces and nephews who really communicate mostly online, and it is a great way to keep up when you're away. And I travel a lot. I mean 30-50% of the year I think. So I'm often in different time zones, etc.

What I am still trying to figure out
I represent a company. So in some ways, the more people "follow" me, or rather in this case have me in their circles, the better for me. The means sharing a lot of things publicly. But that means everyone I share with sees all of that. And everyone who has put me in their circle. But if I limit what I post to certain circles I've defined, it means people don't see it when looking at whether they want to follow me. So that's interesting. Certainly hashtags and following someone by only certain hashtags would solve that, and I know the team has said they're looking into it.

I Heart Hangouts
Great experience, a really nice feature. I want to start doing some evening "let's have a beer" hangouts with folks. Or perhaps use it while playing video games or something. Anyway, still many use cases that haven't been figured out yet.

Lots of people adding me to circles quickly
I think there's probably a lot of overlap between my Twitter/Facebook followers and my Google+ community. I am already over 1000 people with me in their circles, a number it took a lot longer to reach on Twitter.

Conclusions:
Basically, loving it so far. I think that more tools to manage your stream will help a lot.

Fictional Worlds in Street View

2011-06-04T07:26:00.000-07:00

I hope I'm not shocking anyone, but Liberty City isn't a real place. The central location in the video game Grand Theft Auto is a fictional self-contained world, a sandbox in gaming terms where there are missions but you essentially have free reign of the city, not confined to a narrow path like in some other kinds of RPG video games.

So why am I talking about video games, aside from the fact that video games are AWESOME?

Recently, I discovered a map on a fan site for Grand Theft Auto IV. It's actually not the first one, the first one I've seen was on IGN. Both these sites use Custom Projections and map tiles to define a map that shows only Liberty City with no reference to Google Map tiles.

The new site though has a significant new feature, it uses Custom Street View Panoramas to display the Street View of Liberty City. Go ahead, try it, drop pegman onto the city and check out the panoramas. I'll wait.

...

Pretty awesome, huh? There's an interesting hack going on too. When you add a custom panorama it doesn't become part of the blue overlay that happens when you grab pegman but before you drop it. Since the designer had access to all the road data, since they designed the map, they were able to create a custom pegman that created their own custom blue overlay that allows you to see where Street View is.

I'm hoping we'll see more of these kind of fictional places in Street View Maps API implementations. The code for it is reasonably simple, creating the actual panoramas is more difficult. I hope this sort of thing inspires people to use the Maps API to show planning projects too, showing interiors of buildings yet to be built, etc.

Hacking Google Earth to show Earth Interiors

2011-06-03T16:22:00.000-07:00

At the Smithsonian today I showed off Declan de Paor's classic interior of Mt. Fuji which uses time animation and SketchUp models to show half Fuji sliding out to reveal the interior cone. I mentioned this to Declan and he pointed me to his recent use of Mars in Google Earth to represent the core of the Earth, and building on top of that. Pretty cool. Here's his more complete write-up.

Working with People

2011-06-02T18:21:00.000-07:00

The Humanities are traditionally a lonely profession. While in the hard sciences it's not uncommon to see a long list of names on papers, in Humanities professions there's little reward for multiple people working on a project. Tenure was based on articles you wrote, sole project work. One of the reasons I love digital humanities work is that people are coming together, breaking the restrictive bonds of solitary work. It was the reason I went back to grad school in 2004, tired of being the lone computer guy at a non-profit doing interesting data management work for people who largely didn't understand what I did.

Today I was lucky enough to meet with folks at the Scholar's Lab and the SHANTI at the University of Virginia in Charlottesville. It was inspiring to see the work they're doing by coming together, and building great projects. As David Germano explained it, the early days of DH work were dominated by projects built around individuals. Which mean highly specialized projects that had to be continuously maintained by individuals who also had to fundraise for them. These projects were difficult to bring to other applications. Now, they're trying to build infrastructure projects, or better yet build on top of infrastructure projects to build the capacity for all DH professionals to do powerful projects. And of course, since a huge number of their projects have a geospatial component, they were fans of Google Maps and Earth. But Fusion Tables was pretty new, so I hopefully helped them out there.

Tomorrow, I am meeting with people at the Smithsonian and then hurrying back to Fairfax for THATCamp where I'm leading a self-guided a workshop on Google Geo. Come and join us. And work together to develop projects.

Memories and Ghosts-Berlin then and Now

2011-05-30T01:45:00.001-07:00

My first stint in grad school I had a roommate, an air force officer who was getting her masters. One day she showed me a medal she got, as part of the occupation force in Berlin. She was one of the last to get it, having been stationed there in 1990 just before reunification.

I'm in Berlin, my last few hours here actually. People have been asking me a lot "is this your first time in Berlin?" and I say no, and they assume I know my way around. I say "Actually, it's my third, but the last time was 21 years ago." I'm 42, that makes it half my lifetime.

I visited Berlin twice before, in 1989, just after the Berlin Wall opened up. I don't like to say "fell" because the physical wall was in place for weeks afterwards, and in fact they've preserved at least one portion of it for posterity, but in an out of the way place. The second time was the following summer, I came here with my mother.

Walking around Berlin, I feel ghosts. Not actual ghosts of course, but things I catch out of the corner of my mind's eye. Standing in front of the Brandenburg Gate, I remember standing there, looking up at the Wall, looking up at the East German guards still patrolling the Wall, a little confused I'm guessing at why they were still there since other portions had been removed, and the checkpoints open. I remember the French activist - activist for what I'm not sure, just he had a sign and long hair - yelling up at the guards to tear down this portion of the wall, to open the gate. He was very impassioned, but the rest of Berlin was having a party.

I was a hungry student, traveling with three other hungry students. It was cold, and we'd been used to living the East (Budapest) for awhile. Several major companies decided that the opening of the Wall was a great marketing opportunity. Marlboro had people hand cigarettes to East Germans as they drove their Traubants through the checkpoint. Western cigarettes had been greatly prized, and often used for bartering in Eastern Europe. Candy companies got into the act too. We four followed someone from Snickers around, as she handed out candy in front of Brandenburg Gate. We positioned ourselves in front of her, and acted surprised and pleased every time she handed us some. I felt vaguely sick that night from all the sugar, but I had saved a lot of money on food for the day.

There's no trace of the Wall left around the Gate, except for some signs for tourists explaining a little of the history. Walking through the Gate was weird. Greenpeace activists were staging a small anti-nuke demonstration last night. They'd put up big banners on the Gate. And I realized with a start that I had walked from former East Berlin into former West Berlin. Without really realizing I had been in the East. I saw in my mind my mother breaking off pieces of the Wall. Or perhaps I am thinking of the actual picture I have of her doing just that. Funny that my memory might actually be of a picture of the event not the event itself.

The Berlin GTUG meeting yesterday was at c-base. In my talk I discussed Fusion Tables and the Places API. The group is so vibrant and interested. Before hand we discussed different scripting languages that run on the JVM, someone was really excited about Scala. c-base itself is modeled on a crashed space station, if the space station had been designed for a 1970s movie. Very nice space with a great view of the Spree. I drank Czech beer and ate barbecue and looked at the Spree, and forgot about the memories and ghosts of Berlin past. The present in Berlin is so much more beautiful.

More Ruminations on Place

2011-05-19T06:48:00.000-07:00

Back in March, I wrote a post, Ruminations on Place, after a day walking around in Kyoto. I've been in Paris for a few days and I'm thinking more about Place, and what is a Parisian sense of place. Or rather, I should say, what is my sense of Paris as a Place. These are just some random thoughts.

I have passed approximately 50 - or it may be as low as 10 - souvenir shops selling postcards in distinctive black and white or sepia styles. Do an image search on "Paris France" - "Paris" gets you a lot of Paris Hilton stuff - and you'll see examples of it. It's a style that I realized unconsciously made up my sense of Paris as a Place, and made me long for a black and white camera.

Americans, I think, often have the stereotype that Parisians are rude, particularly when you try to speak French. Apparently, this is no longer true. Oh no, I didn't try to speak French, my partner is here with me and has a great accent, though not a lot of vocabulary, and it's be wonderfully fun. And everywhere we go, Parisians have been definitely not rude. I realize this too was part of my sense of the Place that is Paris, and strangely I feel a conflict between my internal sense of the Place and my current experience. At the same time I am of course happy to have that proven wrong.

Like many of my generation, I am used to relying on guide books for travel. My partner had one, which I will no name here, which had much inaccurate information. I am proud to say that the Google Places app on my Android device was really helpful in situations like "Oh, I need a place to eat now that I have time before the concert I just decided to go to." It was not so helpful in finding a breakfast place because breakfast isn't a big deal here like it is in the US. So we found a restaurant nearby, but not one that served crepes, which we wanted. Because apparently Parisians don't do that. But nothing in Places, or an online search, or a guide book would have told me that. So, still work to do.

Speaking of Places, and to earn my keep, the Places API is now open to everyone and has some really cool capabilities, check it out.

Memories: When the Soviets tried to recruit me to be a spy

2011-04-29T14:15:00.000-07:00

Dan Morrill on the Android team reminded me today of a story I told him awhile back that is worth sharing.

In 1989 I was on my junior year abroad, in Budapest Hungary. Nice random chance. I had gone into the education abroad program at UCSD and told them "I need to get out of the country!" and they obliged. When I'd applied, it was 1988 and there was maybe some interesting stuff going on in Poland with Solidarity, but no one knew what was coming. I arrived in August of 1989, and a week later the Hungarian government opened the borders, allowing East Germans to leave to Austria. At the time, this was hugely momentous. East Germany and Hungary were part of the Warsaw Pact, an alliance of Soviet backed Socialist states run by the Communist parties and mostly dictated to by Moscow. Travel between those countries was permitted but travel to the West was severely restricted. East Germany (officially known at the time as the Germany Democratic Republic, or it's German abbreviation "DDR") cut off travel to Hungary, but the chain of events that had started (arguably) with the Solidarity strikes in Poland started to cascade. The end result was a very different Europe. But no one knew that at the time.

In fact, I didn't find out about the Hungarian government opening the border for a week. There were few English language news sources at the time (remember, no public Internet) and the faculty of Karl Marx University of Economics, where my program was situated, didn't bother to tell us, perhaps a little scared about what was going on.

Anyway, I was studying to be a historian at the time and historic things were happening so I was in heaven. My mother was a peace activist, and decided to go to Moscow to do work there. She was also an artist and had no real concept of dealing with bureaucratic institutions. She wanted me to visit her for Christmas in Moscow, and since I was in Budapest I thought why not? But to go, I needed an invitation to get a visa. Not an easy process. BTW, still not an easy process. So I asked her to get me an invitation.

My mother responded by saying "I invite you". I told her I needed something written, and she sent me a telegram that went something like this "Here's your invitation. I don't understand why you need this. Seems like you just don't want to spend Christmas with your mother." I may be mis-remembering that last sentence.

Sighing in frustration, and realizing I wasn't going to get anything better out of her, I showed up at the Soviet consulate. Now most embassies have nice neat orderly lines that move people through the place. The Soviet consulate in Budapest, at the time a part of the Soviet embassy, a huge concrete monstrosity, ran things differently. They let people crowd around and push their way to the front of the line. Being a shy retiring type at the time (unlike now) it took me an hour to get to a window. I tried to communicate to the official who just stared at me, and then turned and summoned someone who could speak to me in English. He frowned at me, looked at my passport, this young idealistic many trying to get a visa, glanced at the telegram dismissively, and said only "You come back tomorrow, 12:00".

Dutifully and full of dread, I came back at 12:00 the next day. "Sasha" grilled me for two hours at his desk. What did my father do (episcopal minister, deceased) ? What did my grandfather do (stockbroker)? Why was my mother in Moscow (peace work with the Peace Committee, a known KGB front organization)? What was I studying (History, very interested in Eastern Europe)? Where did I grow up (Berkeley, at the time home of radicalism and Marxism in the US)? Would I like to learn more about the Soviet Union (oh yes of course? and the kicker Would I like to study Russian with his daughter, trade language lessons (Oh yes, of course, sounds great)?

At the time I wasn't a Quaker, so a little lying and stretching the truth wasn't beyond me. He seemed satisfied, and pulled out a ticket, he actually paid for my train ticket to Moscow. How exciting! At the time your ticket came with a separate seating assignment. He gave me the top bunk in a sleeping car. Wow, how generous, thank you! Afterwards I learned that the ticket was about $7 or something, depending on whether you exchanged money officially or on the black market.

So the day finally came, and I get on the train. Top bunk, all the way to Moscow, more than a day of travel. In my mind I remember it as 36 hours, but I'm not sure now. While I was boarding the train, a revolt broke out in Romania. By the time I got off the train Ceauşescu, the communist dictator was arrested. He was executed a couple days later. I'm against the death penalty in general, but I certainly didn't mourn his death.

But I didn't find out about the arrest because I was on the train. No phones or Internet in those days, and no one announcing what was happening. And I was the only American in my car, as far as I could tell. My grand mother had sent me a big box of candy, and I learned the value of american candy, so I shared them with those in my cabin. There were six people in my cabin and lo and behold in the bunk across from me there was this beautiful Russian woman who spoke English with just a slight and endearing accent. She engaged me in constant delightful conversation. Turns out her father worked at the embassy, what a coincidence! Now you have to understand that this sort of thing never happens to me. So a clever man of the times, with a slight paranoid bent, I easily put together what was happening. When she gave me her number in Moscow as we got off the train, I wasn't surprised. I never called of course. My mother had her own KGB handler, and I'm sure everything was bugged in my mother's apartment. That's not paranoia, that's just how it was back then.

"Sasha", btw a very common Russian name, short of Aleksandr, wasn't at the embassy when I went back. He had asked me to "report" on what I saw in Moscow. I thought I'd be dutiful about that, thankful for the ticket, and then just not follow-up. But he wasn't there and the embassy staff acted as if they had never heard of anyone named Sasha and it was strange that I would ask and would I please go away. Sorry Sasha, you failed to recruit a spy, but thanks for the ticket!

NoGIS: Round Two of Neo vs. Paleo Geography?

2011-03-31T18:36:00.000-07:00

Can't we all just get along?

I started, apparently, a few days ago with the posting of NoGIS Meetup, and the subsequent blog post, "What Does NoGIS Mean?" by Sean Gorman on the GeoIQ blog. Among other things, Sean said

"For decades, location and geography have been their own special niche, served by GIS technology from a fairly small number of vendors. As many have pointed out “spatial is no longer special” and as a result location is quickly becoming a feature of many technologies. As location base apps become ubiquitous the characteristics of geographic data are changing as well. The data of this new paradigm does not look like the static parcel data, which is stereotypical of much traditional GIS work. As we saw in the NoSQL characteristics data is now high volume, dynamic and users/developers want to see/query it in real time."

And the usual statement by neogeographers (in one of the comments)

"I’m not inferring that GIS and NoGIS are mutually exclusive..."

James Fee, traditional defender of the Paleo...OK, that's a bit much but let's say a frequent defender of all things Paleo responded in his blog post:

As with anything, everyone is quick to say we’ve all been doing this since the 1960′s so ignore it and move on unless you’ve got one of the following to accomplish:

And then goes on to list the geography industry's equivalent of hipster evils.

And I thought that the Neogeography vs. Paleogeography wars were largely over and we were all getting along. But perhaps it was just a Christmas ceasefire.

That said, I'll be at the NoGIS workshop, should be interesting. No one ever said geographers were boring. Oh, wait, many people have. But they are, of course, wrong.

My Slides for CalGIS: What you should be asking us The future of geography, and the hard - and easy - questions that follow

2011-03-29T20:58:00.000-07:00

I do a lot of talking in my job. This quarter, January 1-March 31st, by the end of the quarter I'll have given 19 different talks at 16 different events. Many of the talks are the same or similar, targeted to a specific audience but conveying some of the same essentials in terms of content. That's because I'm doing the "spreading the word" part of my job, telling people about new technologies.

Sometimes, though, I like to do new things. I was invited to do a keynote at CalGIS's conference, and decided to write an abstract that was different from what I've been doing, so that I could explore different themes. I talked for the first time in a public talk about Google's on-going work on the Japan Crisis, and how cloud providers can help maintain a web presence for public service agencies - most of the participants were from government agencies - and related topics. I think the slides convey more of a "of course you should go with a cloud provider" then I actually presented, but most of what I was trying to convey was that these are hard choices and here's a place to start.

BTW, I hate the word cloud in this context. It's so market-y. Anyone have recommendations for a real replacement?

Recent Slides

2011-03-27T05:19:00.000-07:00

By the end of March, I'll have done 19 talks this year, at 16 different events. It's been a busy quarter. I'm not going to put all 19 sets of slides in here. Some of them had a lot of similarity. So I thought I'd post a couple of representative ones.

I did a talk on March 1st at the North Capital Area GTUG, at the Google DC office, on Fusion Tables. This was the culmination of a US East Coast tour that took me to College Park Maryland, Boston, and DC. It was video taped, but unfortunately the tape hasn't been digitized yet, I'll post that when I get it.

In Tokyo, at that Yahoo! Japan Geolocation conference, I gave a talk on Fusion Tables and mapping in the cloud.

I then went on to give talks at the Tokyo and Kyoto GTUGs. Here's the slides from Kyoto, largely the same as the Tokyo ones.

Tragedy in Japan and how to help

2011-03-16T16:36:00.000-07:00

Like everyone, I am horrified by the loss of life and homes in the recent devastation in Japan. The scale is enormous, and as we hear more about it, many of us feel hopeless to help and scared about what it means for the future. Aftershocks and worries about radiation are all over the news, along with calls for donations, and occasional worries about scams, people promising to help, but taking money anyway.

At the same time, I am consistently impressed by the people who jumped in to help. From the first responders, the military, to the nuclear workers working day and night to prevent greater tragedy. There are people on the ground in Japan doing great work, mapping transportation routes, helping people find loved ones, providing food and shelter.
The Google Crisis Response team has been putting together many of these resources here: http://www.google.com/crisisresponse/japanquake2011.html, including ways for people to donate money, maps of the situation, and links to the Japan Person Finder app.

The Humanitarian OSM Team is also on the ground, with more information here: http://openstreetmap.jp/crisis/ about how to help with their mapping efforts.

Millions have already been donated. Millions more will be needed. Please help out now. If you want to know more about what Google is doing, follow @earthoutreach on twitter.

If you have other creative ways to contribute, please feel free to leave them in the comments.

Ruminations on Place

2011-03-10T05:05:00.000-08:00

Sunday, I was on a panel with Steve Chase, Hurricane Chase, and Murata Takehiko at the Yahoo! Japan Geolocation Conference (#geoconf). Yahoo! Japan (a distinct entity from Yahoo! Inc. btw), put on an excellent conference, and had announced that they would be contributing data to Open Street Map, in a similar fashion to Bing and MapQuest. There was a lot of excitement in the room, with many OSM participants present (by design I imagine). And there were inevitable questions about Google's future involvement in the project - something I'm interested in personally but can make no commitment to on behalf of Google of course. But more importantly, to this blog post at least, a question came up about whether Google would contribute Street View imagery to OSM, and whether there were any OSM related projects developing open versions of Street View. As a panelist, and someone who can't avoid talking, because I'm just that way, I had to answer of course. My answer went something like this, speaking as myself not foreshadowing a future Google product:

I don't think in the future, Google should have to be driving all the streets in the world. Ideally, we will move toward a time in which everyone is providing data about the world in open formats, in ways that are search-able and crawl-able. What makes Street View so compelling, I think, is that it gives us a sense of place, a sense of the essence, be being-ness of the location we're looking at. And in fact, in places where we don't drive cars, you can still see geolocated photos. Look at Moscow, for instance, with Pegman selected. There are photos all over. Ideally, this is something self-perpetuating, something people want to open up.

That got me to thinking, especially after reading Ed Parson's recent post about a 1930's video of Teddition. He, and the commenters, talks about what can be seen there, and what has stayed the same. Ed ends with this statement:

"We are lucky to have rare video like this for it’s completeness but at the same time it’s disappointing that for future generations Google Street View which could offer a similar resouce has had to be mutilated to accomodate privacy concerns."

Indeed, there is something we miss out on. I am not arguing for doing away with privacy, I'm just ruminating here on pictures and meaning. The word "place" has many meanings in English, but they are interrelated. The reason satellite imagery, and particularly street view imagery is so compelling is that it relates something that standard maps can't, a sense of the place or essence of a location. And that place-ness, that sense that is tied to location, time, people, trees, buildings, smell, sounds, all of that is incredibly powerful, and I think Street View is only starting to touch on what that means to people.

I've been in Japan this last week, and at two GTUG meetings, I did demonstrations of custom Street View. In particular, people were interested in this simple holiday greeting from Digitas. In it, the developers added some panoramas to their Street View application through the API, which of course anyone could do. The panoramas lead off street into the digitas office where employees hold up signs and there are thought and word bubbles around them. This conveys a sense of the place, a sense of celebration and fun and associates it with Digitas. I hope it really is that fun to work there, they seem like really nice folks in the picture!

The point I'm trying to make is that by this simple set of panoramas and a few lines of code, they were able to extend the sense of place that is the city around them, and change it to create a view of the place that is their company.

I know there are lots of academic writings on this subject, but frankly since 1994 I get bored when post-modernism or marxist dialectics get mentioned, so my forays into those studies have been unsuccessful at moving me. I read the works of China Meivelle, or Armistead Maupin instead, each of whom conveys vividly this sense of place

Today, like most days I have free in cities not my own, I walked. I walked around Kyoto. Not to find temples or castles, but to find that sense of place. I walked around, a lot, often going back over the same streets, visiting occasionally the same shops. I tried my extremely limited Japanese, and people graciously helped me out. I walked not just the big streets, but the small ones, gathering into myself my own impressions, my own sense of Kyoto-ness. Of course, this is very different from living here, what the place is like, and especially different from being a native. I tried to take pictures (posting them later) that captured this sense. How will technology solve this? Street View, satellite imagery, they are a start, but I believe augmented reality applications are going to be the best bet for capturing the "-ness" of a place. And I believe at some point, most of us will be involved in that capturing. The popularity of check-in applications, the rise of geotagged photos and videos, providing reviews, Foodspotting, and more are already starting it. It's only been 6 years since Google Earth and the Maps API came out, I wonder what we're going to see in another 6 years.

Private Tables in Fusion Tables: Better privacy for your data

2011-03-04T00:21:00.000-08:00

Back in June, I wrote about "Techniques for protecting your data." Yesterday, Google announced another way to protect your data, Protected Map Layers.

Protected map layers, a part of the Google Maps API Premier package, allow you to create a data table in Google Fusion Tables and keep that table private. Maps API Free allows you to use only tables marked Public. Protected map layers are rendered as clickable rasters. You still have to be careful what data you put into the infowindow, make sure it's only what you want public.

Of course, Maps API Premier also allows you to make maps that are password protected, so this will be really useful for enterprises that want to keep their data entirely private but still take advantage of the performance improvements and spatial queries that are allowed by using a FusionTableLayer.

JSON in KML Templates

2011-02-09T15:33:00.000-08:00

Since Google Earth 5.0, the description balloon has handled pretty close to full JavaScript. There's some interesting things you can do with this. In this example, I'm showing a very basic use of JSON using KML balloon templates.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
  <Document>
    <Style id="testStyle">
      <BalloonStyle>
        <text><![CDATA[
          <script type="text/javascript">
             function loadData(){
               var foo = $[foo];
               var myObj = foo.myData[1].value;
               document.getElementById("example").innerHTML = myObj;
             }
           </script>
           <body onload="loadData()">
             <div id="example">test</div>
           </body>
          ]]>
        </text>
      </BalloonStyle>
    </Style>
    <Placemark>
      <styleUrl>#testStyle</styleUrl>
      <ExtendedData>
        <Data name="foo"><value>{"myData":[
          {"value":"1"},{"value":"50"}]
          }</value></Data>
        <Data name = "bar"><value>23094</value></Data>
      </ExtendedData>
      <Point>
        <coordinates>0,0</coordinates>
      </Point>
    </Placemark>
  </Document>
</kml>

Balloon templates are designed to allow you to use a single <Style> element for all or a set of your <Placemark;gt; elements, and then just insert <Feature> specific data into the template. Named <Data> elements inside a <ExtendedData> element provide name/value pairs. However, this can be a little flat. You can put JSON elements inside the <value> element (a child of <Data>, which gives you a lot more flexibility.

The good folks at Secorra provide ObsKML, an ocean observations format, using this technique.

Interactivity with FusionTableLayer and mouse clicks

2011-01-29T13:24:00.000-08:00

The FusionTablesLayer in the Google Maps API is a bit of a black box. Fusion Tables generates a clickable tile overlay to use place in your Maps API application, but you don't get direct to change anything. If you want to change data, you have to do server side calls. However, the FusionTablesLayer does give you access to row data on click, which allows you to do interesting things server side.

In my sample interactiveftlayer, I use a Fusion Table of Google's corporate addresses. The table has a simple Color column, with values of 0 or 1. In the Fusion Tables map visualization for the table, I configured the Marker Icon such that values of 0 are green, and 1 are blue. The color means nothing of course about the actual office.

In interactiveftlayer, I suppress the InfoWindow by adding a suppressInfoWindows: true option to the layer initialization. I then add an event listener to the layer that captures the click event on the layer. This gives me row information about the feature clicked. I then send a XMLHttpRequest to the server (in this case App Engine) with the other color (1 if the color is 0, 0 if it is 1). After the response from the XMLHttpRequest, I reload the layer.

There's one tricky part, and hopefully we can find a way to improve this on the Maps API end. The FusionTableLayer is cached by the browser. Which means that even if the data has changed, the layer stays the same, at least at zoom levels visited while it was visible. This includes, BTW, not only the images but the row data associated with them. In order to defeat that caching, I append a Where parameter to the layer selecting for a Id greater than a random number from -1000 to 0. Since I know all Ids are greater than 0, I can do this. I'm not proud of that strategy, but it works.

Two Thumbed Closure Slider for Time Slider

2011-01-25T11:59:00.000-08:00

I've posted a couple of times about sliders - Playing with Closure UI Library: Slider and a basic Basic Time Slider in Closure. My final slider example uses the same data set as the last, and incorporates a Two Thumb slider, meaning you can use the slider to set a range of values, not just a less-than or greater-than value. I also corrected something that impacted performance in the last sample, that is I checked whether the query changed by the different events, and change the query on the map layer when the query changes from the last. This is an artifact of the events that I'm listening form, MOUSEUP, MOUSEOUT, and KEYUP. You may remember in the first sample, I realized that firing off a query change whenever the slider had a CHANGE caused too many queries to hit the overlay server, causing the Map to show the missing overlay error. Listening for mouse and key events caused fewer events to fire, but still more than one per change. The change simply tests to see if the new query is different from the old query, and only re-query the layer server if the query changes.

It's 2011: Time to think about the future of Geo

2011-01-09T20:30:00.000-08:00

It's fairly traditional to start a year with either a retrospective of the previous year, or a look forward. Or, of course, both. In that tradition, I'd like to start the year thinking about some trends that I think are going to be important for Geo this year.

Powerful Easy Analysis Tools

This is the year. In 2005, the Google Maps API and Google Earth broadened the use of geography tools far beyond the traditional GIS crowd, sparking a debate between so-called neo- and paleo-geographers that lasted for years. In the last year or so that debate seems to have calmed down a bit, as traditional GIS tools adapted to the web, and neo-geographers, were everywhere. So there's room for a fresh controversy.

The Google Maps API, followed shortly by a host of other APIs from Yahoo!, Microsoft, OpenLayers and others, allowed developers to easily place maps on their site. But as I pointed out in my Ignite Spatial talk in September, developers and GIS professionals aren't the only ones who want to share spatial data. In fact, I'm guessing the vast majority of spatial data, by volume if not quality, is in tabular form. Geocommons has recognized this for years, providing easy tools for uploading, combining, and sharing spatial data. With the addition of Google Fusion Tables, and easy mapping of spreadsheets and sharing of data, powerful tools for data analysis are in the hands of anyone with a Google account. I won't call them "low-end" tools, though certainly they lack the power of ESRI's tools, or any of a host of other proprietary and open source GIS applications. I predict that this year will see a lot of people migrating to Fusion Tables, and others using the API to back-end store the data. Which leads us to

Cloud

OK, I almost had to slap myself for saying "Cloud." After all, of all the buzzwords going around, I think it is the least penetrable to people not "in-the-know" and perhaps has the most number of definitions. To make matters worse, Microsoft has diluted the term even more with this wacked commercial campaign. However, it is being used a lot, so let me be clear, I think this is the year of Cloud Data Hosting.

I predict more and more spatial data will go into "The Cloud." We're already seeing that happening with services like SimpleGeo, Microsoft Azure's support for spatial data, and many other services. Fusion Tables of course has an API which I anticipate will be useful for a number of spatial data storage services.

Cloud Analysis

It's still early days on this. I think that 2011 will be the year of early adoption. In particular, tools like Google Earth Engine will allow you to run high-end analysis in cloud data centers. Some of these tools are already out there, but the introduction of Earth Engine allows you to do things we're used to in the spatial world, namely raster data analysis, but do it faster and cheaper than before.

Location

Well, really, who am I kidding? 2010 was the year of local and location. Facebook's Places and Places API were a very powerful entrance into the location and local scene. Google Places with Hotpot is a big entry into the local market, but it came pretty late in the year. 2010 was about local, 2011 local will become mainstream, such that everyone will have forgotten that it wasn't part of our sites. Remember when there weren't maps everywhere? That was only 6 years ago, now it's taken for granted. Local will get that way by 2012. When of course the world ends, right? That's what the movies tell us anyway.

Prediction posts are fun, because rarely are you held to them. But really they tell you more about where you are now. The things we can't talk about, or don't know about yet, those are the real surprises. Happy New Year everyone.

Basic Time Slider in Closure

2010-12-18T09:52:00.000-08:00

Earlier this month I was playing with a the Google Closure library and using the slider. I wanted to play with it for developing a time slider too. Since I'm going to History Hack Day at the end of January in London, I thought I'd show off some samples there and this was a perfect hack for that.

Anyway, it's pretty simple. You set minimum and maximum values on the slider that correspond to the millisecond values used by JavaScript to represent time. Then use the JS Date object to convert those to readable date and time for display under the slider. You can see the sample here. I set the slider to between Wed Dec 31 2003 23:59:59 and Fri Jan 01 2010 00:00:00 since I knew all dates in the underlying table fell between them. I used another Wikileaks table, the Afghan War Diary, 2004-2010 table.

The tricky part is that Fusion Tables doesn't document the date formats that it accepts for queries. We just haven't gotten around to changing the docs, but I happen to know that we accept these formats:

MM/dd/yy
MM-dd-yy
MMM-dd-yy
yyyy.MM.dd
dd-MMM-yy
MMM/yy
MMM yy
dd/MMM/yy
yyyy

AGU 2010: Data Visualization

2010-12-13T17:32:00.000-08:00

So, this was the first year without a digital globes track. I think two things happened:

Organizers got tired of doing it every year
More important: Digital globes are now all over the American Geophysical Union's annual meeting. There's really no need to separately call it out.

So this year, I presented to Visualization Aided Data Analysis: Tools and Techniques for the Geophysical Sciences. Smaller than my other years at AGU, this was still a vital record of the different tools needed for data analysis. I was particularly impressed by Hank Childs' presentation on VisIt: A Tool for Visualizing and Analyzing Very Large Data, which really seemed to be a tool for visualizing anything in any possible way. Wow, awesome tool. I was also really impressed by Vis Research: Advances in Visualization Research by Ken Joy, U.C. Davis. He had really impressive visualizations of flow patterns.

My own discussions of Fusion Tables, Earth, and Earth Engine seemed a bit more basic, but perhaps that's the point I've been trying to make, that visualization tools can be powerful and hard to use, putting incredible versatility and detail in the hands or professionals, or easy to use and basic for everyone. And when they are easy and basic, everyone can use them effectively.
Anyway, here's my slides:

Freebase Meetup and JSONP

2010-12-09T11:00:00.000-08:00

So, I was invited to speak at a Freebase meetup by Kirrily Roberts. I thought "Cool, Freebase. Oh, I should learn that!" Especially now that Freebase is part of Google! So apologies if you're looking for my intended next sample which was closure as a time slider, that will come.

I did a couple of maps mashups to try and learn the query language. But at the same time, I wanted to learn more about Google Closure. So, here's a couple of samples that I did. Let me explain in further detail:

<link href="http://code.google.com/apis/maps/documentation/javascript/examples/standard.css" rel="stylesheet" type="text/css" />
<script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>
<script src="goog/base.js"></script>
<script>
   
goog.require('goog.net.Jsonp');
var obj;
var map;
 
function initialize() {
  var latlng = new google.maps.LatLng(0,0);
  var mapOptions = {
        zoom: 2,
        center: latlng,
        mapTypeId: google.maps.MapTypeId.ROADMAP
  };
 
 
  map = new google.maps.Map(document.getElementById("map_canvas"), mapOptions);
  getEvents(); 
} 

function getEvents() {
  var envelope = 'http://www.freebase.com/api/service/mqlread?query={%20%22query%22:%20[{%20
        %22type%22:%20%22/time/event%22,%20%22id%22:%20null,%20%22name%22:%20null,
        %20%22included_in_event%22:%20[{%20%22name%22:%20%22world%20war%20ii%22%20}],
        %20%22locations%22:%20[{%20%22geolocation%22:%20{%20%22latitude%22:%20null,
        %20%22longitude%22:%20null%20}%20}]%20}]%20}';
  getData(envelope,'');
} 

function getData(dataUrl,data) {
  var jsonp=new goog.net.Jsonp(dataUrl);
  jsonp.setRequestTimeout(100000);
  jsonp.send('',setObject);
}

function setObject(response){
  obj = response['result'];
  addMarkers();
}
function addMarkers(){
  for(var i in obj){
    for(var j in obj[i].locations){
      var latLng = new google.maps.LatLng(obj[i].locations[j].
            geolocation.latitude,obj[i].locations[j].geolocation.longitude);
      var marker = new google.maps.Marker({
          map: map,
          position: latLng,
          title: obj[i].name
      });}
      console.log(obj[i].name);
  }
}

So what I did there was use goog.net.jsonp from Closure to load the JSON from the query results and then create markers. The first sample finds events during World War II and find their geolocation. Since some events have more than one geolocation, it gives each location a marker with the same title, which will show up as a tool tip on hovering over the marker. In this case, the event contains a /location/location node which has /location/location/geolocation nodes. Someone comment if I'm not getting that quite right.

For those that don't know, I used the JSONP utility to get around the same domain policy of browsers, which prevents you from loading scripts directly from domains other than the one you are loading from. Server side scripting would have solved that issue for me easily, but I decided to set myself the additional challenge of finding that workaround. JSONP takes advantage of the caveat to the same domain policy that allows you to load scripts into <script> tags in HTML. In the case of the Closure library, it creates a temporary <script> tag, uses it to get the JSON object, and then gets rid of the script tag. One must exercise caution when using this approach, of course, that the source of your JSON is trusted to prevent attacks on visitors to your site.

My second sample took a /location/location, in this case Berkeley California (where I grew up) and found all /location/location nodes that had a containedby relationship to Berkeley California. And then mapped them. Then I added in animation to, so if you click on one place it bounces. Just for fun.

It was pretty basic. Figuring out the query language is not easy. Actually, before I spoke at the meetup, Jamie Taylor did a great talk about the schema which really cleared up some things for me. Unfortunately, we experienced a complete A/V failure in the room the meetup was in> Jaime did his talk with a white board, which I think made him happy. I jokingly referred to my talk as interpretive dance. I described my approach, why I was using JSONP, what tools were available, and then we went and had a beer.

Update

I should have credited this article on JQuery and JSONP with part of my inspiration.

Playing with Closure UI Library: Slider

2010-12-03T16:35:00.000-08:00

For Google Developer Day in Saõ Paulo, my colleague Ossama Alami created this sample, which allows you to select between three different Google Fusion Tables layers and play with queries against them. I looked at that and decided that I would use it to play with the Closure Library. Specifically, I wanted to play with the UI library. So for the GDD events in Munich, Moscow, and Prague, I decided to add a slider. The results were fun, and instructional for me. The code to add a slider was pretty simple. Here's my sample, and here's the slider code:


<script>

  var el = document.getElementById('s1');
  s = new goog.ui.Slider;
  s.decorate(el);
  s.addEventListener(goog.ui.Component.EventType.CHANGE, function() {
    document.getElementById('out1').innerHTML = s.getValue();
    });

  goog.events.listen(
            s.getContentElement(),
            [goog.events.EventType.MOUSEOUT,
             goog.events.EventType.KEYUP,
             goog.events.EventType.MOUSEUP],
            function() { 
    var preset = document.getElementById("preset").selectedIndex;
    var query = presets[preset].sampleQuery + s.getValue();
    document.getElementById('query').value = query;
    layer.setTableId(parseInt(document.getElementById('preset').value));
    layer.setQuery(query);
     });
  </script>

I just took that from the documentation. Of course, there's a an HTML element for the slider, and some CSS to style it, and loading the library itself. Well, you can view source. The instructional bit was that of course as soon as you move the slider, events start firing. If you listen for a CHANGE event, as you slide the slider, it'll fire too fast. Each tick, it'll try to grab a new layer from Fusion Tables. That'll consume lots of bandwidth as FT tries to return a layer with each tick. So instead, I listen for MOUSEOUT, KEYUP, and MOUSEUP events, which fire when the user is done moving the slider, either moving off it or releasing the mouse.

Next up, I'm going to try to use it to simulate a timeslider.

Google Developer Days in Europe

2010-11-21T08:01:00.000-08:00

I just got back from Google Developer Days in Munich, Moscow and Prague. OK, I got back on Thursday, but I'm just coming out of my jetlagged state. I presented similar talks in all three places, though in Munich I talked about Latitude, and in Prague Jarda Bengl joined me and talked about Street View. My demos are all linked in the slides. The theme was New Features in Google Geo. Here's the Munich slides:

Here's the Prague slides, with Street View at the end:

In Moscow, I went a little slower, cut out the Latitude and Street View slides because of the simultaneous translation. I did have a separate slide deck, but it adds nothing over the other ones.

Fusion Tables was the killer, though, very popular in all three cities, especially with the new spatial queries. Turns out, people love one click visualizations and easy Maps API integration. Especially fun was the demo I did using a Closure slider to dynamically change the FusionTableLayer query. I'm going to refine that some more, play with it and then do a write-up. I'm trying to take the time to learn Closure finally.

Finally, here's some pictures I took from GDD Moscow and the GTUG hackathon the next day. Unfortunately, people were pretty wiped by the next day, as there was a Android/HTML5 hackathon the day before. Some folks had traveled 16 hours on the train each way to go to GDD. Only about 20 people showed to the Geo hackathon the next day, but the energy level was great and people seemed to have a lot of fun.

MoscowGDDAndGTUG2010

Presentation at NACIS

2010-10-15T10:42:00.000-07:00

I presented today at the North American Cartographic Information Society annual meeting, on new features in Google Earth Pro, Fusion Tables, and the Google Maps API. NACIS focuses a lot more on design, on cartography, and therefore it was a very interesting conference to be at, different from the usual GIS and developer conferences I present at. I sat in a session on rethinking the bike map afterward. Unfortunately, I had to leave after that, so didn't get to participate in much of the conference.

Here's my slides. There were a lot of questions on all aspects, but particularly on imagery.

My Trip to University of North Texas

2010-10-14T19:54:00.000-07:00

I had a great two days at the University of North Texas, organized by Andrew Torget, professor of History. I met with historians, engineers, folks from the College of Information, geographers, and more. Most of the meetings were unstructured, free form meetings. Those are sometimes the most fun. Today I did a presentation that was mostly demos from the new features in Google Earth Pro 5.2 and Google Fusion Tables. There was a lot of interest in the different features, and there's a link there in a getting start tutorial I threw together.

Plus I'm really starting to like goo.gl, especially now that it's open and you can track the click.

Slides from Google Developer Day Tokyo: Mapping on your phone

2010-09-30T19:12:00.000-07:00

Different from previous mobile presentations I've done, focuses more on optimizations like Fusion Tables layers, KML layers, styled maps and a few other things. I also presented the following demos:

Styled Maps

Geolocation Chrome didn't do so well at geolocation, but the Samsung Galaxy did.

Washington Post Marathon site

Fusion Tables in V3 sample

Post Google Developer Day Fusion Tables Tech Talk

2010-09-29T06:43:00.001-07:00

Today, I presented a tech talk on Google Fusion Tables. It was really the first time I did one. The slides (below) are pretty minimal, because Fusion Tables is an amazing presentation platform. I mostly presented from a boundaries KML that I had uploaded to Fusion Tables. I got the data originally from Natural Earth, which is an AWESOME site, with tons of great data. I also used this simple page to show Maps API V3 integration as well as embedding. Fusion Tables is officially the new hotness.

My Slides from Last Night's Ignite Spatial NoCo

2010-09-15T06:22:00.000-07:00

So, Google failed me. The first link for Ignite Spatial NoCo was a previous event, and I didn't check the date. I ended up going to Fort Collins, and then back to Windsor. Unfortunately, I missed half the other great speakers, but the event was fun. This is my 5 minute talk from last night.

My Slides from TimesOpen 2.0 Event on Mobile/Geo

2010-09-03T10:33:00.000-07:00

There were about 100 people there, and it was great fun. The developers were all really engaged. When I asked how many had used the Google Maps API, almost everyone raised their hands. I almost went home at that point. And lots of interest in Fusion Tables, which I only did a demo of, didn't put in the slides, but check it out if you haven't already.

Accounting for GIS

2010-08-25T15:21:00.000-07:00

I've been thinking a lot lately about what is missing from the Geoweb. As I am 41, I naturally look back to the early days of mass computing for helpful comparisons. Thinking about the 80s and 90s, I realized just recently what I was looking for. I was looking for everyone.

Stay with me here.

Before the advent of mass computing, accountants were a specialized profession. In order to balance your books, generally you would have a bookkeeper or accountant who went through reams of paper, and did what most people thought was an arcane specialized job. Today, there are still accountants, they still do an arcane specialized job. The difference is that there is a standard type of application, the spreadsheet, that opened up a tool of their job to pretty much anyone.

So a portion of their job, playing with numbers, became open to millions of users. Spreadsheets, and the analysis of data, have become an essential tool for business. People can publish and analyze data in ways they were never able to do before, and have come up with many unique ways to use spreadsheets that accountants never would have. Arguably, Microsoft Excel is the most used database in the world, and the vast majority of users are not accountants.

That's what I want to see happen with GIS. I want to see applications where people are able to manipulate and visualize Geo data even without programming experience or GIS training. I think we've come a short ways toward that. Applications like Google Earth and Google Maps let you visualize data through a standard GUI interface. Maps APIs allow people with some programming skills to put a map on their site.

But nothing makes it easy. People come to me all the time and say "Here's a spreadsheet, how do I put all my stores on the web? Add in driving directions? OK now I want to add in other things..." Data driven apps are still harder to do. I think Fusion Tables is getting there, but needs more. It needs basic spatial queries (point in polygon for instance). And it needs the ability to upload other GIS data formats, not just KML.

But wait, I hear some GIS Pros cry, you'll take jobs from us! If people can visualize their own data, why would they need us? And besides, we do a much better job than they do.

It's true, good GIS pros will produce better analysis than the average person on their own. But I doubt that GIS Pros will lose jobs. What happened with spreadsheets was that whole new uses for data were developed. People created spreadsheets to track and chart all sorts of things. Sure, probably a few accountants lost jobs, but there are still almost 1.3 million accountants and auditors in the US, with their prospects growing. I think putting the ability to visualize geographic data into the hands of everyday users will increase the demand for specialized knowledge, increase the general awareness of geographic data opportunities, and only be good for the profession.

Thoughts on GeoWeb 2010

2010-08-07T15:33:00.000-07:00

I went to GeoWeb in Vancouver this year, and ended up presenting two workshops.

New Features in Google Earth
https://docs.google.com/present/view?id=df795pd2_204d6g5n7d2

Going Mobile with Google Geo APIs
https://docs.google.com/present/view?id=df795pd2_207hpdznv5n

GeoWeb is an interesting conference. It has roots back to at least 2002, where it was a GML conference. It's origins as a GIS conference have colored participation for a number of years. But this year it was more than that. It's clear that most GIS professionals, vendors, etc. have gotten it, the web is important.

Though attendance was sparse, like most conferences this year, the approximately 200 people in Vancouver were enthusiastic and really engaged. I had a bunch of GIS types making their first Google Maps API app in my second session.

And there's spontaneity as well, with new sessions added during the conference.

Google's had a historic commitment to GeoWeb, and I don't expect that to change. There are some key decision makers there, and a fair number of developers as well.

One thing we have to all figure out, though, as a community is how to bridge the final gap between the web and GIS. There's a lot of efforts to do it, but nothing easy. That's the fundamental thing I think the GIS community misses out about the web, it's supposed to be easy. Any truly disruptive technology starts out simple. Maybe that's why Wave didn't work out. And anything that requires an Arc* license isn't easy either.

I'm not saying Google's figured it either, though we're moving there with Fusion Tables and the next conversion features in Google Earth Pro etc. But fundamentally, we need more features and tools that are easy to use. And cheap. Without that, we're not going to make GIS pros into mashup developers.

Ruminations on the 5th Birthday of the Maps API

2010-08-05T11:58:00.000-07:00

A few weeks ago, we celebrated the 5th birthday of the Google Maps API. We celebrated again the next week. In a transpacific video conference, the Geo teams in Mountain View and Sydney ate cake and drank champagne. Speeches were made, memories recounted.

In particular, Paul Rademacher gave a brief history of Housingmaps, the first known Google Maps mashup, created initially before there was even an official API. It was thanks to the work of Paul, and several other mashup creators, that Google saw the potential to create something really special.

I often say that Google had two choices at the time. We could either sue Paul and others who reverse engineered the API and created mashups. Or, we could go with it. We went with it. I say "we" btw, as if I had anything to do with it, but it wasn't until a year later that I started at Google.

As I recounted that story at WhereCamp Socal on Sunday, I asked people what our choices were, and Tim Craig shouted out "Kill him or hire him!" Maybe that's the difference between ESRI and Google :-). (Tim is actually a great, gentle guy. As far as I know...)

It's hard to remember a time before mashups now. It's not that Google Maps was the first mashup ever, but it was the first monster mashup platform, and still the biggest. And it is having repercussions beyond people putting maps on their sites. As soon as people figured out that they could mashup maps with data, they started looking for data to do it with. And putting pressure on governments and companies to make that data available. Or simply going out and generating their own.

OK, those of you old enough to think back that far, try to remember the web before all these mashups. Still a cool thing, but not nearly as exciting. I'm old enough to remember a time before the web, of course, but that's beside the point. The fact that we can put a map with high resolution satellite imagery on our web site for free is amazing in retrospect.

The impact on the mapping community was also pretty profound. I think we're still figuring out what the implications of that are. Neogeography, the geoweb and all the other things many of us hold dear, they take off with the Google Maps API.* It is hard to imagine this, but 5 years ago who would have thought this would happen? I know it's a cliche to say that about the web in general, especially for those of us old enough to remember before the web, but 5 years. 5! Can you imagine going to a site for a retail store and not seeing a map to their location?

But there's a lot more here than putting dots on your map, showing where your store is. That's important, revolutionary indeed, but think beyond that.

Maps mashups have been used to map election violence in Kenya, report potholes in the UK, show reports of people in need after the Haiti Earthquake, convey comprehensive data about Africa, and much more. Creating advocacy maps, informative maps, or fun maps no longer requires a professional cartographer. Nothing against cartographers, but put the power in the hands of the people.

So, what happens when you start mashing up data? You start looking for more data. The confluence of the Open Source movement with the mashup community produced a call for more open data. And because of the power of these mashups to put data in front of the people's faces quickly and easily, governments had to respond. In the US, in the UK and around the world, more and more data is being opened up by governments, NGOs, and to a lesser extent corporations. Bringing data to the public is a democratization. Knowledgeable, informed citizenry can respond to their governments, and help out as we saw in the case of the CrisisCamps that sprang up after the Haiti earthquake.

We still have a long way to go. Good tools are developing, but most people aren't programmers, to take advantage of the API, or know much about geographic data. That's why I'm excited about GeoCommons and Fusion Tables, because they allow people to use tabular data (say, in a Excel), probably the most used "databases" in the world.

But all those involved in mapping on the web, pat yourselves on the back. Look where we are compared to 5 years ago. I'm guessing most of those reading this blog are involved in Geo in some way, and so I'm saying to you: Thank you, you've done a great thing for thing for the world. You're bring democracy to the world. I know that sounds hyperbolic, and I know it's not the only thing driving increased access to information. But mashups, driven by mapping mashups (Google and otherwise) are helping change the world.

Map Styles and Usability: Please Help

2010-06-15T06:02:00.000-07:00

I've been away for a couple of weeks, and I'm now catching up with what's been going in the Geo blogging community. I saw a couple of posts on the Google Maps API new styling features:

Thoughts about Google’s Styled Maps announcement

Google Map Styles

There were more of course, but Steven Romalewski and Richard Treves raise some good points about usability. We've basically provided no guidance on usability of the new styles. The examples that we provide are designed to show extremes of styling to get the point across.

The truth is, we're mostly engineers, not cartographers. I'd love to see some great guides to how to style your map. Anyone want to give it a go? Anything good out there, I will make sure we link to it and talk about it.

To be fair, Steven is also concerned that it'll actually drive more people to use Google Maps API. His concern, our hope of course :-). He is concerned that this will reduce the commitment to other mapping platforms and perpetuate a mono-culture of maps. I don't see any danger of that right now, but I do appreciate that concern. Competition is good for us, it does help drive us to better things. So Cloudmade, Bing, OSM, everyone else, please make your mapping better. It helps us too.

Techniques for protecting your data

2010-06-15T05:00:00.000-07:00

I was asked in the comments in this post: Maps to KML?, to talk about techniques for protecting your data in Google Maps applications. I'm finally getting around to that.

First off, let's just say that like any part of the web, it is probably impossible to totally protect your data that is published on a Google Map. People can always get access to it at very least by just viewing the data, which you want, and making notes. Screenshots, viewing source, intercepts, and other techniques can be used by the truly ambitious. True data privacy in a completely public page is an oxymoron. Note, I'm not saying privacy on the web is bad or not possible, but we're talking here about displaying data. As long as it's displayed, someone can get at it.

That being said, there are steps you can take to make it harder to get at, to make people work at it.

The most important thing you can do is avoid hard coding any information into the page. That should be fairly obvious, but many people fall into the trap. That means:

Don't have any code in your JavaScript that uses a specific Latitude/Longitude pair, an address, or anything of that nature.
Use calls to server resources to plot only the data necessary to display at that moment. Try to verify the origin of the requests to prevent people from scraping. Generating your data on the fly prevents someone from getting all your data at once.
Obfuscate/compile your Javascript code to make it harder to read.

You can also rasterize your data, or turn it into image overlays. There's a lot of techniques for doing this. This talk by John Coryat is a couple of years old, and was oriented to Maps API V2. However, in it he discusses many techniques that are applicable to V3.

Rasterization makes it difficult to extract the data directly. It can also increase your performance in some cases where you have lots of data.

If you're working with KML, you can distribute the KML to only trusted people to load in their Google Earth instances, but this is subject to trusting them. Any KML used in a Maps API application will be easily findable by someone who can get passed any obfuscation you have.

That's all I've got. Feel free to post any additional techniques in the comments.

My slides from iHub Nairobi's Mobile Monday

2010-06-14T09:30:00.000-07:00

Slides from BarCamp Nairobi/ WhereCampAfrica

2010-06-14T03:23:00.000-07:00

So, it was pretty free-form, and I didn't stick to the slides, but there's still good links and resources here.

And, I hate this, but OpenOffice on the Mac corrupts PowerPoint export somehow. This prevented me from converting to Google Presentations, so I created both a PDF version and an OpenOffice version.

The images in the Eye Candy section are clickable.

BarCamp/WhereCamp Nairobi

2010-06-14T00:05:00.000-07:00

Over the weekend, I attended BarCamp Nairobi, which was combined with WhereCampAfrica. It was a great event, which filled both the iHub and Nailab spaces. Despite a fist fight that developed between myself, Stefan Magdalinski, and Mikel Marron, the event was otherwise very friendly and cooperative.

I went to great talks on:

Shika (a local ICT4D organization working in the slums in Kenya. If anyone has a link, please pass it on in comments)
AkiraChix
OSM
Seven Seas Technologies and entrepreneurship
Map Kibera

And too much more to post on. But like all Barcamps, the most interesting stuff is in the halls. I learned a lot about the emerging tech community here, and the difficulties of getting work in the face of a small percentage of the population being online. Like most of the developing world, they are jumping straight to mobile, largely skipping a large scale computer market. And mobile devices are where most people get their net access.

Of course, the event had to break each day for World Cup games, which were put up on the projection screen at the iHub.

It looks like the iHub is the place to be. My only regret on this trip is that I won't be here on the 26th, when the iHub hosts a big LAN party for gamers.

For those of you in Nairobi, I'll be presenting tonight at the iHub for Mobile Monday, about 20 minutes on using Google Mapping technologies on mobile devices.

Geo Highlights from Day 2 of Google I/O

2010-05-21T14:29:00.001-07:00

Wow, Day 1 at I/O was such a big day Geo, it would be hard to top it. But there were some amazing gems. Check out these highlights:

Styled Maps! Probably the biggest news of day 2. Maps API V3 now gives you the option to style your maps. Don't like golden highways and green forests? Change it! Check out our announcement for more details and links.
Matt Lowrie gave a great talk on the SketchUp API and using SketchUp.
Josh Livni and I previewed a whole bunch of additions to the Earth API and a new KML extension. In particular, Earth API now has control over the time slider, and better balloon handling. Now, you can preserve your JS and Flash in the balloons. In KML, we previewed the Track extension, which will allow you to assign multiple way points to a model or point and move it around, rather than recreating them with multiple Timestamps.
Finally, and this was actually announced on Wednesday, you can now add a FusionTable layer to a Maps API V3 app, right from the API.

It's really exciting to see all this. We're closing the loop on a lot of developer requested features, and we're really happy. Thanks for those of you who came or watch it on video.

Geo Highlights from Day 1 at Google I/O

2010-05-19T22:40:00.000-07:00

Boy, are my feet sore!

Wow, what an amazing day! Geo rocked Day 1 at Google I/O.

First, Daniels Lee announced that the Google Maps API V3 has graduated from labs and now is the recommended version of the Maps API to use. It also means that V3 is part of Google Maps API Premier, which is something people have been asking me about.

That also means that V2 is deprecated. We'll continue to support it, and fix bugs, for at least the next 3 years. Check out the deprecation policy in the terms of service. We're also deprecating Mapplets.

We also announced Street View in the V3 API, Flash-less so you can use it on mobile browsers. See my talk for more details. Videos and slides should post soon.

We announced a Directions web service as well, allowing us to close by far the single most requested feature in the issue tracker.

And finally, we previewed a Places widget, allowing you to show Places nearby your current location. It's built on the Places web service, now in Developer Preview.

There was also a fireside chat with Geo engineers and Product Managers, a Developer Sandbox with lots of great stuff on display, and a talk on Maps Data API by Tom Manshrek.

Plus, there were tons of Geo developers all over the conference. I think I talked to half of them. If you're in the other half, come and talk to me tomorrow!

Googe I/O Excitement

2010-05-18T10:25:00.001-07:00

At this point, I could just list every single Geo session at I/O and tell you I was excited to see them. Particularly the two that I'm presenting in. And that would be true. I could also rave about all the sandbox partners, and tell you how they were going to be fabulous. And of course they are. But the truth is, what I'm most interested in for Google I/O is all the great people who I'll meet. Honestly, developers are some of the most interesting people in the world, and I'm really happy that I'm going to be spending 2.5 days meeting those of you who are coming. So, come say hi, introduce yourself, I'd love to spend some time learning about what you're doing.

Maps to KML?

2010-04-21T10:40:00.001-07:00

People often ask me, can I take a Google Maps mashup, and make a KML file out of it (or in some other data format)? This question never comes from a Maps developer, only from people who see a cool mashup and want to overlay the data on their own map.

The short answer is: no, you can't.

The long answer is: no, you can't, and that would be wrong.

The longest answer is: no, you can't, unless the developer chooses to make a KML file available to you. It's possible, in fact, that they are using the KML file loaded onto their map using GeoXml or some other method. Otherwise, no, and it would be wrong.

I can understand the desire to do so. After all, a mashup is a combination of data from different sources, and a Google Maps API mashup is a mashup of data with Google's mapping API, in JavaScript, Flash, or just plain image URLs using the Static Maps API. And people often use public sources, or sources that are about things that are interesting or impacting lots of people, like Sports or the volcanic eruption in Iceland. Why wouldn't that be available to you?

First, from a moral point of view, the construction of that site belongs to the developer of that site. Usually, the data isn't presented to them raw, a certain amount of development has to happen to transform the data into a format usable for the app. In fact, they may have agreements with the data provider that they don't, or some license in the download from a public site restricts redistribution.

Of course, it's possible they just didn't think of it. Ping them, maybe they would do it for you just to be nice. Depends on the data of course.

Second, think about it from a technical point of view. Flash of course is it's own beast, and can write files. If the developer so chose, they could put a download data link or something. But as far as the JavaScript APIs go, JavaScript deliberately doesn't give you file access for security reasons. So a data file can't be taken from a JavaScript page. Nor has Google hidden an API inside the Google Maps API to allow other people to pull data out programatically. Again, that would be wrong.

I'm not saying that people can't look at your code and figure out your data sources, if you're the developer. If you want to protect that data source, you should design accordingly.

More thoughts on ARML

2010-04-02T15:56:00.000-07:00

Sean Gillies commented on my last post on ARML:

Is there anything in the Wikitude namespace that isn't already provided by KML or Atom (KML uses a bunch of Atom elements already and should probably use more)?

It got me thinking a bit more about the subject. First, yes, he's right, if KML supported more Atom links, then everything could be more readily handled by Atom. Or by their KML equivalents. I'm not sure why there's an ar:description, which could be handled by a KML description, or a wikitude:providerUrl, which could be handled by an Atom link element.

I'm concerned too about the extensions. I'd rather have a larger initial set of tags that are optional than put too much on provider specific extensions. If ARML is to really thrive, it'll be because killer browsers come along early, and standard ARML is robust enough to do what most of them will want to do. And, if developers come up with really awesome ways in which ARML can be used that aren't conceived of by the creators of ARML. I'm not sure that Augmented Reality will thrive yet, but if cross-browser AR apps are going to work, the initial standard needs to be robust enough and flexible enough without extensions. I'm not saying they're can't be extensions, just that it has to be able to stand on it's own. Browsers can choose what to implement.

BTW, despite being a KML fan, I'm not saying that KML is necessarily the only way to specify the location. It's great to see it used this way, and it has a lot of potential for cross-platform integration that way. One option might be to actually wrap things in Atom like the Maps Data API does and have additional AR specific elements surface in the Atom.

Hopefully, we can talk more about this at WhereCamp tomorrow.

Initial thoughts on ARML

2010-03-31T11:43:00.000-07:00

Yesterday at Where 2.0, I went to a great talk by Derek Smith of SimpleGeo and Martin Lechner of Mobilizy on Augmented Reality.. It was a great talk, but readers of my blog will probably not be surprised that I latched on to document formats as an issue. Here's an example from the ARML Specification:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
...

I love seeing KML used in other markups. However, I'd prefer to see something more like this:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2"
xmlns:ar="http://www.openarml.org/arml/1.0"
xmlns:wikitude="http://www.openarml.org/wikitude/1.0"
xmlns:wikitudeInternal="http://www.openarml.org/wikitudeInternal/1.0"><Document>
<ExtendedData>
<ar:provider id="mountain-tours-I-love.com">
<ar:name>Mountain Tours I Love</ar:name>
<ar:description>My preferred mountain tours in the alps. Summer and Winter.</ar:description>
<wikitude:providerUrl>http://www.providerhomepage.com </wikitude:providerUrl>
<wikitude:tags>travel, hiking, skiing, mountains</wikitude:tags>
<wikitude:logo>http://www.mountain-tours-I-love.com/wikitude-logo.png </wikitude:logo>
<wikitude:icon>http://www.mountain-tours-I-love.com/wikitude-icon.png </wikitude:icon>
</ar:provider>
</ExtendedData>
...

It may seem like a small change, but ExtendedData was explicitly designed to be a bucket of tags that are carried around with any KML Feature. While the current specification "works" in Earth, that's because it is forgiving, and other KML implementers might not parse it as well. Using ExtendedData would stick closer to the KML specification.

My Slides from Where 2.0 GeoSpatial Cloud

2010-03-31T11:41:00.000-07:00

My Where 2.0 Slides Going Mobile

2010-03-31T11:40:00.000-07:00

I Hate Slides

2010-03-22T10:23:00.000-07:00

OK, I admit it, I hate slides. Maybe it dates to when I was a kid and had to sit through many slide shows by parents, teachers, and other adults in my life. Oh, I should say for those of you who are not old enough to remember this, slides used to be bits of film, with a special projectors. That may have shaped my early dislike of a static medium.

Now, today's presentations are much more lively, you can do lots of fun stuff with them. Play videos, animate text, blink tags (nooooo!!!!!) and much more. My company even makes a product where you can collaboratively develop slides.

But, fundamentally slides are a data sink. How do you get data out of a slide? Copy and paste. It promotes a style of presentation that is artfully designed, and final. Sure, you can copy slides back and forth from one deck to another. Sure, you can change how they look, and copy and paste the text into other documents. But try to get the data out of the slides programatically. Or better yet, auto generate some slides from data. Wouldn't it be great to have all your slides tagged for content, or have it in an XML format that allowed you to XQuery for specific content?

The other thing I hate is it encourages people to think they can present a topic if they have the slides. That they don't need to know the actual content, just be able to read the slides and ad lib a bit. Slides should be an end point, a representation of an accumulated knowledge.

I prefer to show code or run demos, but there's usually not a great way to share that kind of presentation off a conference web site. But there's a well established tradition of sharing slides.

I actually developed a really not great version of a slides database back in 2005. It ran on Berkeley DB and had a Java interface. I realize how hard it is to conceptualize this kind of app.

I'm not saying I have the answer either. But I'm really tired of apps that are just about presenting data and don't actually allow any access to that data. How did we get to a point where the static visualization of content was the final point of how to communicate it? Why is our model for content sharing an outmoded form of film production?

Slides from WebMapSocial on Crisis Mapping

2010-03-16T20:32:00.000-07:00

My slides from WebMapSocial tonight. The whole subject was crisis response, and it was a great crowd!

My Slides from DevFest 2010 Tokyo

2010-03-11T05:12:00.000-08:00

Mobile Mapping with Google APIs

Client-Side Geocoding Rocks!

2010-03-05T19:31:00.000-08:00

Part of my job is to help partners with their code, making sure they are successfully implementing on Google Earth and Maps. The most common problem people come to me with is something like "Can you give me more quota, my http geocoding limit is reached." This is for the free Google Maps API. There's a simple solution. Don't do it!

The way the quota for the Google Maps API geocoder works is by IP address. If you put the geocoder in your JavaScript code (or Flash) it's rendering in the browser. And therefore counts against the IP address quota of the browser, not your site. If you have a single server side script that does an IP address lookup, it's going to be against the single IP address of the server. In other words, if your site is popular, you're going to run out quickly.

So, geocode in the browser and then send the geocode to do your lookup. We even allow for caching, as long as you're going to display on a Google Map or Google Earth. Caches can happen in the browser or server side.

A few additional things to think about:

1) Cloud computing services often share a range of IP addresses. If you're running on AWS or Google App Engine or any other cloud computing service, you particularly want to do client side geocoding, as you may be running quotas parallel to other apps.

2) Some mobile networks share IP addresses as well. That can cause problems for your client side software, as lots of people on their smart phones are looking at your map. If you anticipate heavy mobile use, you might consider having a server-side back-up as a fail-over. Try to geocode and if that doesn't work, send the address to your server for http geocoding.

3) If you're still running into problems with quota, you may consider a Maps API Premier license. Check out http://maps.google.com/getmaps for more information on the differences and to get started.

Slides from my upcoming talk at Berkeley

2010-01-22T18:57:00.000-08:00

These are still rough and un-pretty. Feel free to offer suggestions, additions, etc.

Ways to Help and Resources in Haiti

2010-01-14T11:25:00.000-08:00

This is a sort of dump of things I know about going on, piling on what others have written:

Google's Official Response Page

This page contains information on how to donate, satellite imagery, news links, and links to hospitals. We'll probably update this with more resources as they become available.

Google MapMaker

Google MapMaker is a providing data directly to the UN for relief operations, so data you provide will go directly to them.

OpenStreetMap Extracts for Haiti

Get involved with OpenStreetMaps and contribute data.

Contribute health alerts to HealthMap

Either through #healthmap or directly through their site.

Go to CrisisCamp

There's a special CrisisCamp on Saturday in Mountain View, getting volunteers together to work on technical projects, including data, maps, and technical assistance to NGOs working in the region.

GeoCommons Map of the News

Well, the title says it all doesn't it?

Get Involved with CrisisCommons

"Inspired by and started CrisisCamp DC in June, 2009. CrisisCommons is meant to capture knowledge, information, best practices, and tools that support crisis preparedness, prevention, response, and rebuilding"

Check Out James Fee's Post

He's put together a long list of resources and ways to get involved.

Workshop: Working with Geospatial Data Using Open Source Tools

2010-01-07T10:58:00.000-08:00

I'm giving the following workshop. Any suggestions for content are welcome. And you are welcome to attend, whether or not you're a UC student.

Title: Working with Geospatial Data Using Open Source Tools

Location: 110 South Hall, UC Berkeley,

Date/Time: Monday, January 25, 2-5pm

There are hundreds of millions of geospatial data files available on the web. Many of these files are in a format that makes it hard to access the data, or combine it with other data sets. This workshop will introduce the basics of geospatial data formats, and using open source tools to work directly with geospatial data. In particular, you will learn about shapefiles, KML, GeoRSS, GeoJSON and other standard formats. And you will learn about using the GDAL/OGR packages, GeoServer, and other tools to work with the data. And finally, you'll learn about some options for displaying the data on a map.

Geography: A New Hope

2009-12-21T05:38:00.001-08:00

Last week, I gave a talk to my nephew's geography club at. He's in middle school, and it gives me immense pride that he is so fascinated by geography. When I was a bit younger than him, I was thrown out of a geography contest in my class. Apparently it wasn't fair to the other team to have me be able to identify all the U.S. states by shape. I have high hopes for my nephew that the same thing will happen to him some day.

It was amazing to see this club though. About 25 or 30 students, all eager to see a demo of Google Earth. When I give a talk now, no matter the age group, it's the same result. But these kids instinctively understand the 3d nature of the application, and are curious about subjects like "How do you get the imagery to conform to the terrain when you drape it over?" "Geometry," I said, getting a big smile from the teacher. All of the questions were at least as smart as the questions adults ask me, leading me to think that either these are special kids, or there is hope for America's schools.

Mobile Maps and Physicality

2009-12-10T10:15:00.000-08:00

I've been working a lot with Google Maps API V3 and running it in an embedded browser on an Android or iPhone device. And this has got me thinking a lot about the sheer physicality of mobile maps.

I'm pushing 41 this coming February, and I'm old enough to remember when maps were only on paper. There was a presence to maps, holding it in your hand, tracing it with your finger. You could put a map of the city in your pocket, literally carrying the city around with you. The way you folded it was really important, a good test of whether someone was a maps person or not.

I confess the other way I loved maps was a gamer. Those hexagons on a map, moving chits around to conquer territories. I loved it. As an early teen, we'd read game maps and make up pronunciations for city names and argue over them. Back when people argued about facts instead of looking them up.

Don't get me wrong, I love Google Maps on my computer, I love the slippy nature of it, and the ability to manipulate it in ways that I couldn't have with a paper map. But working with mobile mapping has brought back some of that old feeling. Sticking the phone with the map open into my pocket. Moving it around with my finger. There's a physicality to it that I miss on my non-mobile computer. Sure, I don't have to learn a special folding technique. But still, I can carry the city around with me. And best yet, when I pull it out, it'll be right there, showing me where I am without me having to figure it out. OK, maybe I'll miss the figuring it out part.

Open Source Disaster Response CodeJam

2009-10-14T15:10:00.001-07:00

Please join me and the Google Disaster Response team for the first of hopefully many Open Source Disaster Response CodeJams in Mountain View, November 12-14, 2009. http://randomhacksofkindness.eventbrite.com/

Slides from NIEM Training

2009-10-12T15:34:00.000-07:00

Here are my slides from the National Information Exchange Model Training in Baltimore October 1st.

My Slides from Location Intelligence

2009-10-06T15:17:00.000-07:00

Here are my slides from the Location Intelligence Cloud Computing panel.

More Musings on the HTML Model and the GeoWeb

2009-09-30T19:04:00.000-07:00

In my last post, I mused about the state of GeoWeb standards and wandered off into a discussion mostly on findability and linking between different files. I've been thinking more about the HTML web and how we use it as a model for what we call the GeoWeb.

The HTML web famously starts with Tim Burners Lee, Robert Cailliau, and their famous 20 tags, 13 of which we still use today. HTML sprang out of SGML, the Standard General Markup Language, but was far simpler, and, well, usable. We all know that the web took off like a rocket, and now the Web is synonymous with the Internet in the eyes of many people. HTML is the vehicle, but as developers know, it is only a part of web. Granted, it's the part that the other parts all depend on, but here's a partial list of the technologies without which the Web as we know it would be fundamentally different:

1) HTML

2) HTTP

3) JavaScript

4) XML

5) JSON

6) Flash

7) CSS

OK, Flash is controversial, I'll give you that, but you have to agree, the Web would be really different without it. Some of you would say better, but very different. Of course, there are other technologies people could put in there, like ASP, PHP, Python, etc. My point isn't the specific technologies, though I'll talk about JavaScript in a minute, but the fact that any developer could come up with this list. I'd like to compare that to a list an enterprise apps developer might come up with:

1) Java

A much shorter list. Again, you could argue about the content of the list, but the fact is, for many other domains, a developer can learn one language, one piece of technology, or perhaps two, and that's it.

So the web is different. It is hard to imagine a serious web designer now who doesn't know JS and HTML and probably PHP or Python or some other server-side scripting language. Maybe Flash instead of JS, but you get the idea. I call it the HTML Web because without HTML the rest couldn't work. But, it's hard to imagine a serious web site without using these other technologies.

I was thinking about this because I am asked all the time, why doesn't Google add JavaScript or some other scripting capabilities to KML. Of course, we no longer own KML, we gave it to the OGC, so we can't just "add" something to KML, aside from our own language extensions. But the question shows that there is a fundamental desire on behalf of developers to have that functionality. Serious GeoWeb developers learn a variety of technologies too. For a basic Google Maps mashup (or Bing, or Yahoo! Maps too), you need at least HTML, CSS, and JavaScript. More complicated mashups will use KML, GeoRSS, or GeoJSON, maybe Flash instead of JS, and wait. Notice the first one of those technologies, HTML. Still fundamentally, the GeoWeb relies on HTML. Sure, with a GeoWeb browser like Google Earth, you don't need HTML, though you can use it in the description balloons. But most mashups require HTML as the carrier.

So for the GeoWeb, we've learned the lessons of the HTML web. The question I'm asking, I guess, is, is this the right thing? Is building on the HTML model, with it's confusing amalgam of technologies, many of which don't work easily together, really the best model for the GeoWeb. I don't have the definitive answer to that question, but I think it's worth asking. Would it be better if we had a single technology to create Maps and distribute them?

OK, I have a preliminary answer, yes it would be better to follow the HTML model, and have an amalgam of technologies than to be locked into a single one.

Washing DC Plug: Google Partnership Exploration Workshops and Hackathon

2009-09-22T15:53:00.000-07:00

Google is putting on Partnership Exploration Workshops and a hackathon in a couple of weeks. This DC event is going to be great. How do I know? I'm helping to plan it. The focus will be on Geo and Visualization technologies. Anyway, you should come. Here's more info:

Partnership Exploration Workshops: Your Mission & Google Technology

We are happy to invite you to an interactive session with Google product experts and Google.org members, featuring a keynote address by Craig Fugate, FEMA Administrator, and followed by an optional developer hackathon.

There's no doubt that technology, developed both in-house and by 3rd parties, plays a role in helping your organization. With the convergence of GPS and mobile devices, the emergence of crowdsourcing tools, and improving large data set visualization capabilities, new opportunities to guide, manage and empower your field operations are opening up everyday.

Join us for a day of interactive workshops where we explore how Google geo & data visualization technologies can further your mission. Craig Fugate, FEMA Administrator, will give a keynote address on technology and disaster preparedness. Google product experts will provide overviews and analysis of recent developments and upcoming innovations for Google Earth, Maps, Map Maker, Fusion Tables and more.

Before the workshops, we will work with you individually to identify projects that might benefit from Google technology. Select attendees will present and lead discussions on their geo & data visualization technology needs. In addition, we will host an optional developer hackathon the day after the workshop to kick-start your integration of Google technologies. We look forward to seeing you there!

When

Interactive Workshops on Thursday, October 8

9:30am–4:30pm (lunch provided)

Optional Developer Hackathon on Friday, October 9

9:30am–4:30pm (lunch provided)

Where

Google DC Office

Rose Garden Tech Talk, 2nd floor

1101 New York Avenue, N.W.

Washington, DC 20005

RSVP

Please click here to RSVP. These events are by invitation only.

Thoughts on GeoWeb Standards

2009-07-28T13:32:00.000-07:00

Andrew Turner recently asked for our thoughts on GeoWeb standards, and I thought I'd put it as a post here instead of cluttering too much of his comment stream.

I've been thinking about the different standards and their place in the world a lot recently. I'm not someone who takes strong stances on anything, and you're not, I hope, going to read this post and think that I'm a KML partisan, and that it's only because I work at Google that I think positive thoughts about it. I prefer instead to explore the problem space.

The problem isn't adoption, clearly. It's findability.

Adoption rate:

There's no question that KML has a phenomenal adoption rate. Michael Jones went over the numbers during his GeoWeb talk, but in case you missed it:

More than 500,000,000 KML/KMZ documents on the Internet
More than 250,000 Internet websites hosting KML/KMZ content
2 billion placemarks accessible on the public Internet

Those are staggering numbers, especially compared to just last year when Google announced it had indexed tens of millions of KML files on a hundred thousand unique domains. Growth by a power of ten in a year is a lot.

GeoRSS has also expanded rapidly. I don't have numbers on it, but I'm sure it's also a very large number.

There are other formats, too, like GeoJSON, that are great, and I really look forward to seeing what happens with them.

Findability

Frankly, I think we can do a much better job. Fundamentally, one of the problems is that geographic data doesn't lend itself well to linkablity. Sure, you can link within the data, but few people do. A limited number of KML files link to other KML files. GeoRSS can contain a variety of links, but often not to other geographic data files, but rather to HTML or binary media.

KML has been described as the HTML of geographic data. Whether that's true or not is a matter of some discussion, though I happen to think it is (more on that in another post I guess, after lots of people tell me I'm full of it). But one of the principle characteristics of HMTL is linking, which is weakly implemented in KML. Linking happens in two places, the Atom link element, and in the description balloon. Atom links usually refer to HTML media, as in "this is the site credited with authorship." In the description balloon, you're operating in essentially an HTML environment, leading people to be less likely to author KML with links to other KML files, but rather with links to HTML. When authors do put links to KML, it's mostly within their site, not to other KML files elsewhere.

My point isn't to encourage people to link to KMLs created by others, but rather that for findability purposes, on the HTML web we rely primarily on a link structure. The early web was made up of pages that delivered content, and linked to other sites. Whole pages developed early as directories of other sites, and they linked to other directories. Google web search was built on using the number and authority of links of others to rank pages. The "GeoWeb" isn't really a web in the same way. It uses the technologies that built the web, that live on the web, but it itself doesn't constitute a web in a meaningful sense. The vast majority of links to geographic data that I've found are HTML links within full HTML pages, with the next set being programmatic.

Is that the nature of geographic data? Or have we just not found the true linkability of it? I tend to think it's the former. Geographic data is heirarchical, it is ontological, it is content rich, it is combinable. It is linkable through common ontologies. But geographic data doesn't lend itself to easy linking in the same way. It's the nature of structured data, it must relate to a structure. Ontologies are almost the antithesis of linkablity outside the domain.

So that suggests that we need to find another mechanism for findability. Deep searches are possible, but generally when you want geographic data, you either want points on a map, "This is where that thing is" which is fairly easy to do, and I think we've done it well. Or, you want a metadata search of some kind, "Give me all the polygons that fall within this bounding box, and where property X is between Y and Z." That no one does well on a global scale, only within limited sets of data. Searching on text is great for web pages, because they are composed primarily of text. But searching for data is a whole other problem not easily solved by our current mechanisms.

Some people have written about using Semantic Web technologies to provide the linking, and particularly Andrew notes LinkedGeoData in his comments on his blog post. I've always been of the opinion that the Semantic Web is too complex. One of the joys of HTML is the ease by which you can link pages. The authoring tools aren't really there yet either. I'd be happy to be proved wrong. I used to think that standards, like RDF, that have languished for so long will never take off. However, the explosion of Ajax in the last few years has made me less skeptical. I don't know if the Semantic Web is the technology of the future and it always will be, or if it will actually take off. I remain fairly skeptical however, and as yet there's no widely adopted viewer for it either.

Combinability

Perhaps the true value of XML based formats comes from their combinability. Whether it's Atom (or RSS) and GML to make GeoRSS, or Atom and KML to produce a Google Data API, or Atom and KML to produce, well, KML containing Atom. This greatly increases their usability, and I think I sense another post coming on since this one is getting long. But my point being the XML standards provide the only really good way of doing this while retaining proper namespaces. The downside is, of course, the verbocity of XML and the pain of XML schema.

Wrap Up

Don't get me wrong, I think that KML and GeoRSS are great, as are a lot of other formats I haven't mentioned, like GeoJSON and others. Andrew asked also about other interesting topics, like expressiveness and durability, which I haven't gotten to. Ultimately, though, if we can't solve the findability problem, other technologies will come in that do.

Wrap-Up on GRUPP

2009-07-06T18:13:00.001-07:00

I took a vacation immediately after GRUPP, so I didn't write a follow-up post to my Day 1 post. But I did want to just follow-up with a couple of resources that stood out for me beyond what I already wrote:

JavaRosa
JavaRosa is an open-source platform for data collection on mobile devices. Basically, a supped-up XForms browser for Java ME, it's pretty awesome.
Open Data Kit
Built on JavaRosa, it's designed specifically for Android, and to take advantage of all the awesomeness that is an Android phone.

I've decided in the coming weeks to play with ODK and blog about it, and try to think about other solutions for mobile offline/online data collection. I haven't done mobile development before, so this will be interesting.

GRUPP Day 1: Thoughts on data collection

2009-06-23T23:11:00.000-07:00

Some preliminary thoughts at lunch:

I'm at the GCamp@RUPP meeting in Phnom Penh right now, hosted by the Royal University of Phnom Penh. The subtitle is "Exploring Emerging Technologies to Address Emerging Infections."

Wow, data collection is hard. I knew this from spending 10 years designing and implementing systems in non-profits in California, but I had no idea what the challenges were in the developing world. Many of us here from Google, primarily engineers, came here with some naive notions of how we can help. Here are just some of the issues we've been rocked by:

Lack of connectivity: So, it is common knowledge that SMS is used the world over, aside from the US, often more than voice connections. But in some emerging countries, it isn't. Think about this: Khmer, the language of Cambodia, isn't yet represented in Unicode. No UNICODE! Apparently, they just didn't have people representing Khmer during the meetings that setup Unicode. Klingon and Tolkein's Elivish are in Unicode, but not Khmer. This has a deep impact on technology: SMS doesn't work in Khmer. Plus no phones are made with Khmer keyboards, and to even produce an app, like a form, you have to render Khmer as an image file. Fortunately, this is being remedied, but it'll be a long time before phones and computers can support it.

Incentives: What incentive do people have to provide information? User generated content needs to give something back, and not create perverse incentives.

Unavailability of data: Many in the Open Source community want all data to be free. But many governments have incentives not to share. For instance, a disease outbreak can cause a catastrophic drop-off in tourism. So if the government doesn't share the information that there is one, then people are less likely to stay away. See also: Privacy

Anyway, just some thoughts, more later.

HASTAC Presentation

2009-04-20T14:48:00.000-07:00

I had a sinus infection, so I couldn't fly. HASTAC let me present via WebEx, which was pretty cool I thought:

Blogging versus Twitter

2009-03-12T17:05:00.000-07:00

I've been thinking about this a lot. I've noticed that my private blogging, which is not available to the world only a few friends, has dropped precipitiously since I started using Twitter. And this blog, which I thought I would use more, hasn't had a lot of posts either. This is definitely the Twitter effect on me, and I kind of like it. Feeling Entropy has been thinking about this too, likening quick easy blogging to fast food, as opposed to longer well crafted posts.

Here's the difference, for me:

I try to use the blog for long, well thought out posts. I don't really care for posts that are simply a link and "Hey, look at this!". Not that I mind linking to other sites, as long as the post does a nice job explaining why it's important that you examine this particular link. I prefer to use blogs as a discussion, an explanation of a topic.

Twitter, on the other hand, has become my fast food post. The 140 character limit allows me to just do a quick "Hey, look at this!" post. Also, people seem to be scanning their Twitter feeds more frequently than their blog readers. Personally, I have almost 1000 unread blog posts in Google Reader. I say almost, because I fight to have it under 1000 as Reader flips a bit and says 1000+. Apparently, I like to know the numbers exactly. But I do scan almost all the 143 people I'm following on Twitter daily.

So, where is all this leading us? Are we truly going down the route, as Feeling Entropy says, of a fast food blogging culture? Now that it is easy, we'll produce less quality? I don't know. But strangely, I feel as if my Twitter feed is having more impact than this blog.

Or maybe that's just because the Twitter effect means I write here less.

Slides from Visualizing the Past

2009-02-21T10:25:00.000-08:00

I presented at Visualizing the Past on Google Geo technologies and their use for historical visualization. I didn't get to the part on the Google Visualization API, but the links are in the slides.

Finding Love Somewhere on Google Earth

2009-02-14T00:38:00.000-08:00

For those of you still trying to find love, or who want to get a date for next year, I put together this sample app. Oh, BTW, careful, some of the images might not be safe for work. Sometimes people put risque things in their profiles.

I queried personal ads in Google Base using the Base API. Several different personals sites push their ads into Base to make them more discoverable. Base allows for location based queries, including a bounding box query that looks like this:

http://www.google.com/base/feeds/snippets?bq=[item type:personals][location(location): @+34-086..@+37-092]&content=geocodes

That query returns an Atom feed of personal ads within the bounding box. Don't worry, they don't put addresses in the ads, only City, State/Province, and Postal Code. And Base allows you to directly geocode within the query, returning additional g:latitude and g:longitude elements, to save you the hassle.

KML provides a convenient View Based Refresh. Simply put a viewRefreshMode of in a Link element, and Earth and Maps will send query parameters defining the bounding box of what is visible to a server. So, I put a simply python script up on an App Engine application, let it parse the bounding box parameters, and generate the queries, returning new KML every time the view pauses for more than a few seconds. Then I created a simple Earth API page, nothing fancy since it was getting late and I was tired, and loaded up the KML.

I could have used one of the Google Data Client Libraries, OK really only the Python one because I was using App Engine. But frankly, it was such a simple query, and I really love raw XML, I decided to stay with direct querying and DOM parsing.

If you want more info on View Based Refresh, look at the KML reference for viewFormat.

First!

2008-11-18T17:42:00.000-08:00

OK, here's the obligatory first post, to let people know who I am and what I'm doing. I am Mano Marks. I am, at this point, 39 years old, and I've done a fairly wide variety of things with my life. In 1993, I got a Masters in History from Columbia, and in 2006 I got a Masters of Information Management and Systems (like a post-modern Library Science degree) from UC Berkeley. In between, I worked at a high school, a Masters program, taught college, worked at a rape crisis center, a community center, a youth services organization, and a hardware distributor. I've imagined myself a professor, a social worker, a programmer, and most recently a technology evangelist. What really shines through, though, is that I am and always will be, a geek. I don't tell you all this to brag, just as a warning, my ideas and interests tend to come from a lot of different places.

I work at Google, but this isn't an official Google blog, just my personal one. I started it because I have a lot of thoughts on interesting projects and like to record things that I bump into, great apps and code, things that might not make it into an official blog. Also, this way I can blog about some of my other interests, things that don't necessarily directly relate to Geo, like history, linguistics, disaster relief, politics, document engineering, and possibly even video games. Hopefully, some of it will be interesting. Anyway, now that the first post is out of the way, we'll see how long it takes me to get to the next one.