Semergence

After reading about Sam Ruby’s issues with JSON for Map/Reduce, it got me thinking. How long before Google will let us run our own Map/Reduce programs on their clusters?

We all know, one of the best ways to scale is to push the operation to the data. And who has all the data? Why, that’s Google, of course. They are in a perfect position to host and run applications which map/reduce the web for individuals or organizations.

Imagine that I want to start collecting results by microformats such as hCard. It would be nice to formulate a query which understands microformats as a map/reduce and send it off to Google.

Facebook got this right by creating a Platform. Developers can write applications which link directly into Facebook. I want to do the same with Google.

By the looks of things at Google, I’d guess that Google Gears is part of Google’s plan to take their Google Apps offline.

Google Gears bundles SQLite, and provides strategies for syncing data back to the web server.

Just thinking about how to get better ROI from my del.icio.us links. I typically put links in there that I want to retrieve later, usually multiple times. I don’t put transiently interesting links in there usually, as I know I’ll never go back there. I use del.icio.us to store links I know I’ll want to refer to again and again.

Problem is, del.icio.us is a black hole. I throw stuff in there, and I never return to it. Why? Because the little Google search box in firefox makes it way to easy to just search google.

What I need is a single search box, but with the ability to search google and del.icio.us. I want to see my del.icio.us links have very high priority on my google searches. If I can integrate the two databases, then I’d have something special.

I’m sure there’s a way to do this with a firefox plugin.

There real issue is, “How do I know I put something into del.icio.us?” Once it’s in there, how do I know I’m even able to retrieve it? That’s why google works so well, because I don’t have to know if it exists or not.. I just assume it does. With del.icio.us, that assumption isn’t valid all the time. But my workflow doesn’t involve me asking myself, “Maybe I bookmarked it, so first let me search del.icio.us” I need the searching of my bookmarks and the whole web to be integrated into one simple search.

Andrew’s Website has a very interesting writeup on Google’s BigTable, their system for storing large amounts of data in a semi-structured manner.

> BigTable has been in development since early 2004 and has been in active use for about eight months (about February 2005). There are currently around 100 cells for services such as Print, Search History, Maps, and Orkut. Following Google’s philosophy, BigTable was an in-house development designed to run on commodity hardware. BigTable allows Google to have a very small incremental cost for new services and expanded computing power (they don’t have to buy a license for every machine, for example). BigTable is built atop their other services, specifically GFS, Scheduler, Lock Service, and MapReduce.

There’s some audio available, too.