Wednesday, April 30, 2008

Microsoft's Big Bang: The World Wide Telescope

I had came across a few months ago, an article saying something about Microsoft's new research project that was coming into light. It was from a former Microsoft employee, I guess, that on watching a demo of this project burst into tears-sorry but I seem not to be able to find the original article. I initially thought, "Well, Yet-Another-Viral-Campaign", and quickly forgot about that. After all, I am pretty much satisfied with Stellarium and I also know Google Sky is out there. Well, it seems I was totally wrong.

The project is named World Wide Telescope, and was created by Jim Gray, a computer scientist who received the Turing Award in 1998, joined Microsoft in 1995, embarked on this ambitious project 6 years ago (June 2002) but disappeared in 2007, during a short boat route to scatter his mother's ashes. From that time until now Microsoft is continuing his research with astonishing results, and based on the people that have experienced the new technology, it has by far no match out there. You can find his original paper here. It worths reading, trust me.

The World Wide Telescope is an integration of a thousand of technologies. For the user it is a great tool to virtually (or actually) roam into the universe. For computer scientists it is a fascinating theory put into breath-taking practice: federating data from multiple sources, correlating and producing new insights, to paraphrase a little. So, the WWT is a lot more than a new killer application: It is a Big Bang for Computer Science, since it manages to process multiple sources of huge amounts of data from astronomy sites such as http://SkyServer.SDSS.org/. This leads to better understanding over the data and making scientists more productive.

The computational challenge lies not only on the volume of the data which however scale to petabytes. The hardest challenge comes from the need to integrate heterogeneous data, its with its own format of course. This actually requires deep astronomy knowledge and strong processing capabilities. The creators of the WWT give a crystal clear example:

"A colleague challenged us to find “fast moving” asteroids. This was an excellent test case – our colleague had written a 12 page Tcl script that had run for 3 days on the flat files of the dataset. So we had a benchmark to work against. It took a long day to debug our understanding of the data and to develop an 18-line SQL query. The resulting query runs in a minute."

What the creators had to do, was to understand the format of the astronomical data to analyze and construct the proper (but sophisticated) SQL query. Their result was more than convincing and a strong proof that an idea like the WWT could actually be implemented. Now, can you guess why Jim Gray won the Turing Award?? That's right, for contributions to database and transaction processes. Of course, I might say!

But enough with the talk. Here you can find the video on TED where Microsoft first presents a glimpse of the World Wide Telescope. The official website is already up but the WWT is not yet available to the public. Enjoy grandma-Microsoft's new creation!