Microsoft's Big Bang: The World Wide Telescope

I had came across a few months ago, an article saying something about Microsoft's new research project that was coming into light. It was from a former Microsoft employee, I guess, that on watching a demo of this project burst into tears-sorry but I seem not to be able to find the original article. I initially thought, "Well, Yet-Another-Viral-Campaign", and quickly forgot about that. After all, I am pretty much satisfied with Stellarium and I also know Google Sky is out there. Well, it seems I was totally wrong.

The project is named World Wide Telescope, and was created by Jim Gray, a computer scientist who received the Turing Award in 1998, joined Microsoft in 1995, embarked on this ambitious project 6 years ago (June 2002) but disappeared in 2007, during a short boat route to scatter his mother's ashes. From that time until now Microsoft is continuing his research with astonishing results, and based on the people that have experienced the new technology, it has by far no match out there. You can find his original paper here. It worths reading, trust me.

The World Wide Telescope is an integration of a thousand of technologies. For the user it is a great tool to virtually (or actually) roam into the universe. For computer scientists it is a fascinating theory put into breath-taking practice: federating data from multiple sources, correlating and producing new insights, to paraphrase a little. So, the WWT is a lot more than a new killer application: It is a Big Bang for Computer Science, since it manages to process multiple sources of huge amounts of data from astronomy sites such as http://SkyServer.SDSS.org/. This leads to better understanding over the data and making scientists more productive.

The computational challenge lies not only on the volume of the data which however scale to petabytes. The hardest challenge comes from the need to integrate heterogeneous data, its with its own format of course. This actually requires deep astronomy knowledge and strong processing capabilities. The creators of the WWT give a crystal clear example:

"A colleague challenged us to find “fast moving” asteroids. This was an excellent test case – our colleague had written a 12 page Tcl script that had run for 3 days on the flat files of the dataset. So we had a benchmark to work against. It took a long day to debug our understanding of the data and to develop an 18-line SQL query. The resulting query runs in a minute."

What the creators had to do, was to understand the format of the astronomical data to analyze and construct the proper (but sophisticated) SQL query. Their result was more than convincing and a strong proof that an idea like the WWT could actually be implemented. Now, can you guess why Jim Gray won the Turing Award?? That's right, for contributions to database and transaction processes. Of course, I might say!

But enough with the talk. Here you can find the video on TED where Microsoft first presents a glimpse of the World Wide Telescope. The official website is already up but the WWT is not yet available to the public. Enjoy grandma-Microsoft's new creation!

European ICT Report 2007: "Greece Sucks, Denmark Rules"

I was recently in Brussels, having some work in the European Commission. Having finished and while waiting in the lounge of the building of our meeting, I grabbed a booklet, from the shelves around, titled "European Commission's Annual ICT Report: 2007". This special branch of the EU publishes every year statistics about the performance of its members in the area of Information and Communication Technologies.

I browsed immediately to the page talking about Greece but the phrase "among the lowest" kept showing over and over again in the text! The summary was: "Information Society in Greece is still developing slowly and on most indicators Greece is close to the bottom of the EU. There are signs of increasing gaps compared to other EU countries". Damn...EU also says Greece is among the lowest in broadband take-up, use of internet, eGovernment use, number of connected PC's in schools and ICT skills among employees.. Wow! We are also among the lowest nearly in every indicator.

On the other hand, Denmark seems to be the champion. Denmark places 1st in many crucial indicators like DSL coverage, e-Learning and e-Commerce. Skill level is considered among the highest in Europe and Denmark is considered as a leader in developing the European ICT society.
You can find more statistics about the European ICT status here.

This is not news for us, however. Our strategy is void because people in charge, are void. Having skilled employees while having a Prime Minister that has never been employed anywhere, would be a contradiction.

The good news is that, since we are at the bottom, there is only one way: Going up! :)

"Time Is What Prevents Everything From Happening At Once.." - John Wheeler (1911-2008)

The quote continues: "Space is what prevents everything from happening to me!"
John Wheeler, has for a long time, been studying advanced concepts of theoretical physics, which for the most people are somewhat taken for granted, space,time, gravity and so on. He passed away in Sunday, 13th of April 2008, at the age of 97.

John Wheeler was one among the last of the 'big school' of theoretical physics. He has worked with legendary figures such as Niels Bohr, Oppenheimer and Albert Einstein(related photo) and he is the inventor of the popular term 'black hole', this strange 'creature' that is so big that even its own image cannot evade from its infinite gravity (visible light cannot reflect or be emitted) At some point in 1941 Wheeler joined the famous (or infamous) Manhattan project. Among his graduate students as a professor in Princeton, were Richard Feynman and Kip Thorne.

I suppose, everybody from a CS background has developed at some stage of his life a certain interest for astronomy and physics. I, my self, at a young age was a fan of such fields along with science fiction. I never actually understood what was going on with these strange quantum equations but I think I find my self equally astonished when confronted with concepts like 'black holes' or 'time bending'. Astonished at least.

You can find lots more information in the official press release from the Princeton University site.

Why The Array Index Should Start From 0

I recently came across an article from Dijkstra(photo) explaining a fact that looked obvious to me before: Why should start indexing an array from 0. I was amused to find out that there is some clever thinking behind this.

All popular languages, like C/C++, Java or Perl start indexing an array from 0 while the last index is the array length minus 1. While this is usual to most developers, it is not a de facto situation for all programming languages. For example in Fortran, when you declare an array with 'integer a(10)', aka an int array having 10 elements, the index starts from 1 and ends at 10 (however you can override this behavior using a statement like, integer a(0:9), declaring an array with indices from 0 to 9, but anyway).

The discussion over why the array index should start at zero is not a trivial one and relates to interesting concepts from computer science. First of all, it has strong relation to language design. For example in C, the name of an array is essentially a pointer, a reference to a memory location, and so the expression array[n] refers to a memory location n-elements away from the starting element. This means that the index is used as an offset. The first element of the array is exactly contained in the memory location that array refers (0 elements away), so it should be denoted as array[0]. Most programming languages have been designed this way, so indexing from 0 is pretty much inherent to the language.

However, Dijkstra explains why we should index from 0. This is a problem on how to denote a subsequence of natural numbers, say for example 1,2,3,...,10. We have four solutions available:
a. 0<i<11
b. 1<=i<11
c. 0<i<=10
d. 1<=i<=10

Dijkstra argues that the proper notation should be able to denote naturally the two following cases:
1. The subsequence includes the smallest natural number, 0
2. The subsequence is empty

Requirement 1. leaves out a. and c. since they would have the form -1<i which uses a number not lying in the natural number set (Dijkstra says this is ugly). So we are left with b. and d. Now requirement 2. leaves out d. since for a set including 0 that is shrunk to the empty one, d. takes the form 0<=i<=-1, which is a little...well, messed up! Subtracting the ranges in b. we also get the sequence length, which is another plus. Hence we are left with b. which is by far the most widely used notation in programming now.

Now you know. So, remember and take pride in the fact that each time you write something like

for(i=0;i<N;i++)
{
sum+= a[i];
}
you are not just following the rules of language notation. You are also promoting mathematical beauty!

Popular Demand: The Youtube Download PHP Scripts

I had posted an article explaining how to download YouTube videos by knowing its URL. This technique can be implemented in any language, however I chose to make it in PHP.
This enables you to have it on your own sites.

The regular readers of DOL might have noticed a little iFrame, at the bottom right of the blog, that hosted the PHP scripts in my home machine. However, sometimes I have to go to sleep and sometimes I tend to shut the PC down. Also, my connection is a miserable 2M ADSL line so my home network usually was crumbed by heavy traffic. As a result the YouTube download tool was not always working and this was frustrating to many people. I agree and I apologize.
So, under popular demand here are is the source code. Enjoy!

iFrame HTML code (trivial):

<html>
<head>
<form action="dnlUTube.php" method="post">
<p><b>YouTube URL:</b> <input size= 28 type="text" name="name" /></p>
<p><input type="submit" value="Get it!" /></p>
</form>
<i>Example: http://www.youtube.com/watch?v=319hsZ12TAE</i>
</head>
<body>

</body>
</html>

Now this refers to the PHP file dnlUTube.php, that actually retrieves the download URL of the video. The source code is:

dnlUTube.php : extracts YouTube download link

<html>
<?php
#
# Script to download YouTube videos
# Originally crafted by panefsky on Developer On Line- Salonica,Greece 2008
# Use free. Credits welcome.#
# Input: Post parameter 'name' is the YouTube URL
# Known issues: sometimes the parsing fails either because
#YouTube does not send the proper format or
#whatever.. However 2nd invocation of script always succeeds! You might contribute at that.
#
# Enjoy!
if(!empty($_POST['name'])) {

$location = htmlspecialchars($_POST['name']);
try {
$handle = fopen($location, "r");
if($handle) {
$contents = '';
while (!feof($handle)) {
$contents .= fread($handle, 8192);
}
fclose($handle);
$result1 = preg_match("/&t=([\w]*)&/",$contents,$tickets);
$result2 = preg_match("/v=(\w*)/",$location,$video_id);

if($result1) {
echo "<a href = \"http://www.youtube.com/get_video?video_id=";
echo $video_id[1];
echo "&t=";
echo $tickets[1];
echo "\">Download link.</a>";
echo "<br>";
echo "Click n Save with flv extension";
}
else echo "Damn! Click Back and try again..Sorry :(";
}
else echo "\nYou liked that? Ha?";
}
catch(Exception $e) {echo "I made an error. So what?";}
}

else echo "Empty input! Bad boy...!";
?>
<br><br>
<a href="dolUTubeFrame.html">Back</a>
</hmtl>


That's it! Happy downloading!

Perl by Example: English Dictionary In 22 Lines

[Update #1] This post has been cited from Princeton's WordNet Project as a nice Perl extension. You can find lots of more of information about WordNet here.

[Update #2] Many thanks to my brother Costas, for buying me the ultimate Perl book: "Programming Perl, 3rd edition" by Larry Wall. Thanx bro! :)

WordNet is an online English dictionary from the Princeton University. It has some hundreds of thousands of words and it keeps growing. You have absolute access to the resource and there are interfaces in practically all programming languages. It is not tough to build one from your own, but I ceize the opportunity to make an introduction to a really special programming language: Perl. Of course a crappy blog post cannot introduce a such extensive language as Perl, but additional Perl articles are sure to follow!

For the history, Perl was designed as a so-called 'glue' language for Unix, simply for doing things that with other tools were somewhat difficult. Created in the mid-80's by Larry Wall, in a one-man project as a fast reporting tool, it is now spread to all machines and applications. I have never used Perl for large GUI-demanding projects, but it has been a killer for simple but heavy tasks.

Whatever the objections, you may find Perl great for one simple but rare thing: Perl has culture. Its creator and all people actively involved in Perl's development share a number of values: Freedom, innovation and a unique sense of humor. To explain, I have two Perl books in my library: "Learning Perl" and "Programming Perl", affectionately called by the Perl community as the "Alpaca Book" and the "Camel Book" for the animals on the cover (Camel is the Perl mascot as shown in the figure). For me they are the most easy-read programming books that I have read. Anything else I use as a reference because I find programming books too hard to follow for long. But these differ. The style of writing and the humor that they transmit make you read them in literally one breath.

Anyway, the best way to introduce a language is by example, so here is mine for Perl: To build an English dictionary based on Princeton's WordNet. Believe it or not, all in all it takes 22 lines to do it! You may see the interface with the program in the following video. When you are done you might want to check the source code with some comments for better understanding.


The Perl source code follows:

my $word; #declares a scalar variable.Perl has two basic types: scalars and plurals(lists,hashes)
our $message = "\nPlease enter a word to look for:\n>";
print $message; #prints our message
while($word = <>) #reads the word the user searches for
{
chomp($word); #remove last character (the '\n', new line)
if($word eq "<") { system("cls"); #if user types "<" clear screen
print $message; }
else {
print "Connecting..";
my $url = "http://wordnet.princeton.edu/perl/webwn"."?"."s=".$word; #WordNet URL
`wget -q -Oindex.html $url`; #download the results
open FILE,"<","index.html"; #open the result web page locally
print "\rOk.Results:";
while(<FILE>) #for every line in file
{
if(/<\s*b>$word<\/b>[^\(]+\(([^\)]*)\)/) #match a pattern, don't be scared!
{ print "\n#def#$1"; } #..and print the word definition !
}
close FILE; #close file
print "\n>";
}
}


What might scare you off is the pattern matching line, especially if you have never come across regular expressions before. The logic is this: In WordNet result page, each word definition comes with original word in bold and the explanation in parentheses. For example if we search for 'developer' it may have the following format:
...developer...(a man who develops)...
...developer...(sometimes referring people creating software)...and so on
What the pattern matching does it to match if the line has the word we search for in bold and then extracts the sentence in parentheses.
When it comes to pattern matching, Perl is the king.

That's all. If you already have a Perl distribution, you can test it yourself. If you don't and you would like to give it try, for Windows there mainly two distributions. The one is from ActiveState and you can download a Windows Perl distribution named ActivePerl here (just click the download button). The other is called Strawberry Perl, it recently came out of beta and you can find it here. I haven't tried but it may well be good.

Needless to say, Perl is free and always will be! So ride the camel and enjoy!

Greece on Macedonia Dispute: "Quid pro Quo"

In the article, some weeks ago, "Macedonia: It's All In The Name", discussing why this issue is so vitally important to my country, Greece, Ι tried to briefly summarize the history of the naming dispute.

Today, the Minister of Foreign Affairs of Greece, Dora Bakoyannis, wrote an article for "The Independent", summarizing the Greek opinions over the Macedonia naming dispute, and practically arriving to some form of ultimatum: "We need to see something from FYROM or else no Greek government and no Greek parliament, will endorse our neighbors entering NATO or the EU." So, now Greece waits for the another proposal from the 'magician' Mr. Matthew Nimetz (photo,Nimic or Nimitz or whatever) or officially shuts down the doors.

Here are some key points of Bakoyannis's article. First there is the birth of the problem, beginning in 1944 when Tito renamed the area of Skopje, from Vardana Binovina to Republic of Macedonia. The plan was to slowly move to the Aegean Sea but now that United Yugoslavia is over, Skopje has failed to get rid of being part of this plan (or forgot?).

The article continues with depicting the heart of the problem and to paraphrase: "If you, FYROM, would like to have the term "Macedonia", you can do it. But you will do it because you like the name and not because you have any kind of relation with ancient or modern Macedonia. So, drop anything that distorts historical facts from textbooks, maps, articles and even from your constitution." That's all we care about. Allegations about "Macedonian minorities" (?) or "Aegean Macedonia" are absurd so forget them and also "United Macedonia" and any other such master plans. Greece does not deny that a small part of the Ancient Macedonian empire was lying in the Skopje territory. So keep a geographic qualification, like "New", "Upper", "North", "Whatever" and forget plain "Macedonia".

Bakoyannis goes on to explain the bonds of collaboration that Greece has already been creating. Greece is the prime investor in FYROM (almost a billion of dollars) and that shows our willingness to work peacefully and creatively with this new country. After some outlines over FYROM's unwillingness to move towards a solution, Bakoyannis closes with the phrase: "We cannot go any farther. As long as the problem persists we cannot and will not endorse FYROM joining NATO or the European Union. No Greek government will ever agree to it. No Greek parliament will ever approve it."

Well said.

That's all. We are all fed up with this story. Spending millions to buy an article in the "Independent", trying to support a crystal-clear thesis, is not fun. Greek politics has been extremely unsuccessful on this subject and our governments have made so many blunders so far, that we have managed to be considered the main problem of the dispute (well actually the present government has done nothing, so not blunders either..) Anyhow, we are all united now (unlike FYROM which is on a critical edge as far as 'density' is concerned). "No solution, no invitation" is the government's motto and we all sign for this. FYROM should take responsibilities. Negative results would mean isolation for both parts and that could 'kill us all'.

So, tomorrow it is the critical meeting in Bucharest and we are not delusional. Greece is playing against United States, not FYROM. And it is pretty risky to go against the US but we are more than willing to do so in that case. So our finger is on the 'Veto' button and more importantly, we are ready for the sanctions of pushing it.