The Most Influential Person On Computer Science

Who was the most influential person on computer science (or computing if you prefer)? A web search is not particularly helpful. Most of the times, the pages I got were confusing CS with programming. However, to quote Edsger Dijkstra, "Computer Science is no more about computers, than astronomy is about telescopes". Some names may come quickly to

mind: Alan Turing, Church, Kleene etc, were all pivotal to the development of the computing science. But a deeper search can reveal the person who had the most critical impact: David Hilbert.

Hilbert was a German mathematician who worked on an incredible range of subjects: number theory, logic, geometry and over to special relativity theory. Yet, it was not his direct contribution that matters, but the influence he had to others on working on the fundamentals of computation. And it all started by his zealous stance towards a deep philosophical debate of the "ignoramus et ignorabimus".

And now the story begins.

Back in 1872 Emil du-Bois Reymond was a well-respected physiologist with considerable work on electro-physiology, studying the electric "properties" of human and animal tissues. At this year, he publishes a paper in which he claims his philosophical belief of "ignoramus et ignorabimus" (=we don't know and we won't know). This was a slightly more pessimistic version of the well known Socratic view "I know one thing; that I know nothing" but it was gaining acceptance through the philosophical circles at the time. The main point was that there are some stuff that humans do not know and they will never know (e.g. the origin of life or the universe) One might agree that it seems like a yet-another harmless philosophical idea. Some people were supporting it, others would be skeptical, but all in all, life would go on. But science was meant to take a dramatic turn because a man would take this debate to the extreme.

At that time David Hilbert was 10 years old. In subsequent years, through his work, he would confront deep philosophical problems, even within mathematics. However, the "ignorabimus" movement had really hit his nerve. In 1900, in the International Mathematics Conference in Paris, he addresses a call to arms to mathematicians by claiming: "in mathematics there is no ignrorabimus!". He uses mathematics to exterminate once and for all the pessimistic view, and claims that there should be a finite process of inference rules, that when applied, they would infer any correct statement. He then asks from mathematicians to help him in this direction (the 2nd problem in the problem list he announces at this speech was related)

Almost 28 years later, in 1928, Hilbert poses yet another challenge: The Entscheidungsproblem asks specifically for an algorithm that will be able to decide if a specific statement (e.g. 1+1=2) is true or not. During the 30's great things just start happening. In 1931, Kurt Godel breaks the first problem with his ground-breaking theorem of incompleteness. In short, most interesting systems, can either infer contradictory statements or they cannot infer some correct statements. Later, in 1936 and 1937 Church and Turing independently solve the last puzzle by proving that there are statements that cannot be proved or disproved. Hilbert's dream was shuttered but a new scientific field was born out of the proofs of these very problems! It is important to note here that, 5 years before his proof, Alonzo Church was in the famous Gottingen University during the years 1929-1931, visiting...David Hilbert.

But wait, there is more! It was not only theoretical computer science that was catalyzed by the influence of David Hilbert. The creation of the first general-purpose electronic computers would be pioneered by his assistant. Even in 1942, one year before the ENIAC project, Hilbert's former assistant in the University of Gottingen, would start an ambitious project of creating a general-purpose computer in the Institute of Advanced Studies at Princeton University, that ended in 1952. His work all these years marked the beginning of this era and his architecture is preserved more or less even in today's computers. His name was ...John Von Neumann!

Other than Hilbert, who else can claim of having a bigger impact on computer science?

If Philosophers Were Programmers

Although not obvious, philosophy actually has a strong relation with programming, at least for me. If you think about it, software code reflects much of how the developer perceives the problem and its solution. Before starting to program, developers spend some time thinking over the problem, identifying important properties and their underlying connections, a process that reveals their philosophy as the way they perceive real-word situations. Likewise, philosophers are constantly trying to identify the most important properties of the issues they reflect on, like life, conscience or God.

Under this perspective one might be able to make a consistent mapping of the ideas behind programming languages and the ideas that philosophers have come up over the years. It is perfectly reasonable to consider the programming languages as the different philosophies of a virtual world, in which entities do exist and interact with each other. To this respect, even the fundamental philosophical questions receive an interesting transformation: For example "What is self-conscience?" can be rephrased as "What is reflection?".

To the fun part, one might ask: "What if philosophers were programmers? What programming language they would use?". Well, here are my answers!


Socrates : The Hardcore Assembly Programmer
Socrates was one of the founders of philosophy but this is not where the connection ends. Socrates had devised a clever methodology to win every debate. He kept asking questions until a contradiction was reached. So, when someone would claim "morality is important", Socrates would ask "How do you define morality?".


In a similar manner, everything in Assembly begs for a question. There is nothing pre-assumed (at least in pure Assembly, not the distros filled with pre-processed libraries and other junk) and everything has to be as succinct as possible to have a meaning. If you were to work with the programmer Socrates and shared something like "var x = null;", your partner would start by asking "What is var?" !



Aristotle : The Influential C Programmer
Aristotle had a huge impact on Western philosophy, founding many scientific areas, from physics to biology. He was the first to closely examine real entities as the real essence of everything, in contrast to Plato's abstractions. His philosophy is driven by the golden mean as the key to reaching morality or understanding life (matter and form).

The C programming language was equally influential to the design of all other "programming philosophies", most obviously in the syntactical level. In addition, by the time of its writing in the early 70's, C was supposed to be the golden mean between the so-called high-level languages and the Assembly language, combining the capability to write machine-independent code combined with the power of low-level access.



Plato : The Idealistic C++ Evangelist
Plato is a huge figure in philosophy, student of Socrates and teacher of Aristotle. That said, I owe you an explanation about the obvious anomaly: How come that C++ is coming after C? Let me explain. Plato is famous for his Forms or Ideas, that refer to the archetypical versions of the things around us. So, the cup in your desk has is a shadow of a similar oval-shaped archetype in the world of Ideas. In programming words, it is an instance of the Cup class.

Similarly, C++ , as an extension of C, is the first language that tries to capture this idea of forms by giving the developers the capability to abstract the problem before doing anything else. This is a major step by itself, since even if no actual code solving the problem is provided, the classification and the problem modelling are evident and valuable to others. You might wonder, why Plato would not program in Java. Well he could, but there is another parameter to the story: Plato is not so confident how symbols can represent his Forms, and clearly prefers the spoken dialogue (as mentioned in Phaedrus). In a similar manner, C++, not entirely confident in its direction, remains a superset of C, being fully backwards-compatible with the more non-ideal syntax of C.



Stoics : The Happy Perl Community
Stoics and their philosophy (Stoicism) had silently, a far-reaching impact not only to Western philosophy but to the philosophy and the global culture as a whole. Interestingly enough, there is no single man behind it, but it was actually a collaborative intellectual achievement. Stoicism denies anything immaterial and tries to explain the world through propositional logic. So, Stoics reject everything Ideal and concenrate in morality, in which they call us to get free from anything we can't control, but rather appreciate the freedom to self-introspect and reach true wisdom. Stoicism rejects political systems and other formalities, and promotes Socrates' citizen of the world for everyone. People are meant to be brothers, away from distinctions, aiming to contribute happily to a society of friendship and love (jus commune gentium). You should already notice the influences to most widespread religions, like Christianism and Buddism.

Most interestingly, Perl was created in the 80's, a decade in which finally logic/functional programming had found its place in the programming languages world. However, the Perl community (and language) shares much more striking similarites with the Stoics and their philosophy. Perl as a language is to the best possible extent, free of form. Actually the most common phrase in the Perl world is "there is more than one way to do it" or TIMTOADY for short. The philosophy behind Perl rejects syntactical constraints, giving the freedom to its programmers to code in their style, but at the same time encouraging sharing and contribution to the community. Perl's power lies to a great extent to the existence of CPAN, the archive of modules and software happily shared by Perl programmers all around the globe. The language's influence to the programming world has been silent, but much more far-reaching than what is immediately observable. One could mention its strong influence to scripting, dynamic typing and functional programming, but it could be summarized to a joke which is familiar to Perl fans: The next market's crash will be triggered by a bug in someone's Perl script.



Rene Descartes : The True Java Guru
Descartes was the first philosopher of the Western culture to stand up against the Classical Ancient Greek philosophy. His core philosophy as mentioned in his famous Article 7 of the "Les principles de la philosophie" is based on the concept of cogito (=intellectual ego). Descartes believes that doubt is a proof of existence, and cogito is the cause of doubt, arriving to the famous "cogito ergo sum" (=I think therefore I exist). The cogito is not just another process we do, but actually all we do. So, what we want, imagine or feel is directly accessible through it. Descartes nearly 'proves' the existence of God, by the fact that we are able to think about the necessity of his existence. In fact any Ideal or Form can be directly accessible by our cogito. Descartes also marks another landmark in the history of philosophy: Beginning from his work, philosophy is trying to avoid confusing abstractions and to establish a succinct, almost geometrical form. Descartes presents his ideas nearly in the form of theorems.

Descartes would be the perfect Java guru. Java was the first strongly-typed language, in which everything must have a type (or share a Form) before it is being used, matching perfectly the Descartes' efforts to be always exact for what he is talking about. Descarte's cogito is in fact a revisit of Plato's Forms, with a slight variation in which ideals exist because we think about them and not in another universe. To that respect, his philosophy is purely object-oriented, as the solutions in which we arrive, are direct products of our intellects.




Immanuel Kant: The First Python Programmer
Kant found the 'easy' way to the pantheon of philosophy by rejecting two prevailing and opposing methodologies, Descarte's cogito and the empiricism, by shouting 'It's both!'. Kant investigated how humans reason, claiming that experience offers the truth, but which has already been filtered by intellectual judgement (a priori). At his mature years, he examined aesthetics, and the theory trying to explaining the way we perceive beauty. Kant was an extremely concise personality, being obsessed with tideness and exactness, doing the same things, exactly at the same time every day, to the extent that his acquaintances were 'using' him to calculate time!

Similarly, Python is a programming language that tries to combine different solutions and promote it as a new one. As a language it accepts multiple programming paradigms, from object-oriented to contract-based programming. Python programmers reject the free formats of languages such as Perl, and although they borrow several features from it, they emphasize on simple and explicit code. Python becomes so 'obsessive' that imposes whitespace identation as delimiters for code blocks to its users. In the "Zen of Python", the first out of the 19 commandments, the first one is "Beatiful is better than ugly". Kant's obsession to beauty and aesthetics, makes him triumphantly the first Python programmer ever.



Ludwig Wittgenstein: Natural Born Haskell Programmer
Wittgenstein reformed Western philosophy going as deep as to examine Socrates' 'recipe' for debate success. His monumental work, the Tractatus Logico-Philosophicus, can be compared to a hard graduate mathematical book in Logic. Wittgenstein identifies the semantic and symbolic forms as the root of all philosophical problems, leaving the rest that can be explicitly defined as the subject of science. Using pure logic, he deducts that language inherent ambiguity is what makes philosophy repeat itself, and closing his book with the famous 'What we cannot speak of, we must pass over in silence', claims to have solved,..., all philosophical problems.

Wittgenstein is a natural born Haskell programmer. Haskell was not the first functional programming language in town, but from late 80's and onwards, it has prevailed as the most important among the group. Haskell is not meant to be accesible by anyone, and just like the austere and succinct Tractatus, as Wikipedia states, it has a strict mathematical and logical form. Haskell, being purely functional, goes as deep as redefining the way we treat abstract data types, the same way Wittgenstein goes back to Socrates' dialectic to reform modern philosophy.


These all may sound weird, but for programmers, it is easy to realize these deeper connections. I am not quite sure if the same holds for philosophers. Anyway, at least by now, it should make much more sense why in every article in Wikipedia, presenting a programming language, there is special section named "Language philosophy".

Donald Knuth And The Complexity Of Songs

Donald Knuth is a living legend among computer scientists. His monumental work-of-life "The Art Of Computer Programming" (I was never able to fully follow it) is a standard reference/textbook/work of art for computer science. Mr. Knuth took a new, immature geek-only hobby and transformed it into a solid and complete scientific discipline.

Among his contributions, the systematic analysis of the complexity of algorithms, really stands out. As a concept complexity is very familiar to developers. It refers to how a specific algorithm or algorithmic solution to a problem scale to the 'problem size'.

If his work is a reason to admire him, his humor is a reason to love him. Recently, I had the most splendid time reading his hilarious paper titled "The Complexity Of Songs" ! Here is the story.

A song has some lyrics which we have to remember in order to sing it. Humans are trying to learn lyrics of length s (space complexity) to sing a song of length n. You would expect mathematically that s ~ n, but Knuth investigates all human 'inventions' to reduce the song space complexity!!

After a series of 'theorems', Knuth proves the existence of songs with s= a.n, a<1, s="O(sqrt(n))," s="O(logn)">and finally.....s=O(1)!! Yes, you read well. 

Knuth argues the first step was the invention of the refrain, the repeated part of a song. If the song has m verses of length V and the refrain has length R, then the song length is roughly n = V.m+R.m and the complexity s = m.V+R. Hence we have a reduction of V/V+R in song complexity. So, a refrain is also a tool to save some memory space to remember the song!

Remember the old song "Old Mc'Donald had a farm, Ei-gh,Ei-gh,Oh!" ? Well this has a complexity s = O(sqrt(n)). It is much like the Greek song "Οταν θα πας κυρα μου στο παζαρι...". In this pattern, each verse includes all previous verses. For m verses n = o(m^2) and s = O(m), so s = O(sqrt(n)). This means that it is much easier to remember! That's why they are most suitable for children! A same pattern can yield a log complexity.

Now to the best part, Theorem 2 is the real killer! The introductory text goes as follows:"However, the advent of modern drugs has led to demands for still less memory, and the ultimate improvement of Theorem 1 has consequently just been announced:

Theorem 2 (Donald Knuth)
There exist arbitrarily long song of complexity O(1)

PROOF. Define U = 'uh huh','uh huh' and the k-th verse Vk = 'That's the way', U, 'I like it', U . This is a constant V. Then the song V^k, completes our proof!! Oh dear!

This last one really cracked me up!!!   :)

Catalan Numbers, Dyck Words and Robots Climbing Stairs

Repeating is the very nature of recursion, so this article continues over from the previous post over recursion and stack usage. If you don't remember the initial problem here it is again:

"Imagine a robot(say Asimo) that climbs up stairs and has one very simple function step() which succeeds with a probability p and climbs up one stair, and fails with a probability q and climbs down one stair, with p>q and p+q=1. Write a function step_up() guaranteeing that the robot will climb up exactly one stair."

It is easy to create a quick and dirty recursive solution:

void step_up() {

if(!step())
{
step_up();
step_up();
}

}

While this solution is not the best (a better one is provided also in the previous post with the while version), it is most interesting to calculate the number of times that step_up() will be called, in other words to calculate how large the stack size will be.

There were two solutions available: To calculate it analytically or use a recursive technique to do it. With the recursive technique we can set: If X is the number of average calls to step_up() then X = p.1+q.(1+2.X) which yields X = 1 / (p-q) . Both easy and really smart!

However, going analytically is much more difficult but it is most amusing! As I said in the previous post, going the analytical way would mean trouble. Trying it myself I left it unfinished when facing a difficult combinatorics sub-problem. By a really weird coincidence, today I stumbled upon a number of interesting combinatorics tools which could help solve it analytically once and for all!. Here is how it goes.

First, it is clear that is impossible that the step_up() function will be executed an even number of times, for each failed execution two more executions occur. We should wonder: What is the probability that the function will enter 1 time? Clearly it is p. What is the probability of entering 3 times? This would mean failing the first time and succeeding the other 2 times, a total probability of q.p.p = q.p^2. If we want to calculate how many times the function will be executed 5 times we will have to count all possible 'combinations'. We will follow the encoding X1-X2-X3-...-Xn , if we would like to denote a case where the function is executed n times where Xi is either F or S, meaning failure or success at step i respectively.

With this notation we have:
step_up() enters 1 time: Cases: S Combinations:1
step_up() enters 3 times: Cases: F-S-S Combinations:1
step_up() enters 5 times: Cases: F-F-S-S-S, F-S-F-S-S Combinations: 2
step_up() enters 7 times:
Cases: F-F-F-S-S-S-S, F-F-S-F-S-S-S, F-F-S-S-F-S-S, F-S-F-F-S-S-S, F-S-F-S-F-S-S
Combinations: 5

Stop here. There is one important property of every X-X-X-X.. string with n elements, which is that for any initial substring the number of S's is not greater than of the F's. Why? Because if this was the case , the robot would climb up one stair sooner than n steps! Cool! Now, one more thing: It is easy to establish that the last element of the every string will be S, which actually means that the last step of the robot will be to the upwards direction. Ok, this is clear. So let's formulate again the problem.

We have a string with two possible letters, F and S, of length 2.n, in which we have n F's and n S's and for every initial substring the number of S's does not exceed the number of F's. Now we want to calculate the number of all valid combinations for this case. But how to do that? Well, here comes the so-called Dyck words because what we described previously is what exactly how a Dyck word is defined! Now, there is this theorem that states that there exactly Cn Dyck words of length 2.n. What is Cn? But the Catalan numbers of course!

What are the Catalan numbers?? There are defined with the following equation

and they occur very often in combinatorics problems. In our problem they calculate all possible 'configurations' in which the step_up() function will be called exactly (2.n+1) times (Remember that the last step is always up, this means the last call is always successful). This means that the average calls of step_up() is calculated by, Now the generating function of the Catalan numbers in the following equation

is given by the equation

Using these equations in combination with the derivative of c(x) for the case with the n coefficient it is easy to arrive to to X = 1 / (p-q)

Wow! Recursion, probabilities, encoding, Dyck words, Catalan numbers, generating functions, all combined to arrive analytically to the complexity of the recursive function step_up()!! Now, what you will prefer, depends on you: Either an elegant and smart solution or a rigorous, brute but educating analytical approach! Your call!

Why The Array Index Should Start From 0

I recently came across an article from Dijkstra(photo) explaining a fact that looked obvious to me before: Why should start indexing an array from 0. I was amused to find out that there is some clever thinking behind this.

All popular languages, like C/C++, Java or Perl start indexing an array from 0 while the last index is the array length minus 1. While this is usual to most developers, it is not a de facto situation for all programming languages. For example in Fortran, when you declare an array with 'integer a(10)', aka an int array having 10 elements, the index starts from 1 and ends at 10 (however you can override this behavior using a statement like, integer a(0:9), declaring an array with indices from 0 to 9, but anyway).

The discussion over why the array index should start at zero is not a trivial one and relates to interesting concepts from computer science. First of all, it has strong relation to language design. For example in C, the name of an array is essentially a pointer, a reference to a memory location, and so the expression array[n] refers to a memory location n-elements away from the starting element. This means that the index is used as an offset. The first element of the array is exactly contained in the memory location that array refers (0 elements away), so it should be denoted as array[0]. Most programming languages have been designed this way, so indexing from 0 is pretty much inherent to the language.

However, Dijkstra explains why we should index from 0. This is a problem on how to denote a subsequence of natural numbers, say for example 1,2,3,...,10. We have four solutions available:
a. 0<i<11
b. 1<=i<11
c. 0<i<=10
d. 1<=i<=10

Dijkstra argues that the proper notation should be able to denote naturally the two following cases:
1. The subsequence includes the smallest natural number, 0
2. The subsequence is empty

Requirement 1. leaves out a. and c. since they would have the form -1<i which uses a number not lying in the natural number set (Dijkstra says this is ugly). So we are left with b. and d. Now requirement 2. leaves out d. since for a set including 0 that is shrunk to the empty one, d. takes the form 0<=i<=-1, which is a little...well, messed up! Subtracting the ranges in b. we also get the sequence length, which is another plus. Hence we are left with b. which is by far the most widely used notation in programming now.

Now you know. So, remember and take pride in the fact that each time you write something like

for(i=0;i<N;i++)
{
sum+= a[i];
}
you are not just following the rules of language notation. You are also promoting mathematical beauty!