# Information Theory: 400 years in the making.

**Posted:**February 6, 2011

**Filed under:**Information Theory 1 Comment

Mathematics is a language. We use it to describe and quantify things. Our first exposure to the language is when we learn to describe counts of things: one apple, two cats, three dogs, etc. Later in life, we use Mathematics innocuously: when we order a pizza, we order a certain diameter – 16″. Our sub-concious mathematician, visualizes the area of the pizza as π*(16″/2)². It then splits the pizza eight-ways and figures out that we probably need another large pizza to feed the guests on game night. In our day-to-day lives, this deductive language is never spoken except when it renders a result (we need another pizza) or succinctly describes an event (a 16″ pizza). We are out of practice when it comes to communicating with each other using our innate mathematical language. And so like a student that learns French grammar for a year and is dropped in Paris, a student with less than a year of college calculus finds himself incapable of communicating more than his name and awkwardly revealing how English is his first (and only) language when dropped into a graduate Maths course. Read the rest of this entry »

# I do hate sums

**Posted:**February 1, 2011

**Filed under:**Data Analysis Leave a comment

I do hate sums.

There is no greater mistake than to call arithmetic an exact science.

There are … hidden laws of number which it requires a mind like mine to perceive.

For instance, if you add a sum from the bottom up, and then again from the top down, the result is always different.”

— Mrs. La Touche, Letters of a Noble Woman

The abacus is perhaps the oldest computational device humans used. Its primary purpose is to aid in addition. Computational devices at our disposal evolved from the first abacus (circa 2400 BC) to warehouses of thousands of computers capable of performing complex computations, simulations and data analysis. The most important operation performed by any computational device, however, is still addition. It is hard to find an operation more central to computing than repeated addition or summation. Sums are everywhere. No mean, no variance, no model can be computed without summing up some data points. When approximating irrational numbers like π or integrals of functions, we use summations. Despite our technological advances, we haven’t created an error-free summation device. The reason for this deficiency is that computers are incapable of dealing with the continuity of real numbers. They live in a discrete world, while numbers live in a continuous one. I will describe the sources of error in a single addition, how these errors add-up with summation and how to compensate for these errors. Read the rest of this entry »