Half Truths

Book Review

THE DRUNKARD’S WALK-How Randomness Rules Our Lives-By Leonard Mlodinow

The doctor told me I had a deviated nasal septum and had to get the same operated. He asked me to get all the possible blood tests done, including tests for Hepatitis and AIDS. And though I knew my chances of both were almost zero, I could not stop worrying a couple of times before the report. What if I had tested positive? What are the chances that I will be tested positive for AIDS when I don’t have the virus (also known as False Positives)? Is there a difference between the chances that I would test positive if I was not HIV positive and the chances that I would not be HIV positive if I tested positive? What is the role of false positives in the world of medicine?

Well, it so happens that knowledge of conditional probability will tell us that the chances that someone does not have HIV Virus if he tested positive and the chances that someone tested positive even thought he did not have the virus are different. The author explains this in details in the chapter -False Positive and Positive Fallacies.

I had promised that I would shortly write a review on the book- The Drunkard’s Work.  The epilogue is so interesting that I was immediately hooked to the book. The author narrates the story of a man who won the Spanish lottery-

A few years ago a man won the Spanish national lottery with a ticket that ended in the number 48. Proud of his “accomplishment,” he revealed the theory that brought him the riches. I dreamed of the number 7 for seven straight nights, he said, “and 7 times 7 is 48.” Those of us with a better command of our multiplication tables might chuckle at the man’s error, but we all create our own view of the world and then employ it to filter and process our perceptions, extracting meaning from the ocean of data that washes over us in daily life. And we often make errors that, though less obvious, are just as significant as his.

The Drunkards book is a fascinating account of how randomness rules our lives and how often we interpret the random events erroneously. We look at the world around us, we filter a lot of data and interpret what is happening around based on our intuition and we come up with certain results. And the results are often wrong because when it comes to questions involving uncertainly and randomness we often misinprepret the world around us. We have our own directions and goals but we are also bombarded continuously with unpredictable and uncontrollable events that have a very great influence in the direction we take. For those of us who have seen a pollen grain in water moving in a zigzag fashion, there might not be an apparent direction in its movement, yet over a period of time it goes from Point A to point B. Hence while at hindsight we might think that the pollen moved from point A to point B as part of a deterministic trajectory (on purpose), it went in a totally random fashion. So even though the movement of the pollen grain looked totally random with no apparent preferential direction it did actually over longer period of tome move in a direction. Thus extrapolating it to life, even if we have no direction we will get somewhere.

The last line requires more elaboration as it an important thought which author builds on. He talks about our biases and illusions arising from randomness and the illusion of causality and the law of small numbers (there is actually a law of large number; law of small number denotes to wrong use of the law of large numbers for smaller numbers) are important ones. We often attribute more importance to success and failure than it is due. In a span of 5 years we judge a CEOs performance when mathematically speaking the probability that a CEO with a certain success rate will demonstrate that success rate in a 5 year period is only 1in 3. Also the author questions the fact that how come someone treated as a genius business manager change overnight into a dumb one.  The book is interspersed with calculations on why using small numbers for prediction may be fallacious and how certain random processes are viewed erroneously.

The author’s objective is not to declare everything as random but to tell us that the usual attributes of success and failure, of genius and mediocrity are premature and not totally a matter of effort but also have an element of random luck thrown in. Given that there is no denying the role of effort. Even JK Rowling got rejected for her novel 9 times before she became a world renowned author. The authors quotes a former IBM executive, Thomas J Watson -If you want to succeed double your failure rate.

The latter chapters of the book also look into the psychological reasons of why we make mistakes around randomness. Borrowing from some of the experiments of Nobel laureate Daniel Kahneman and Tversky he points out how for human beings being in control (what the author says as illusion of control) is an important need and hence we don’t like to believe that we don’t have control over life and environment.

The book is filled with interesting life accounts of mathematicians who have contributed to the development of probability (you will have to read the book to find out if all of them were mathematicians or not), interesting problems in probability (Monty Hall problem) and statistical concepts like regression towards the mean (which also has implications in Biology).

For me the book is a crash course on probability, statistics, psychology and randomness and how they affect our daily lives.

Prithwish