Gödel's Incompleteness Theorems: Underground

This blog has moved to Steemit. Feel free to follow me there (https://steemit.com/@gargunzola).

Bayesian Analysis: To probably be, or probably not be, depending on something else's probability

That man of science, Urikalish, loves to torment us superstitious, intuitive mortals who adore the Secret and the Law of Attraction with logic. This is his latest riddle:

Urikalish's Riddle

Let’s assume there’s a deadly disease that affects 1 in every 10 people.

Let's say there’s a pretty good test for this disease which is 90% accurate (that means that if a person is sick – the test will say he’s sick in 90% of the cases, and if a person is healthy – the test will say he’s healthy in 90% of the cases).

Now, assuming you took this 90% accurate test, and got a positive result that implies that you are sick - what are the odds that you are really sick?

Bayes's Theorem

The key to solving this riddle is Bayes's Theorem. Don't ask me why. I'm an elitist and I don't need to explain myself to the likes of you. Especially when I have no idea what I'm talking about.

• P(A|B): This is the probability of A, given B. This is what you are looking for. This is the posterior or revised probability that you have the disease, given the fact that you test positive.


• P(B|A): This is the conditional probability, or the probability of B given A. In other words, the probability that you test positive, given that you have the disease.


• P(A): This is the prior probability of A, or the prior probability that you have the disease, regardless of whether your test is positive or not.


• P(B): This is the prior probability of B, or the prior probability that you test positive, whether you have the disease or not.

Smart people can fill in the data directly from Urikalish's riddle into Bayes's Theorem and get a result. Unfortunately, I played more with the glue and crayons than with the abacus and the clay when I was a kid, so I need something simpler.

An Intuitive Explanation of Bayes's Theorem

Eliezer S. Yudkowsky, that boy genius who tells us about his AI (that is, Almost Implemented) research, has the remarkable ability to make some really, really smart stuff seem really, really simple. This is Yudkowsky's explanation of Bayes's Theorem, which still went over my head (must've been toxic glue), so I tried something that is even more simple and more familiar to me, namely decision trees.

Using Decision Trees in Bayesian Analysis

Start at the beginning and make a split for those who have the disease, and those who are healthy. Branching of each node of the tree has to add up to 1, otherwise you're cooking Dutch books. Proceed up the tree, splitting each node into all the possible scenarios, and you'll eventually cover your entire sample space and thus have a visual representation of your full joint distribution.

Explanation

The probability of having the disease is equal to the amount of people who have the disease and test positive, divided by the total number of people who test positive.

If you said 90%, you are stating the probability that the test is right. This is a different probability than having the disease. If you said 10%, you are stating the probability that anybody has the disease, regardless of taking the test or not. The probability of having the disease is affected by both the probability that the test is right and the rarity of the disease.

The total number of people who have the disease here is 0,1, which is 10 %, or 10 out of the 100. The total number of people who test positive is 0,1 x 0,9 for those who have the disease (or 9 out of the original 100), and 0,9 x 0,1 of those who do not have the disease (also 9 out of the original 100). This means the total amount of people who test positive in the original population is 18 out of 100.

Finally, your answer 0,5 ie 50%. You get this from the 9 who test positive and do have the disease divided by 18 who test positive in the total population. The decision tree makes it visible who tested positive, so you are less likely to get confused. How likely? I don't know, draw your own damn tree.

Problem Solving with Darwin

Genetic Algorithms are a way of solving problems by mimicking the same processes mother nature uses. They use the same combination of selection, recombination and mutation to evolve a solution to a problem.
[AI Junkie Genetic Algorithm Tutorial]

Claudia Schiffer has still not shown herself. I'm beginning to believe that the Law of Attraction is nothing but a farce. They certainly had me fooled, but as George Bush once said: "There's an old saying in Tennessee — I know it's in Texas, probably in Tennessee — that says: Fool me once, shame on — shame on you. Fool me — you can't get fooled again." Something about the shoe being on the other foot in your mouth now.


George Bush. Living proof that mutation is random and we're definitely not intelligently designed.

If the Law of Attraction is not a good problem solving strategy - and that's a big if, I mean the Law of Attraction seems so plausible, especially when you critically evaluate its claims - then what is? This depends largely on the kind of problems you want to solve.

What kind of problems can you solve with the Law of Attraction?

I'm convinced that the Law of Attraction is not a solution, but a symptom of far greater problems bubbling below the surface. Regardless, most people who employ the Law of Attraction wrestle with questions posed by an existential crisis, unrequited love, dashed dreams, shattered hopes, a quarter-life crisis, a mid-life crisis or the ponies they didn't get for Christmas. While ponies posing questions, or a grown man wrestling with a pony might be amusing to some (especially those who produce or purchase snuff films), humorous charades are not part of the solution I had in mind.

These problems have some of the following characteristics in common:

• The search space is incredibly large, clearly misunderstood or overwhelmingly complex.


• It is difficult to employ expert knowledge to narrow the search space. Dr Phil is not going to help beyond asking you how you feel about it.


• A mathematical analysis is unavailable. Q: What should I study? A: 42. Doesn't make sense, does it?


• More traditional search methods for solutions to the problems have failed.

Fortunately, mother nature has already solved problems of this kind for millions of years. Darwin described some of her methods accurately and the field of computer science took his description and encoded it as a simple algorithm that helps to solve problems of this nature.

A Genetic Algorithm in Pseudocode

1. Generate an initial population.


2. Determine the fitness of each candidate in the population.


3. Wash, rinse and repeat these steps until a sufficient solution is found:
   
   1. Select the candidates with the highest rating for reproduction.
   
   
   2. Breed a new generation through crossover and mutation.
   
   
   3. Discard the worst ranking members of the population.
   
   

That might seem like Geek to you, and in fact it is. Rest assured, I will help cast light on the subject by giving you a walkthrough illustrating how you can use mother nature's own problem solving algorithm to solve your problems. Not all your problems. That rash is not going to go away with this. Sorry.

A Walkthrough to Illustrate Problem Solving with Darwin

Given a perplexing problem: What should you study?

1. Generate an initial population. In this instance, you can write down a list of fields you have ever considered in your life. If you can't think of twenty, get a brochure from a tertiary institute and randomly write down a list of courses from it. You can pick any number, but don't make the initial number too small.


2. Determine the fitness of each candidate in the population. The way you determine the fitness of each course is up to you. It is however important to attach a numerical value to each course. You can give each course a rating on a scale of one to ten, for instance.


3. Repeat these steps until you have only one item left on your list:
   
   1. Select the candidates with the highest rating for reproduction. This means you pick all the ones closest to ten for breeding.
   
   
   2. Breed a new generation through crossover and mutation. Crossover means you take properties of one course and combine it with properties of another course and see if there is such a course in the brochure. You'd rather take that course than the other two and give it a higher rating (plus one, unless it is already ten, then its rating just stays at ten).
      
      Mutation means you alter a course slightly. Suggest you had biochemistry as a course. Now you slightly alter it to become anatomy and physiology. Of course, this is subjective. If you'd still prefer to rather do biochemistry than anatomy and physiology, then rather keep biochemistry and discard the mutation. That's one advantage you have over mother nature: you can design your population intelligently.
      
      How do you choose when to mutate and when to crossover? This is also subjective. You could mutate every third course on your list, unless it is one you already feel strongly attached to, then just mutate the one immediately after it. You could crossover every item divisible by four and its immediate predecessor. You could decide to only mutate one round, and only crossover the next round. You can be as random as you like - it works for mother nature - but don't alter your population too much during each round. Mother nature doesn't condone bootstrapping.
   
   
   3. Discard the worst ranking members of the population. In our example, just delete one item from the list after each round. If you arrive at a scenario where all the lowest ranking items have a similar number, pick the course you like the least and delete it. If you can't decide, flip a coin.
   
   

Does This Really Work?

Surprisingly, yes it does. The list of applications for genetic algorithms on Wikipedia contains among others:

• Training artificial neural networks when pre-classified training examples are not readily obtainable


• Scheduling applications, including job-shop scheduling.


• Selection of optimal mathematical model to describe biological systems.


• Plant floor layout.


• Optimisation of data compression systems, for example using wavelets.


• Linguistic analysis.


• Electronic circuit design.


• Container loading optimization.


• Automated design of sophisticated trading systems in the financial sector.


• Automated design of industrial equipment.

Some Useful Links

• A field guide to Genetic Programming. A free e-book for the real geeks.


• Genetic Algorithms. A very broad introduction to the discipline.


• The Wikipedia page on Genetic Algorithms.

Law of Attraction

The Law of Attraction is the kind of snake oil that sells to those who somehow feel absolved when they share ineptitude with others. The most recent phenomenon of this rudder in the calm waters of clear thought resulted from what must be the first instance of a movie that turned into a best-selling book instead of the other way around.


Claudia Schiffer. Come to daddy!

The Law of Attraction

In four easy steps, you can be right back where you started.

1. Know what you want. Women, are you reading this? OK, let's put this theory to the test. I want Claudia Schiffer.


2. Ask the universe for it. Suggest then that the waves of my thought patterns and the sound waves of my spoken voice are likely to be heard over the cosmic microwave background radation. Suggest also that these waves can travel through the theoretically empty vacuum that they would meet in space. Suggest also that Uranus won't block my signals. Suggest also that the Universe does understand English because I sure as hell am not going to Lisp the bitch. Suggest also that Hawking radiation, the music of the spheres and the noise transmitted from the earth's magnetic field (which swallows my brainwaves as soon as they leave my skull) do not prevent something as abstract as the Universe to hear about my inconsequential but by no means whimsical, unrequited affections for Claudia Schiffer. Suggest also that the Universe has the mother of all Beowulf clusters running, because surely there are hungry kids in Ethiopia, war victims in Iraq, cholera sufferers in Zimbabwe and Madonna who Universe knows needs help more than I do but because of the Law of Attraction all of us need to be served and soon. Especially Madge, she doesn't have that much time left. Suggest also that conflicting requests do not cancel each other out but somehow manage to materialise in tandem by means of a Mary Poppins kind of magic. Then, Dear Universe, may I please have me some Claudia Schiffer?


3. Behave as if the object of your desire is on its way. Yes, Claudia Schiffer is going to knock on my door at any minute. Any minute now. I'm wearing a tuxedo. I look more stylish than Vincent Price and because Claudia hasn't pitched up yet, I'll have to sleep in my tux. I've chained myself to my bed with an elaborate pulley system so that I would not risk creasing my tux while lying down or getting up. Must look my best for Claudia, because she's going to knock on my door any minute now. My friends invited me out for drinks and dayglo paint filled condom fights, but I can't go in case Claudia shows up. Not even the shenanigans of viking mead or meatshank the bunny to the revolving bed by lava lamp light are going to cause me to defer from behaving as if Claudia is going to knock on my door at any minute. Even if I were invited to live on my own in the Playboy Mansion, with delicately hand picked Playmates who tickle my fancy and fancy my tickle, will I even consider behaving in any other way. Right.


4. Be open to receiving it. I'm very open. If Claudia walked through my door right now, I'd give her my wallet and my keys and all the sweet petty mutterings I can muster in my broken German and a hushed tone. Law of Attraction or not.

In case you haven't noticed, I'm not entirely convinced that the Law of Attraction is a very practical problem solving approach. Let's demonstrate why by means of a simple thought experiment.

A simple thought experiment to break the Law of Attraction

Imagine that you get to choose the captain for an exhilarating new passenger liner called the RMS Titanic. You have a choice between captain Jack Sparrow and captain Wallace D Wattles. Jack Sparrow has had excruciating and extensive navy training under the Royal Navy. He's a sea dog, through and through. Wallace D Wattles sat in his room all day asking the universe to be a captain, believing that he'd be a captain and being open to receiving the position of captain on your ship. Wallace has never set his foot on a ship and gets seasick on a skateboard. Who would you choose to command the Titanic? Well, obviously you chose Wallace because your ship sank.