This is my first attempt at adding some of the Notability maths into the blog - we’ll see how it goes.
As I mentioned in my previous post I am starting to chronicle my re-learnings in probability and statistics starting with the textbook “Probability, Random Variables and Stochastic Processes” by Papoulis and Pillai. The writing is pretty engaging for a math book and I’m hope to make it at least a chapter before getting distracted. Joe Norman incorporated parts of the book in his Applied Complexity Science course I took recently and I grabbed it off Amazon where I was pleasantly surprised to find a positive review from Taleb.
On to the learning. It’s late here so I’ll start from the beginning and maybe get through some thoughts on a few ideas/equations.
We start with an intro to probability and I think one of the key takeaways is that probability is dealing specifically with mass phenomena. How many events? Well, it depends. More on this later I suppose.
A physical interpretation of probability is provided by the following equation (Notability works great btw):
Where P means probability, A stands for an event, n(A) is the number of times the event occurred and n is the number of experiments run.
In a classic example of flipping a coin, if you flip the coin 10 times and you get heads 4 times out of the 10, then your probability P(A) of getting head (funny undergrad story on this I might tell later) is 4/10 or 0.4.
Easy enough... but is it?
This equation is an approximation of the probability and has a couple key assumptions baked in (assumptions to me are like kernels or rules of an automaton, perhaps more on this later). The first assumption is that the relative frequency of the occurrence of A is close to P(A) provided that n is sufficiently large. Obvious question: how large is large enough? The second assumption is that provided you have meet the first demand then only with a “high degree of certainty” is the equation valid.
Any use of this equation for prediction in the real word takes us down the path of induction. You’ll never be able to run an experiment an infinite amount of times and so to estimate the probability of an event occurring on the n+1 experiment it is going to have to be based on a priori knowledge. Say you flip a coin 100 times and see heads half of the time, then you would induce that the next 100 flips would yield heads half of the time.
David Hume had a problem with inductive logic. Gauche had the following summary of his argument:
“(i)Any verdict on the legitimacy of induction must result from deductive or inductive arguments, because those are the only kinds of reasoning.
(ii)A verdict on induction cannot be reached deductively. No inference from the observed to the unobserved is deductive, specifically because nothing in deductive logic can ensure that the course of nature will not change.
(iii)A verdict cannot be reached inductively. Any appeal to the past successes of inductive logic, such as that bread has continued to be nutritious and that the sun has continued to rise day after day, is but worthless circular reasoning when applied to induction’s future fortunes.
Therefore, because deduction and induction are the only options, and because neither can reach a verdict on induction, the conclusion follows that there is no rational justification for induction.”
Whoops. Now what?
It’s fucking late now and I’m just one page into Papoulis. I’m having fun though so I’ll keep doing this (daily?) and hopefully have time to go back and properly reference things.
Next post, I want to pick this up with a way-out of the inductive reasoning trap which I found in a recent paper by Uwe Saint-Mont. Then maybe I’ll move on to deductive (math-fun) reasoning.