What do you think?
Rate this book
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy
Bayes' rule appears to be a straightforward, one-line by updating our initial beliefs with objective new information, we get a new and improved belief. To its adherents, it is an elegant statement about learning from experience. To its opponents, it is subjectivity run amok. In the first-ever account of Bayes' rule for general readers, Sharon Bertsch McGrayne explores this controversial theorem and the human obsessions surrounding it. She traces its discovery by an amateur mathematician in the 1740s through its development into roughly its modern form by French scientist Pierre Simon Laplace. She reveals why respected statisticians rendered it professionally taboo for 150 years—at the same time that practitioners relied on it to solve crises involving great uncertainty and scanty information (Alan Turing's role in breaking Germany's Enigma code during World War II), and explains how the advent of off-the-shelf computer technology in the 1980s proved to be a game-changer. Today, Bayes' rule is used everywhere from DNA de-coding to Homeland Security. Drawing on primary source material and interviews with statisticians and other scientists, The Theory That Would Not Die is the riveting account of how a seemingly simple theorem ignited one of the greatest controversies of all time.
336 pages, Hardcover
First published May 14, 2011
919 people are currently reading
5727 people want to read
About the author
Sharon Bertsch McGrayne
11Ìýbooks30ÌýfollowersSharon Bertsch McGrayne is the author of highly-praised books about scientific discoveries and the scientists who make them. She is interested in exploring the cutting-edge connection between social issues and scientific progress â€� and in making the science clear and interesting to non-specialists.
Ratings & Reviews
Friends & Following
Create a free account to discover what your friends think of this book!
Community Reviews
Displaying 1 - 30 of 346 reviews
September 3, 2012
If only I had known how to use Bayesian probabilities before reading this book I could have taken the probability of my liking a book well received in a NY Times Book Review as my prior, plugged that in to a Bayesian calculation as to whether or not I would like this book, and quickly would have come up with the answer "highly unlikely" and saved myself a few hours of my life.
According to the author, Bayes' rule is the greatest mathematical equation/formula/thought process in the history of history and can solve any problem from finding a lost submarine, to determining who really wrote which Federalist Paper, to figuring out why people like country music (ok, I made the last one up because you don't need an equation to come up with the answer "insanity"). Anyway, the annoyance of this book is the author pontificates on how great Bayes' Rule is without ever, you know, actually giving us a mathematical example. It's mindblowingly frustrating.
The chapters basically read like this. The British government wanted to crack the Nazi Enigma code in WWII so they hired Alan Turing who employed Bayes' Rule, and voila! Problem solved. Or, the US wanted to find a missing submarine so they used Bayes' Rule and there it was.
Bayes' Rule could have been anything. The fact that the author does such a poor job of defining it and a poorer job of showing how it was actually used and some of the math behind it is irritating. You could find and replace "Bayes' Rule" with "Johnson Rod" throughout the whole book and not lose anything. By keeping Bayes' Rule essentially as a black box, it becomes less interesting overall.
There was an attempt in the appendix to show a simple mathematical application of Bayes' Rule, which was helpful, but even that was screwed up as the verbal description of the variables did not match the equation (numerators were flip-flopped on consecutive pages, doesn't change the outcome, but come on, you need consistency when showing the formula about which the entire book is written).
The history of Bayes' Rule is really what the book is about, and some of it is entertaining, but save yourself some time and just wikipedia it and you'll learn infinitely more in a shorter period of time.
According to the author, Bayes' rule is the greatest mathematical equation/formula/thought process in the history of history and can solve any problem from finding a lost submarine, to determining who really wrote which Federalist Paper, to figuring out why people like country music (ok, I made the last one up because you don't need an equation to come up with the answer "insanity"). Anyway, the annoyance of this book is the author pontificates on how great Bayes' Rule is without ever, you know, actually giving us a mathematical example. It's mindblowingly frustrating.
The chapters basically read like this. The British government wanted to crack the Nazi Enigma code in WWII so they hired Alan Turing who employed Bayes' Rule, and voila! Problem solved. Or, the US wanted to find a missing submarine so they used Bayes' Rule and there it was.
Bayes' Rule could have been anything. The fact that the author does such a poor job of defining it and a poorer job of showing how it was actually used and some of the math behind it is irritating. You could find and replace "Bayes' Rule" with "Johnson Rod" throughout the whole book and not lose anything. By keeping Bayes' Rule essentially as a black box, it becomes less interesting overall.
There was an attempt in the appendix to show a simple mathematical application of Bayes' Rule, which was helpful, but even that was screwed up as the verbal description of the variables did not match the equation (numerators were flip-flopped on consecutive pages, doesn't change the outcome, but come on, you need consistency when showing the formula about which the entire book is written).
The history of Bayes' Rule is really what the book is about, and some of it is entertaining, but save yourself some time and just wikipedia it and you'll learn infinitely more in a shorter period of time.
February 18, 2012
I think this is the first book about Bayes' theorem and its applications, for the general reader. The book does not explicitly state the theorem as a mathematical formula, until the second appendix. However, the general idea is described, as well the general ideas behind it. The history of the theorem is described in some detail.
The ebb and flow in belief in the theorem over the course of 150 years is interesting. Applying Bayes theorem requires a prior probability, and this is often poorly known--it is often an educated, but subjective guess. Mathematicians and statisticians don't like guesses, they don't like subjectivity. As a result, the application of Bayes theorem was often in disrepute. But--despite the subjectivity, Bayes theorem usually works--and works very well! It was used most effectively during World War II, in the decryption of the Enigma code. It was also used effectively in anti-submarine warfare during the war, and in search and rescue operations. But its usage was classified, and as a result its power was hidden from statisticians.
In the past 20 years, Bayes theorem has really taken hold. I personally use it daily in my work, where it is extremely useful. I vaguely remember reading about the controversies surrounding it. Now, at long last, you can learn about the true nature of the controversies in this enjoyable book.
The ebb and flow in belief in the theorem over the course of 150 years is interesting. Applying Bayes theorem requires a prior probability, and this is often poorly known--it is often an educated, but subjective guess. Mathematicians and statisticians don't like guesses, they don't like subjectivity. As a result, the application of Bayes theorem was often in disrepute. But--despite the subjectivity, Bayes theorem usually works--and works very well! It was used most effectively during World War II, in the decryption of the Enigma code. It was also used effectively in anti-submarine warfare during the war, and in search and rescue operations. But its usage was classified, and as a result its power was hidden from statisticians.
In the past 20 years, Bayes theorem has really taken hold. I personally use it daily in my work, where it is extremely useful. I vaguely remember reading about the controversies surrounding it. Now, at long last, you can learn about the true nature of the controversies in this enjoyable book.
January 7, 2012
As someone who actually works with Bayesian methods, I was very much looking forward to reading this book. The strange history of Bayes' Theorem had been briefly mentioned in other, more technical books I had read. I finally wanted to get the whole story.
Alas, that story, at least as presented in this book, turned out to be not quite so exciting. Except for the insights into Laplace's involvement, and in particular the interesting sections on Alan Turing's work, I found this to be a rather lifeless book. With a bit of passion it could have easily been more of a page turner. The author also clearly struggled with the fact that she could not present a fully mathematical explanation and arguments for Bayesian methods due to targeting a lay audience. Hence, "Bayes", as she keeps calling the approach, remains a nebulous entity throughout the book, misrepresented as simply "adding prior information". I cannot begin to imagine what a strange experience it must be to read this book if one has never worked with the formalism and its various benefits (e.g., marginalization).
Still, I am thankful to the author (and the publisher) that she (they) at least tried to make a topic like this more accessible to the public.
Alas, that story, at least as presented in this book, turned out to be not quite so exciting. Except for the insights into Laplace's involvement, and in particular the interesting sections on Alan Turing's work, I found this to be a rather lifeless book. With a bit of passion it could have easily been more of a page turner. The author also clearly struggled with the fact that she could not present a fully mathematical explanation and arguments for Bayesian methods due to targeting a lay audience. Hence, "Bayes", as she keeps calling the approach, remains a nebulous entity throughout the book, misrepresented as simply "adding prior information". I cannot begin to imagine what a strange experience it must be to read this book if one has never worked with the formalism and its various benefits (e.g., marginalization).
Still, I am thankful to the author (and the publisher) that she (they) at least tried to make a topic like this more accessible to the public.
November 22, 2022
What is known today as Bayes' theorem was invented, like most things, by the marquis de Laplace, but Bayes really did come up with the most important part of it. (By modern conventions we'd call it the Bayes-Price-Laplace rule, but it's too late.) The idea was known then as "inverse probability": probability began with problems where the possible outcomes are known (e.g. a game of chance), and tried to calculate the likelihood of a given outcome. The inverse is to begin with an outcome and try to calculate the probability of it happening. This is one of the core problems of science - given 100 white swans and one black, how likely is it that 1% of swans are black, or that this one is a mutant, or that none are black and this observation a trick of the light? Laplace cheerfully proceeded to invent frequentism, the sworn enemy of Bayesian statistics, and followed the latter for the rest of his career (believing, incorrectly, that the two methods always converge on the same solution). Looking for problems to put to the new statistical methodology, Laplace examined if rates of male and female births are identical, and found that male births are consistently higher, but couldn't explain why. (And apparently ! We still don't know why! I've been misled by every schoolchild probability exercise that begins "assume male and female births equally likely�")
This book, a pretty good history of the topic, then moves on to the development of modern statistical research methods, led by R.A. Fisher, Pearson ±èè°ù±ð (Karl) and fils (Egon) and Jerzy Neyman, fanatical anti-Bayesians all. Thus began the modern frequentist-Bayesian war. Bayes method is uncontroversial when the prior probabilities are known, but the debate is over the legitimacy of making up a "subjective" prior based on one's intuition, or using a uniform prior, when one doesn't. The opposition of the leading statisticians to Bayesian methods won out until WWII, when code-breaking became a mathematicians's game. (At least for early adopters, such as the Poles. "The British agency charged with cracking German military codes and ciphers clung to the tradition that decryption was a job for gentlemen with linguistic skills. Instead of hiring mathematicians, the Government Code and Cypher School employed art historians, scholars of ancient Greek and medieval German, crossword puzzlers, and chess players.") An actuarial mathematicians, Marián Rejewski, used group theory to figure out how the wheels of the German Enigma were wired. Alan Turing used Bayesian methods at Bletchley Park to calculate the expected probability of the occurrence of various letters, allowing the breaking of codes which were switched far too often to be broken by brute force.
After the war, the Bayesian cause was taken up by luminaries such as Jimmie Savage, Harold Jeffreys and E.T. Jaynes. ("Jaynes, who said he always checked to see what Laplace had done before tackling an applied problem, turned off many colleagues with his Bayesian fervor.") Savage was a "full-blown subjectivist": science begins as subjective, varying opinions, he believed; and as more data comes in views slowly converge on the truth. The popularity of quantum mechanics, which like stat mech uses frequentist statistics, worked against the neo-Bayesians. Erwin Schrödinger said, "The individual case is entirely devoid of interest." Robert Schlaifer was a historian who got a job teaching statistics at Harvard Business School and basically taught himself on the fly, inventing the field of Bayesian decision theory as he went. John Tukey used Bayesian methodology, but didn't call himself one. Am I losing you? The back end of this book starts to sag, as it goes through a laundry list of statisticians and their allegiances to Bayesianism, as well as the emergence of numerical computational methods which made calculating posteriors much more feasible: MCMC, Gibbs sampling, the Metropolitan-Hastings algorithm, and software like Stan or BUGS. The breadth of the coverage is overwhelming, and hampered by its shallowness - there are almost no mathematical details in the book, aside from the simplest arithmetic examples. And what's worse is how the end slowly descends into the breathless language of tech marketing: here is a list of unrelated problems, they are unsolvable by traditional methods, but guess which magical solution is being used to solve them?
Ultimately, Bayesian thinking is a useful way to make decisions and a powerful statistical paradigm for solving problems with limited information. While it may be the preferred approach now in a lot of fields, it is also not a panacea, nor are frequentist methods being put to pasture. If the Bayesian wars have reached some kind of conclusion, it is that knowing when to reach for MLE or MAP, p-values or posteriors, or some combination of both (paging Empirical Bayesians!), is more important than blind devotion to a method.
This book, a pretty good history of the topic, then moves on to the development of modern statistical research methods, led by R.A. Fisher, Pearson ±èè°ù±ð (Karl) and fils (Egon) and Jerzy Neyman, fanatical anti-Bayesians all. Thus began the modern frequentist-Bayesian war. Bayes method is uncontroversial when the prior probabilities are known, but the debate is over the legitimacy of making up a "subjective" prior based on one's intuition, or using a uniform prior, when one doesn't. The opposition of the leading statisticians to Bayesian methods won out until WWII, when code-breaking became a mathematicians's game. (At least for early adopters, such as the Poles. "The British agency charged with cracking German military codes and ciphers clung to the tradition that decryption was a job for gentlemen with linguistic skills. Instead of hiring mathematicians, the Government Code and Cypher School employed art historians, scholars of ancient Greek and medieval German, crossword puzzlers, and chess players.") An actuarial mathematicians, Marián Rejewski, used group theory to figure out how the wheels of the German Enigma were wired. Alan Turing used Bayesian methods at Bletchley Park to calculate the expected probability of the occurrence of various letters, allowing the breaking of codes which were switched far too often to be broken by brute force.
After the war, the Bayesian cause was taken up by luminaries such as Jimmie Savage, Harold Jeffreys and E.T. Jaynes. ("Jaynes, who said he always checked to see what Laplace had done before tackling an applied problem, turned off many colleagues with his Bayesian fervor.") Savage was a "full-blown subjectivist": science begins as subjective, varying opinions, he believed; and as more data comes in views slowly converge on the truth. The popularity of quantum mechanics, which like stat mech uses frequentist statistics, worked against the neo-Bayesians. Erwin Schrödinger said, "The individual case is entirely devoid of interest." Robert Schlaifer was a historian who got a job teaching statistics at Harvard Business School and basically taught himself on the fly, inventing the field of Bayesian decision theory as he went. John Tukey used Bayesian methodology, but didn't call himself one. Am I losing you? The back end of this book starts to sag, as it goes through a laundry list of statisticians and their allegiances to Bayesianism, as well as the emergence of numerical computational methods which made calculating posteriors much more feasible: MCMC, Gibbs sampling, the Metropolitan-Hastings algorithm, and software like Stan or BUGS. The breadth of the coverage is overwhelming, and hampered by its shallowness - there are almost no mathematical details in the book, aside from the simplest arithmetic examples. And what's worse is how the end slowly descends into the breathless language of tech marketing: here is a list of unrelated problems, they are unsolvable by traditional methods, but guess which magical solution is being used to solve them?
Ultimately, Bayesian thinking is a useful way to make decisions and a powerful statistical paradigm for solving problems with limited information. While it may be the preferred approach now in a lot of fields, it is also not a panacea, nor are frequentist methods being put to pasture. If the Bayesian wars have reached some kind of conclusion, it is that knowing when to reach for MLE or MAP, p-values or posteriors, or some combination of both (paging Empirical Bayesians!), is more important than blind devotion to a method.
December 26, 2011
I wish I had liked this book more than I actually did. Most of the stories reported are very interesting and entertaining, reflecting how academics have been fiercely debating conceptual aspects of Bayes theorem, as well as the bayesian-frequentist feud, while at the same time it was being successfully applied in many crucial issues such as finding stray atomic weapons and linking smoking to lung cancer.
However, I did not find this book well-written at all. It's just not an exciting read - and it could've been, because the stories are indeed very interesting. Also, even with the glossary at the end, many technical aspects are reported without any explanation of why they were important and what they meant to Bayesian statistics, giving the feel of careless "name-throwing". It also feels like the the author tried to cramp together every single case of applied Bayesian approaches, in a way that the important cases are often drowned in a noisy list of achievements. In some cases, after a very long build-up, it turns out that Bayes theorem either wasn't even used or wasn't important, and the story doesn't tie up with a future Bayesian achievement spanning from that. It's just there because it remotely related to Bayes theorem and thus "had" to be included in the book.
So, in my opinion, this is a book that is rich in interesting stories and information, but the reader will need extra willpower to drag through the unexciting narrative and the excessive "noise".
However, I did not find this book well-written at all. It's just not an exciting read - and it could've been, because the stories are indeed very interesting. Also, even with the glossary at the end, many technical aspects are reported without any explanation of why they were important and what they meant to Bayesian statistics, giving the feel of careless "name-throwing". It also feels like the the author tried to cramp together every single case of applied Bayesian approaches, in a way that the important cases are often drowned in a noisy list of achievements. In some cases, after a very long build-up, it turns out that Bayes theorem either wasn't even used or wasn't important, and the story doesn't tie up with a future Bayesian achievement spanning from that. It's just there because it remotely related to Bayes theorem and thus "had" to be included in the book.
So, in my opinion, this is a book that is rich in interesting stories and information, but the reader will need extra willpower to drag through the unexciting narrative and the excessive "noise".
June 17, 2016
This was an excellent biography of Bayes' Rule, which basically glossed over Bayes himself. The author chose instead to examine the lesser known scientists and applications associated with Bayes. As a result, after reading this you are likely to call Bayes' Rule BLP Rule, for Bayes-Laplace Rule. The author was most interested in highlighting the work done by Pierre-Simone Laplace, who I feel I have come to know so much more after this biography. Prior to this book, Laplace came on my radar after watching a Sean Carroll talk , listening toFrederick Gregory's lecture series, History of Science 1700 - 1900, and reading David Bodanis' e=mc2. (My memory indicated Bodanis included Laplace in his book, but attempts to confirm this have not been successful. Maybe credit is due another author. It is possible I have oddly attributed my knowledge of Laplace to someone who didn't even include him a book. Memory is so strange.).
I remember being a bit wowed and intrigued after first learning about Laplace; and yet, not doing any further research. What a shame that would have been. Laplace was an exceptional scientist. He not only came up with Bayes Rule by himself, but he also did more work than Bayes to contribute to humankind's understanding of probability, fought vigorously to separate religion and scientific inquiry, insisted on facts over belief, and was extremely productive in developing a foundation for statistics-- despite receiving so little reward. You will be treated to how, as a thank you from society, his life and reputation were ruined. Poor Laplace.
The author also provided a fairly good biography of other contributors to Bayes' Rule development and application throughout history. The rule itself was extremely unpopular. It's successes were hidden in wartime to protect war secrets. Those who used it were often bullied by the larger statistics community. And yet, the theory lived on, often under the radar, to continue helping researchers solve the hard problems. When the author provided a survey of how Bayes was used, I was familiar with the instances she highlighted but didn't realize Bayes was the method used to solve the problems at hand.
Since the author included, what I can only imagine, was every instance in which Bayes was employed, at times I felt like, "Yes, I have got it. Move on." I got a bit bored at times during the last few chapters. However, I think it would be unfair to criticize a book for including too much information about the focus subject:)
This was solid research, lots of it, that created a very thorough biography of the Rule/Theory itself.
I remember being a bit wowed and intrigued after first learning about Laplace; and yet, not doing any further research. What a shame that would have been. Laplace was an exceptional scientist. He not only came up with Bayes Rule by himself, but he also did more work than Bayes to contribute to humankind's understanding of probability, fought vigorously to separate religion and scientific inquiry, insisted on facts over belief, and was extremely productive in developing a foundation for statistics-- despite receiving so little reward. You will be treated to how, as a thank you from society, his life and reputation were ruined. Poor Laplace.
The author also provided a fairly good biography of other contributors to Bayes' Rule development and application throughout history. The rule itself was extremely unpopular. It's successes were hidden in wartime to protect war secrets. Those who used it were often bullied by the larger statistics community. And yet, the theory lived on, often under the radar, to continue helping researchers solve the hard problems. When the author provided a survey of how Bayes was used, I was familiar with the instances she highlighted but didn't realize Bayes was the method used to solve the problems at hand.
Since the author included, what I can only imagine, was every instance in which Bayes was employed, at times I felt like, "Yes, I have got it. Move on." I got a bit bored at times during the last few chapters. However, I think it would be unfair to criticize a book for including too much information about the focus subject:)
This was solid research, lots of it, that created a very thorough biography of the Rule/Theory itself.
June 8, 2013
Excellent and very readable book about the history of Bayes' theorem. I never realized that Bayesian statistics, one of the cornerstones of modern data science, had such a turbulent history--so turbulent that, during the cold war, being called a "bayesian" was tantamount to being called a Communist.
If you're at all interested in the history of mathematics, this is a surprisingly exciting story. I expected a rather dull and academic history; that is NOT what this book is.
If you're at all interested in the history of mathematics, this is a surprisingly exciting story. I expected a rather dull and academic history; that is NOT what this book is.
October 3, 2011
It probably takes a special sort of person to dive into an entire book about one statistical theory, but for those so-motivated, this one pays off.
The pro's: The author has done a phenomenal job at capturing and richly detailing the very "large" personalities that have championed (or condemned) the use of Bayes' Rule through the centuries, amidst a little-known and long-simmering war that has persisted between statistical Bayesians and frequentists since the concept was first brought forward. This is even more impressive as she is a journalist, rather than a statistician. McGrayne immerses the reader in what can only be called "lush" detail of the history, from personalities to global events.
The con's: This a very dense text. Not dry in an academic sense, but a lot of material to consume. At times I had to summon extra reserves of motivation to proceed to the next chapter. The topic is also a difficult one to communicate solely through narrative - more than once I found myself wishing for just a little bit of math-by-the-way-of-example to help grasp the concepts. (With such, this could actually serve well as an educational vehicle). While already familiar with Bayes, the application in some of the historical examples was, for me, elusive.
Computing power has today made the Bayesian/frequentist conflict somewhat moot, and I found myself wishing for a little more exposition of Bayesian applications in the modern era. (To me, this is where the real excitement lies, if "excitement" is the correct term!)
Overall - if statistics, scientific inference, decision theory or machine learning excite you, this is probably a book to have under your belt. Reading the history of Bayesian vs frequentist wars triggered some good musing and reflection on the critical question of "how to make inferences when too little, rather than too much, data are at hand".
The pro's: The author has done a phenomenal job at capturing and richly detailing the very "large" personalities that have championed (or condemned) the use of Bayes' Rule through the centuries, amidst a little-known and long-simmering war that has persisted between statistical Bayesians and frequentists since the concept was first brought forward. This is even more impressive as she is a journalist, rather than a statistician. McGrayne immerses the reader in what can only be called "lush" detail of the history, from personalities to global events.
The con's: This a very dense text. Not dry in an academic sense, but a lot of material to consume. At times I had to summon extra reserves of motivation to proceed to the next chapter. The topic is also a difficult one to communicate solely through narrative - more than once I found myself wishing for just a little bit of math-by-the-way-of-example to help grasp the concepts. (With such, this could actually serve well as an educational vehicle). While already familiar with Bayes, the application in some of the historical examples was, for me, elusive.
Computing power has today made the Bayesian/frequentist conflict somewhat moot, and I found myself wishing for a little more exposition of Bayesian applications in the modern era. (To me, this is where the real excitement lies, if "excitement" is the correct term!)
Overall - if statistics, scientific inference, decision theory or machine learning excite you, this is probably a book to have under your belt. Reading the history of Bayesian vs frequentist wars triggered some good musing and reflection on the critical question of "how to make inferences when too little, rather than too much, data are at hand".
March 9, 2025
Troves of trying trivia. There is some very important information here but it is buried.
I think the author is trying to force the book to have a narrative by giving us the life of Bayes, the childhood of Laplace, etc. But I doubt that many people buying books about mathematical theories are interested in the minor details of the mathematicians' lives. This type of writing would be bad enough if the importance of Bayesian analysis were clearly explained, but it isn't. For instance, in the chapter on lung cancer, one learns about Jerome Cornfield; unfortunately, the few lines on his research don't establish that it was Cornfield's Bayesian thinking that led to the discovery of the causal link between smoking and lung cancer.
I think the author is trying to force the book to have a narrative by giving us the life of Bayes, the childhood of Laplace, etc. But I doubt that many people buying books about mathematical theories are interested in the minor details of the mathematicians' lives. This type of writing would be bad enough if the importance of Bayesian analysis were clearly explained, but it isn't. For instance, in the chapter on lung cancer, one learns about Jerome Cornfield; unfortunately, the few lines on his research don't establish that it was Cornfield's Bayesian thinking that led to the discovery of the causal link between smoking and lung cancer.
July 13, 2018
(c) Red Rationality (1987) by Wang Guangyi
A slightly strained oral history of the least romanticised scientists: Bayesian statisticians. She makes up for the long-missing romanticism single-handed! The two-hundred year eclipse of the Bayesian method was much longer than that suffered by even the irrationally-maligned theory (50 years). And this neglect and opprobrium was suffered by a paradigm now accepted everywhere as powerful and useful in literally all kinds of research.
She wins us over, particularly with her chapter on the secretive, truculent, omnicompetent genius John Tukey, who used Bayesian methods for elections 40 years before , with comparable success. But her prose is borderline, with lots of clear but dim-bulb sentences.
She has one infuriating mannerism: she constantly refers to Bayes' rule, Bayesian logical foundations after Bayes, Bayesian inference, and personalist Bayesian epistemology by the single terrible metonym "Bayes":
At its heart, Bayes runs counter to the deeply held conviction that science requires total objectivity and precision. Bayes is a measure of belief.
...even many nonstatisticians regarded Swinburne's lack of care and measurement as a black mark against Bayes itself.
...Bayes, on the other hand, seemed to produce results that corresponded more closely to sociologists' intuitions.
...Wagner took along the youngest and greenest of his three-man staff, Henry Richardson, who had earned a PhD in probability theory all of seven months earlier. He would be Bayes' point man at Palomares.
I suppose she did this to elide away jargon, but it both equivocates between very different entities, hides the complexity of the 'Bayesian' marquee, invites the idea that the frequentists were attacking a logically sound theorem, and produces a whole list of bizarre images, where we see the reclusive Reverend doing all these things: cracking Enigma and Tunny, finding H-Bombs lost at sea, calculating appropriate worker's comp amounts in the absence of reliable data, attributing The Federalist Papers to Hamilton, and blocking 99.9% of the spam email from reaching you (yes, you).
It is also even more unfair to Laplace than usual. (It was he who developed Bayesianism into the powerful applied framework it is, into more than a single gambler's theorem. Ok, so "Laplace-Coxism" is admittedly even less admissible as a term to which the wise and honest may repair.)
Grammatical twitching aside this was a fun introduction to an important thing.
She focusses on the soft, social side (and on applications vaguely summarised). There was a huge amount of factional bitching between these serious and cloistered men:
Attending his first Bayesian conference in 1976, Jim Berger was shocked to see half the room yelling at the other half. Everyone seemed to be good friends, but their priors were split between the personally subjective, like Savage's, and the objective, like Jeffrey's - with no definitive experiment to decide the issue.
In a frustrated circle of blame, Persi Diaconis was shocked and angry when John Pratt used frequentist methods to analyze his wife's movie theater attendance data, because it was too much for the era's computers to handle. But one of the low moments of Diaconis' life occurred in a Berkeley coffee shop, where he was correcting proofs of an article of his and where Lindley blamed him for using frequency methods. "And you're our leading Bayesian", Lindley complained. Lindley, in turn, upset Mosteller by passing up a chance to conduct a big project using Bayes instead of frequency...
Asked how to encourage Bayesian theory, Lindley answered tartly, "Attend funerals".
This human focus means she gives no treatment of Cox's theorem, certainly the most remarkable result in formal epistemology (and probability theory?), and one of the main things which rationally warrants the partisanship and excitement she displays for Bayesian thought throughout. ("", as one great commentator puts it!) It proves that any attempt to use numbers to model belief must be Bayesian or logically equivalent to it. With other results, it raises Bayesianism to the only viable quantitative theory of rationality and of right learning, a behemoth of which Aristotelian logic is a mere special case. No doubt I'm unusual in finding this the most exciting bit.
She's to be applauded for digging out novel examples of Bayesian analysis which were classified or which avoided using the word: early actuarial work, Tukey's US election model, the pre-Three-Mile-Island federal report of reactor safety, and the entire field of operational research. But she is so concerned with emphasising the (genuine) long oppression of the paradigm that she under-emphasises the good reasons to resist Bayesian methods before 1980: they were simply computationally intractable before MCMC. (Which makes the sheer effort put in to shortcuts and approximation methods by ingenious people quite tragic; they just aren't needed anymore, thirty years later.) To her credit, she does mention the parallel dogmatism of the 60s Bayesians and the presumptive overenthusiasm of some people in the last 10 years.
(The great contemporary frequentist, Deborah Mayo, is able to subtitle her blog "" without being absurd - even though Stats 101 and "Methods for [Social Science]" courses are still everywhere dominated by canned Fisherian tests and frames. She means exile from the philosophy of statistics and probability.)
Insofar as you want to understand the large trends of the present and coming age, you need to know its economics; insofar as you must understand the new economics, you must understand AI; insofar as you must understand AI, you must understand machine learning and decision theory; insofar as you must understand machine learning, you must understand . Insofar as you do not yet have the mathematics to understand Bayesianism, nor the excitement of the promise of a final, real synthesis of objective with subjective, you must read this gentle prose work. Once you are excited by its vague promises, you can find progressively more rigorous people and will have actually have reason to stomach the formalism.
3/5, 4*/5 for those just beginning the march.
(c) African Sonata (c.2000) by Vladimir Kush
April 20, 2014
As the subtitle proclaims, this book chronicles the history of science It also demonstrates how a simple formula evolved into a sophisticated application that required the invention of high speed computers to exploit its potential for prediction. It complements the information in Nate Silver's book, THE SIGNAL AND THE NOISE.
McGrayne introduces the reader to Bayes's Theorem with the proposal that given the unknown position of a billiard ball, its probable position can be narrowed by collecting data about its spatial relation (right or left) to a succession of balls rolled randomly across the table. The technique is similar to the narrowing process that comes about during the game of �20 Questions.�
McGrayne then explains the concept of inverse probability: “We modify our opinions with objective information: Initial Beliefs � + Recent Objective Data... = A New and Improved Belief.� The initial belief is termed the prior probability and the revised belief is called the posterior probability. The idea was originated by Bayes, but was refined by Pierre-Simon Laplace, a French astronomer and mathematician. McGrayne contrasts the approaches of the two men as follows: “[Bayes] wanted to know the range of probabilities that something will happen in light of previous experience. Laplace wanted more: as a working scientist, he wanted to know the probability that certain measurements and numerical values associated with a phenomenon were realistic.� Her summary of Laplace's formulation is as follows: �...P(C|E), the probability of a hypotheses (given information), equals Pprior(C), our initial estimate of its probability, times P(E|C), the probability of each new piece of information (under the hypothesis), divided by the sum of the probabilities of the data in all possible hypotheses." She makes the case that Laplace had discovered the formulation by 1781.
The history of science is directed by many non-scientific incidents. The theorem fell out of favor in scientific circles, replaced by frequency-based probability methods. The focus of inquiry shifted to sampling methodology. As a result, we are all familiar with analysis based on the bell-shaped curve. How Bayes's Theorem stayed alive consumes the remainder of McGrayne's story. Much of this shift was due to the violently anti-Bayesian position of Ronald Fisher, a prominent British geneticist and statistician.
In the 1920's and '30's, the idea survived in the fields of financial economics, paternity law, biostatistics, and geology. Why? Because it provided practical data that could be used as a basis for action. During World War II the theorem received another boost. It was instrumental in cracking the German Enigma code. The effort was led by Alan Turing, whose interests spanned applied mathematics, machine language, codes and logic. He was a man of unconventional interests, a non-linguist, and a non-statistician. Above all, Turing was attracted to the Enigma problem because it was a challenge. No one else wanted to tackle the problem, and he enjoyed working alone. Meanwhile, Bayes's theorem was being applied in America by Claude Shannon at Bell Labs.
One problem for the resuscitation of Bayes's theorem was that all of this wartime work was classified. Post-war practice was kept alive in schools of business and actuary science. Academic mathematicians and statisticians ignored it.
To me, the most exciting post-war application was Albert Madansky's report on the probability of an accidental H-bomb detonation. Since there had never been an accidental detonation, frequency theory was of no help. Madansky's calculations included the fact that between 1950 and 1958 there had been more than 16 nuclear weapon related incidents � accidental drops, plane crashes, and handling errors of unarmed weapons. He factored in the proposed increase in SAC bombers and ICBM's, as well as the amount of handling that would be required. Even the air force calculated that 5 major plane accidents per 100,000 flying hours was a reasonable assumption. Madansky concluded that the expanded SAC proposal would make 19 “conspicuous� weapons accidents per year a high probability. The report resulted in significant changes in nuclear weapons handling procedures.
The early chapters of the book felt slow. They are primarily of interest to the historian seeking to verify the contributions to the theory by specific mathematicians. However, McGrayne does a convincing job of stressing the importance of Bayes's Theorem without going into the mathematical details. She also reveals interesting information about how academic rivalries, military secrecy, politics, and inter-disciplinary studies influence scientific progress. The latter is perhaps the most significant lesson this book offers to the general reader. Her many detailed historic examples make a compelling case for the importance of Bayes's Theorem in scientific analysis.
McGrayne introduces the reader to Bayes's Theorem with the proposal that given the unknown position of a billiard ball, its probable position can be narrowed by collecting data about its spatial relation (right or left) to a succession of balls rolled randomly across the table. The technique is similar to the narrowing process that comes about during the game of �20 Questions.�
McGrayne then explains the concept of inverse probability: “We modify our opinions with objective information: Initial Beliefs � + Recent Objective Data... = A New and Improved Belief.� The initial belief is termed the prior probability and the revised belief is called the posterior probability. The idea was originated by Bayes, but was refined by Pierre-Simon Laplace, a French astronomer and mathematician. McGrayne contrasts the approaches of the two men as follows: “[Bayes] wanted to know the range of probabilities that something will happen in light of previous experience. Laplace wanted more: as a working scientist, he wanted to know the probability that certain measurements and numerical values associated with a phenomenon were realistic.� Her summary of Laplace's formulation is as follows: �...P(C|E), the probability of a hypotheses (given information), equals Pprior(C), our initial estimate of its probability, times P(E|C), the probability of each new piece of information (under the hypothesis), divided by the sum of the probabilities of the data in all possible hypotheses." She makes the case that Laplace had discovered the formulation by 1781.
The history of science is directed by many non-scientific incidents. The theorem fell out of favor in scientific circles, replaced by frequency-based probability methods. The focus of inquiry shifted to sampling methodology. As a result, we are all familiar with analysis based on the bell-shaped curve. How Bayes's Theorem stayed alive consumes the remainder of McGrayne's story. Much of this shift was due to the violently anti-Bayesian position of Ronald Fisher, a prominent British geneticist and statistician.
In the 1920's and '30's, the idea survived in the fields of financial economics, paternity law, biostatistics, and geology. Why? Because it provided practical data that could be used as a basis for action. During World War II the theorem received another boost. It was instrumental in cracking the German Enigma code. The effort was led by Alan Turing, whose interests spanned applied mathematics, machine language, codes and logic. He was a man of unconventional interests, a non-linguist, and a non-statistician. Above all, Turing was attracted to the Enigma problem because it was a challenge. No one else wanted to tackle the problem, and he enjoyed working alone. Meanwhile, Bayes's theorem was being applied in America by Claude Shannon at Bell Labs.
One problem for the resuscitation of Bayes's theorem was that all of this wartime work was classified. Post-war practice was kept alive in schools of business and actuary science. Academic mathematicians and statisticians ignored it.
To me, the most exciting post-war application was Albert Madansky's report on the probability of an accidental H-bomb detonation. Since there had never been an accidental detonation, frequency theory was of no help. Madansky's calculations included the fact that between 1950 and 1958 there had been more than 16 nuclear weapon related incidents � accidental drops, plane crashes, and handling errors of unarmed weapons. He factored in the proposed increase in SAC bombers and ICBM's, as well as the amount of handling that would be required. Even the air force calculated that 5 major plane accidents per 100,000 flying hours was a reasonable assumption. Madansky concluded that the expanded SAC proposal would make 19 “conspicuous� weapons accidents per year a high probability. The report resulted in significant changes in nuclear weapons handling procedures.
The early chapters of the book felt slow. They are primarily of interest to the historian seeking to verify the contributions to the theory by specific mathematicians. However, McGrayne does a convincing job of stressing the importance of Bayes's Theorem without going into the mathematical details. She also reveals interesting information about how academic rivalries, military secrecy, politics, and inter-disciplinary studies influence scientific progress. The latter is perhaps the most significant lesson this book offers to the general reader. Her many detailed historic examples make a compelling case for the importance of Bayes's Theorem in scientific analysis.
July 22, 2013
It is the third book I read about statistics in a short while and it is probably the strangest. After my dear Taleb and his The Black Swan, after the more classical "Naked Statistics: Stripping the Dread from the Data", here is the history of the Bayesian statistics. If you do not know about Bayes, let me just add that I like the beautiful and symmetric formula: [According to wikipedia] For proposition A and evidence B, P(A|B) P(B) = P(B|A) P(A) with P(A), the prior, is the initial degree of belief in A. P(A|B), the posterior, is the degree of belief having accounted for B. The quotient P(B|A)/P(B) represents the support B provides for A. Another way of explaining it mathematically is Bayes' theorem gives the relationship between the probabilities of A and B, P(A) and P(B), and the conditional probabilities of A given B and B given A, P(A|B) and P(B|A).
I was never really comfortable with its applications. I was probably wrong again, given all what I learnt after reading Sharon Bertsch McGrayne's rich book. But I also understood why I was never comfortable: for three centuries, there's been a quasi-religious war between Bayesians and Frequentists on how to use probabilities. Are these linked to big, frequent numbers only or can they be applied for rare events? What is the probability of a rare event which may never occur or maybe just once? [Let me give you a personal example: I am interested in serial entrepreneurship, and did and still do tons of statistics on Stanford-related companies. I have more than 5'000 entrepreneurs, and more than 1'000 are serial. I have results showing that serial entrepeneurs are not on average better than one-time, using frequency and classical methods. But now I should think about using: P(Success|Serial) = P(Serial|Sucess) P(Success) / P(Serial) I am not sure what will come out, but I should try!].
If you want a good summary of the book, read the review by Andrew I. Daleby (pdf). McGrayne illustrates the "recent" history of statistics and probabilities through famous (Laplace) and less famous (Bayes) scientists, through famous (the Enigma machine and Alan Turing) and less famous (lost nuclear bombs) stories and it is a fascinating book. I am not convinced it is great at explaining the science, but the story telling is great. Indeed, it may not be about science at all. But about belief as is mentioned in the book: Swinburne inserted personal opinions into both the prior hunch and the supposedly objective data of Bayes' theorem to conclude that God was more than 50% likely to exist; later Swinburne would figure the probability of Jesus' resurrection at "something like 97 percent" [Page 177]. It obviously reminded me of Einstein's famous quote: "God does not play dice with the universe." This is not directly related but for the second time in my life, I was reading about links between science, probability and religion.
I was never really comfortable with its applications. I was probably wrong again, given all what I learnt after reading Sharon Bertsch McGrayne's rich book. But I also understood why I was never comfortable: for three centuries, there's been a quasi-religious war between Bayesians and Frequentists on how to use probabilities. Are these linked to big, frequent numbers only or can they be applied for rare events? What is the probability of a rare event which may never occur or maybe just once? [Let me give you a personal example: I am interested in serial entrepreneurship, and did and still do tons of statistics on Stanford-related companies. I have more than 5'000 entrepreneurs, and more than 1'000 are serial. I have results showing that serial entrepeneurs are not on average better than one-time, using frequency and classical methods. But now I should think about using: P(Success|Serial) = P(Serial|Sucess) P(Success) / P(Serial) I am not sure what will come out, but I should try!].
If you want a good summary of the book, read the review by Andrew I. Daleby (pdf). McGrayne illustrates the "recent" history of statistics and probabilities through famous (Laplace) and less famous (Bayes) scientists, through famous (the Enigma machine and Alan Turing) and less famous (lost nuclear bombs) stories and it is a fascinating book. I am not convinced it is great at explaining the science, but the story telling is great. Indeed, it may not be about science at all. But about belief as is mentioned in the book: Swinburne inserted personal opinions into both the prior hunch and the supposedly objective data of Bayes' theorem to conclude that God was more than 50% likely to exist; later Swinburne would figure the probability of Jesus' resurrection at "something like 97 percent" [Page 177]. It obviously reminded me of Einstein's famous quote: "God does not play dice with the universe." This is not directly related but for the second time in my life, I was reading about links between science, probability and religion.
July 12, 2012
First, a cachet: unless you are already interested in one or more of Statistics, Decision Theory, Machine Learning, or the history and philosophy of Science and Mathematics, you are probably not a member of this book's audience. However, if you are, you will find a meticulously researched, erudite, and detailed survey of the history of statistics and decision theory. Undergraduate level familiarity with statistics and a generalist understanding of Bayes' rule would be very helpful but not critical to enjoying the book and understanding the majority of its content (or at least its context in the narrative). The book's thematic focus is, obviously, Bayes' rule, but it touches on a whole panoply of ways in which science and mathematics have evolved into tools for everyday decision-making---including many of the roots of the modern world. Of particular interest is the tension between theory and practice, and how the Rule found its niche amongst those just trying to get work done, even while it was being ignored or rejected by pure theoreticians. A worthy read for anyone interested in these areas.
My only complaints are ones of emphasis, and might be more a matter of taste than anything else: I wish the author had been clearer about how what "Bayes' Rule" means has evolved and the fact that people who objected to it in the 19th century and those that objected to it in the middle part of the 20th were not necessarily objecting to the same thing. She also could have been clearer that what is "Baysian" has (due to its current fashionability) has broadened to the point that Bayes and LaPlace wouldn't recognize it, and that many so-called "Bayesian" techniques are strongly influenced by things that Frequentists would have had no problems with. Objecting to arbitrary priors isn't the same thing as objecting to many of the things that are now called Bayesian techniques. Finally, I wish she had been a little clearer that Bayes' value wasn't strongly evident until the ubiquity of cheap computational resources and the familiarity with the types of problems that large, complex systems generate became widespread in the theoretical community. Both of these trends had to wait until the later part of the 20th century, and so, frankly, did widespread acceptance of probabilistic reasoning under uncertainty.
My only complaints are ones of emphasis, and might be more a matter of taste than anything else: I wish the author had been clearer about how what "Bayes' Rule" means has evolved and the fact that people who objected to it in the 19th century and those that objected to it in the middle part of the 20th were not necessarily objecting to the same thing. She also could have been clearer that what is "Baysian" has (due to its current fashionability) has broadened to the point that Bayes and LaPlace wouldn't recognize it, and that many so-called "Bayesian" techniques are strongly influenced by things that Frequentists would have had no problems with. Objecting to arbitrary priors isn't the same thing as objecting to many of the things that are now called Bayesian techniques. Finally, I wish she had been a little clearer that Bayes' value wasn't strongly evident until the ubiquity of cheap computational resources and the familiarity with the types of problems that large, complex systems generate became widespread in the theoretical community. Both of these trends had to wait until the later part of the 20th century, and so, frankly, did widespread acceptance of probabilistic reasoning under uncertainty.
February 10, 2012
A friend recently pointed out that the term 'Bayesian' is now entering the common parlance such that the NY Times can use it in an article without explanation. This would come as a huge surprise and disappointment to many statisticians from the early part of the 20th century, when Bayesian was a bad word and the theory was largely refuted. Why would a statistical theory be so upsetting, you might ask...well, McGrayne's book explains why, and gives the history of the theory over time and how it managed to win out over many many objections.
As with many pop math books, I wish there was more math, but I also understand why McGrayne made the choices she did to keep the book light on technical details and heavy on the historical anecdotes. She does a wonderful job of telling a very interesting story and one that more people should know.
As with many pop math books, I wish there was more math, but I also understand why McGrayne made the choices she did to keep the book light on technical details and heavy on the historical anecdotes. She does a wonderful job of telling a very interesting story and one that more people should know.
September 7, 2019
Maybe I should stop rating books. I give this a somewhat tepid 4 stars. I enjoyed parts 1, 2, and 5, but found the story to drag at times in parts 3 and 4 (exceptions for the sections on smoking deaths and searching for nuclear weapons). I think that the Signal and the Noise does a better job of showing and actually explaining Bayes theorem, but it's also a much longer book and has room to do that.
March 27, 2021
Really an interesting, unusual book that combines history, math, and decision theory into a discussion about several of the most useful ideas in probability theory. Although a bit preachy at times, and clearly of the perspective that frequentists slowed down several fields of science, I think it tells the tale of the evolution of probability theory very well -- ironically the whole field, not just the Bayesian aspects of probability.
I think McGrayne does a really good job of putting into context the development of probability as a branch of mathematics, with the usual mix of progress being driven by practical problems that need solving, theoretical insights, persistent work, the availability of new tools (especially in this case, computers), and, inevitably, the personalities of the people involved. She highlights the names I knew, and gives attention to some figures that were obscure to me.
Particularly intriguing to me is her speculation that the classified nature of much work on applying Bayes theorem in the twentieth century -- because it proved so useful in cryptography and military operations research -- sustained badly-argued philosophical debates within mathematics and science, and did so specifically because the development of practical Bayesian solutions and improvement of technique could not be shared with the broader scientific community. She makes an intriguing case that this secrecy was the reason the world seemed for a time, in the middle of the last century, to be divided into warring religions of Bayesians and Frequentists.
As a pragmatist myself, I rarely believe that philosophical debates affect the proper role of mathematics. Like Bacon, I believe it's only knowledge if it is useful, and, obviously, both the frequentists and the Bayesians produced tools that are useful. I will admit, though, that I didn't know how many of the tools of decision theory and hypothesis testing that I regularly employ, and even teach, came out of this debate, nor how my approach to work and teaching would likely have been labeled Bayesian if I were in the last century. To me, Bayes is the only practical tool for many problems ... but, as I like to point out to students, in many situations you can get the same answer from either approach.
The book isn't quite a five-star book for me because of two things. First, I really think that the book overdoes a sense of heroes and villains in this story: these mathematicians, theoretical and applied, were largely trying to establish a rigorous basis for solving problems, and I think their differences and passion, on both sides, is largely to be admired; and I think that the fact that they went too far in ostracizing their rivals is just a sign of human failings more than villainy. Second, I wish there had been more math and references, particularly in the last chapters where she begins to show the triumphant application of Bayesian analysis in an age of computing power. I found that she was describing the current synthesis of probability and decision analysis that I came of age in, and I would have liked to seen more detail there. Whole chapters on LaPlace may be justified, but cramming all the great inventions of the 1980s into a few paragraphs seems unbalanced to me.
I think McGrayne does a really good job of putting into context the development of probability as a branch of mathematics, with the usual mix of progress being driven by practical problems that need solving, theoretical insights, persistent work, the availability of new tools (especially in this case, computers), and, inevitably, the personalities of the people involved. She highlights the names I knew, and gives attention to some figures that were obscure to me.
Particularly intriguing to me is her speculation that the classified nature of much work on applying Bayes theorem in the twentieth century -- because it proved so useful in cryptography and military operations research -- sustained badly-argued philosophical debates within mathematics and science, and did so specifically because the development of practical Bayesian solutions and improvement of technique could not be shared with the broader scientific community. She makes an intriguing case that this secrecy was the reason the world seemed for a time, in the middle of the last century, to be divided into warring religions of Bayesians and Frequentists.
As a pragmatist myself, I rarely believe that philosophical debates affect the proper role of mathematics. Like Bacon, I believe it's only knowledge if it is useful, and, obviously, both the frequentists and the Bayesians produced tools that are useful. I will admit, though, that I didn't know how many of the tools of decision theory and hypothesis testing that I regularly employ, and even teach, came out of this debate, nor how my approach to work and teaching would likely have been labeled Bayesian if I were in the last century. To me, Bayes is the only practical tool for many problems ... but, as I like to point out to students, in many situations you can get the same answer from either approach.
The book isn't quite a five-star book for me because of two things. First, I really think that the book overdoes a sense of heroes and villains in this story: these mathematicians, theoretical and applied, were largely trying to establish a rigorous basis for solving problems, and I think their differences and passion, on both sides, is largely to be admired; and I think that the fact that they went too far in ostracizing their rivals is just a sign of human failings more than villainy. Second, I wish there had been more math and references, particularly in the last chapters where she begins to show the triumphant application of Bayesian analysis in an age of computing power. I found that she was describing the current synthesis of probability and decision analysis that I came of age in, and I would have liked to seen more detail there. Whole chapters on LaPlace may be justified, but cramming all the great inventions of the 1980s into a few paragraphs seems unbalanced to me.
April 5, 2021
My reading speed asymptotically approached zero as this book progressed. Alas!
A history of science book, this covers the history of Bayesian statistics - from its origin in 18th century England via Thomas Bayes, to its second origin in 18th century France with Laplace, to its waxing and waning in popularity throughout the 20th century, and eventual redemption thanks to increased computational speed/processing power. A lot of the 20th century stuff seemed to be directly taken from interviews, so the book has an oral history quality to it. And it's certainly written in a lively, impassioned way.
But! It's also frustratingly vague of what "Bayesianism" actually IS (we do see Bayes' Rule once, early on in the Laplace chapter, and we have a lot of discussions of how people get upset about priors). But for a science book, I feel like it fails to convey the core insight of what makes Bayesian stats interesting. We talk a lot ABOUT Bayesian stats, but unlike e.g. Michio Kaku re: fancy physics (and maybe physicists will balk), or Emily Oster (or Esther Duflo and Abhijit Banerjee) and economics, or Michael Lewis and economics/anything, this book doesn't really inform the reader on the core idea. And I get it! It's hard. And I can't even pinpoint specifically HOW McGrayne doesn't get it right... Like, she definitely does spend a lot of time discussing priors, uncertainty, beliefs/probabilities, and frequentist's contrasting ideas of multiple samples (and how who has time/money to do _that_?!). But it almost feels like she spends a lot of time discussing these topics without ever first defining them in great depth.
Similarly, another thing which started to really grate on me was the long, comma-separated lists of applications of Bayesian stats (lists and lists and lists), as well as McGrayne's tendency to introduce numerous people in the same way: Robert ("Bob") Smith, who received a PhD from Ivy League University, blah blah. Do I need to know he went by Bob? Also, these appeals to authority (PhD from Ivy League!) definitely worked on me in the beginning ("wow, Bayes is so mathy, these people are so well-qualified"), but then I started to be like, "wait, why do I need to know Bob goes by Bob and went to Yale if I'm never going to hear from him again?"
I was also was a little miffed by the relatively cursory mention of Bayes's history in economics ("oh yeah, Tversky and Kahneman won the Nobel for showing people aren't super rational"), while we spent - relatively - soooooo much time with military applications. And I mean, I like the Navy! I like to read about naval things! But I also became enamored with Bayes, like a lot of people in my classes, when going through the econ journey of game theory, risk and uncertainty, decision making under risk and uncertainty, and finally behavioral economics. This is a wonderful journey! A very interesting realm of study! So yeah, I was miffed that in a big book about Bayes, we spent so much time at Harvard Business School and the Navy, and so little time with, indeed, Kahneman and Tversky.
A history of science book, this covers the history of Bayesian statistics - from its origin in 18th century England via Thomas Bayes, to its second origin in 18th century France with Laplace, to its waxing and waning in popularity throughout the 20th century, and eventual redemption thanks to increased computational speed/processing power. A lot of the 20th century stuff seemed to be directly taken from interviews, so the book has an oral history quality to it. And it's certainly written in a lively, impassioned way.
But! It's also frustratingly vague of what "Bayesianism" actually IS (we do see Bayes' Rule once, early on in the Laplace chapter, and we have a lot of discussions of how people get upset about priors). But for a science book, I feel like it fails to convey the core insight of what makes Bayesian stats interesting. We talk a lot ABOUT Bayesian stats, but unlike e.g. Michio Kaku re: fancy physics (and maybe physicists will balk), or Emily Oster (or Esther Duflo and Abhijit Banerjee) and economics, or Michael Lewis and economics/anything, this book doesn't really inform the reader on the core idea. And I get it! It's hard. And I can't even pinpoint specifically HOW McGrayne doesn't get it right... Like, she definitely does spend a lot of time discussing priors, uncertainty, beliefs/probabilities, and frequentist's contrasting ideas of multiple samples (and how who has time/money to do _that_?!). But it almost feels like she spends a lot of time discussing these topics without ever first defining them in great depth.
Similarly, another thing which started to really grate on me was the long, comma-separated lists of applications of Bayesian stats (lists and lists and lists), as well as McGrayne's tendency to introduce numerous people in the same way: Robert ("Bob") Smith, who received a PhD from Ivy League University, blah blah. Do I need to know he went by Bob? Also, these appeals to authority (PhD from Ivy League!) definitely worked on me in the beginning ("wow, Bayes is so mathy, these people are so well-qualified"), but then I started to be like, "wait, why do I need to know Bob goes by Bob and went to Yale if I'm never going to hear from him again?"
I was also was a little miffed by the relatively cursory mention of Bayes's history in economics ("oh yeah, Tversky and Kahneman won the Nobel for showing people aren't super rational"), while we spent - relatively - soooooo much time with military applications. And I mean, I like the Navy! I like to read about naval things! But I also became enamored with Bayes, like a lot of people in my classes, when going through the econ journey of game theory, risk and uncertainty, decision making under risk and uncertainty, and finally behavioral economics. This is a wonderful journey! A very interesting realm of study! So yeah, I was miffed that in a big book about Bayes, we spent so much time at Harvard Business School and the Navy, and so little time with, indeed, Kahneman and Tversky.
January 2, 2023
Underwhelming. The book does a poor job of explaining *how* Bayes' theorem was actually used in practice to accomplish the many breakthroughs outlined in the book. I think the author tried to avoid getting too technical, but this made it harder to understand how a frequentist vs a Bayesian would tackle the same problem; it actually requires some math and technical detail to explain this.
March 14, 2016
Impressive research, an enjoyable read about the history of Bayesian statistics. However, lacks a good description about the basic question � what is Bayesian statistics. The book would benefit from a first chapter with a couple engaging examples of how Bayes works. Must be able to last through the first couple dry chapters.
Bayes helps real-life practitioners assess evidence, combine every possible form of information, and cope with the gaps and uncertainties in their knowledge. Some of Bayes� powers is that it allows a researcher to stop running an experiment when data was sufficient, monitor interim results, examine effects on subgroups or follow leads from the data with further unplanned analyses. It helps scientists with scanty data and computing probabilities of highly unlikely events.
Frequentists methods work for repetitive, standardized phenomena, while Bayes incorporates a variety of sources of information. Decisionmakers often want to look at more than one alternative not merely accept or reject a hypothesis. It is useful for decisionmaking, helpful because it allows comparing multiple models. Bayes then expresses answers in terms of probabilities.
Bayes vulnerability is that its magical powers depend on the validity of its probabilistic inputs. There are the computational difficulty of Bayes � as the number of unknowns rise, integration problems become difficult. Recent advances in the combination of Bayes and MCMC (Markov chain Monte Carlo) has been called arguably the most powerful mechanism ever created for processing data and knowledge. I got the sense that this replaces integration with approximation techniques through series.
The book has a favorite quote about statistics�. "no one has ever claimed that statistics was the queen of the sciences..The best alternative that has occurred to me is bedfellow. Statistics � bedfellow of the sciences.� Ends with conclusion that there has been a truce between frequentists and Bayesian and most modern Bayesians accept that the frequentist approach is still effective for most statistical problems.
Bayes helps real-life practitioners assess evidence, combine every possible form of information, and cope with the gaps and uncertainties in their knowledge. Some of Bayes� powers is that it allows a researcher to stop running an experiment when data was sufficient, monitor interim results, examine effects on subgroups or follow leads from the data with further unplanned analyses. It helps scientists with scanty data and computing probabilities of highly unlikely events.
Frequentists methods work for repetitive, standardized phenomena, while Bayes incorporates a variety of sources of information. Decisionmakers often want to look at more than one alternative not merely accept or reject a hypothesis. It is useful for decisionmaking, helpful because it allows comparing multiple models. Bayes then expresses answers in terms of probabilities.
Bayes vulnerability is that its magical powers depend on the validity of its probabilistic inputs. There are the computational difficulty of Bayes � as the number of unknowns rise, integration problems become difficult. Recent advances in the combination of Bayes and MCMC (Markov chain Monte Carlo) has been called arguably the most powerful mechanism ever created for processing data and knowledge. I got the sense that this replaces integration with approximation techniques through series.
The book has a favorite quote about statistics�. "no one has ever claimed that statistics was the queen of the sciences..The best alternative that has occurred to me is bedfellow. Statistics � bedfellow of the sciences.� Ends with conclusion that there has been a truce between frequentists and Bayesian and most modern Bayesians accept that the frequentist approach is still effective for most statistical problems.
October 28, 2017
I initially thought this book would be more scientific and talk about how Bayes Theorem helped solve various problems throughout history. However, this was more of a history book, which is not something that I have ever read as history just isn't something I've ever been interested in. Anyway, I understand that the book was on the side of Bayes' Theorem but I felt as though the author made those who study traditional statistics (not Bayesian stats) were the bad guys and she spent a lot of time, trashing the character of prominent figures in statistics because they were against Bayes' Theorem. I did not pick up this book for that. So I'm putting this down at the halfway mark because I just don't think I can continue with this book.
July 5, 2020
This book was interesting, but also a bit odd. As I see it, if someone thinks a book on the history of Bayes� theorem sounds interesting, then this person is most likely already familiar with Bayes. Yet, it seems like this book was written for an audience with little or no knowledge of Bayes, which means that a lot of the “technical� details were left out. You are presented with interesting stories about how someone used Bayes to, for example, figure out where a bomb was accidentally dropped in the ocean, or to calculate the probability of a nuclear accident, but you are not told HOW. It just seems odd to have a target audience that wants to know when and in what kinds of situations Bayes have previously been used, yet can’t be bothered about how this was done.
January 7, 2012
Covers the history of Bayes' rule but there's little on actual mathematical application. The book was heavy on mathematician drama and light on data-driven aha! moments. If you are interested mainly in History of Mathematics, raise this review to 3.5 stars. But the book's subtitle is misleading: rather than "_how_ Bayes' rule cracked the Enigma code..." it should read "_that_ Bayes' rule cracked the Enigma code..." since there's little "how" to be had. An interesting piece of history, but ultimately unfulfilling for readers hoping to apply its precepts directly to their own work.
November 18, 2022
This is an accessible though somewhat discursive history of the development of Bayesian thought, especially in the context of its intellectual battle royale with "frequentism." However, a reader should familiarize themselves with these statistical approaches, especially the actual math, before reading this work. It will be more enriching that way. The work would benefit from a deeper focus on the actual statistical ideas even considering that the work targets a lay audience.
September 2, 2013
My following review on "The Theory That would Not Die" is not just a review of the book, but also a detailed account of the actual Bayesian theory and its numerous benefits that unfortunately still remain hidden from general public. I hope that by sharing the importance of this great piece of theory it might kindle your interest to learn more and read this delightful book by Sharon Bertsch McGrayne. To understand the importance and the concept of Bayes Theorem it’s important to establish its need & importance and how it differs from its contemporary science of statistics.
Hope keeps us alive. There's never a time in our life when we are not hoping, expecting or anticipating. Although we can control our actions, we cannot yet control our outcomes. The reason is simple, for we live in an interconnected world where an event is not just the result of a single factor, but interplay of several internal/external factors that are connected to each other in a complex network of relationships.
You sleep hoping that your alarm will buzz at the right time in morning, you wake up hoping that hot water is running in your shower and you will be ready by your daily scheduled time, you enter your car hoping that it will start smoothly and you will have enough gas so as to not make you late to office, you reach your office parking hoping to find that precious spot you always like to park your car, you reach your desk and hope not to see any high priority emails and have a smooth sailing at office all day, you have your performance review with your manager today and hope that you will get that long awaited raise and credit for your hard work, you then check the ebay website hoping that the diamond ring you plan to gift your spouse is still at the same price and within your expected budget, you head out for lunch and feel tempted to eat your favorite chicken cheese sandwich hoping that you will control your diet for the next one week to offset for the sandwich. Just within one day you have hoped, guessed, assumed, predicted your future, and you didn’t even realize it. It all went without much thought for most decisions were made on intuition that has been toned through repetition and similar experiences in the past. So, even after being part of such an intricate web of factors that influence our daily lives, we humans have not only survived but have prospered, progressed and have become more skilled and adaptive to the world around us. Though there's still a lot that we don't know about the world and what we do know is akin to scratching the surface, yet our ignorance hasn't deterred or stopped us from performing our actions.
How did we do that? With time we have learned to assign value to every action & non-action and thus continued to live our life by making frequent choices, predictions, decisions and assumptions. The next obvious question that arises is - "How did we compute and assign values to our actions/non actions and measure their effectiveness to an event?" The answer is a single word: "Statistics". This mathematical computation method of predicting future (i.e. developing forecasts) requires at least 2 prerequisites to be met:
1. There is enough past historical data available on the actions and the effects they caused (i.e. Lessons learned)
2. Each action is repeatable and will consistently reproduce the same level of potency every-time it’s performed (i.e. Reliability and Frequency)
Statistics works well with events that are known, common-place and recurring in nature.
Paradoxically events that can be predicted using statistics aren't catastrophic, as events that cannot be predicted using statistics are mostly the ones that are dangerously potent and catastrophic. In addition, many of the non-predictable events are of an unimaginable nature and hard even to be conceived.
So how do we prevent or defend ourselves from such events? To find a solution for such a problem, the first step would be to understand, dissect and state the problem clearly. Upon further analysis it can be divided into a 2-fold problem:
1. The first and high priority problem - To find a solution for predicting the occurrence of an event (Example: Columbia Shuttle Accident)
2. The second and medium priority problem - To establish a standard procedure to control the damage (i.e. mitigate the risks) in the aftermath of an event (Example: The biggest and longest search effort that US government undertook during the crisis of the missing nuclear war head supposed to be lost when flying above the Atlantic Ocean).
As standard statistical forecasting cannot help in such situations, this kind of a problem requires qualitative forecasting. Such a qualitative model was first introduced by Thomas Bayes in the 18th century which is capable of predicting the rarest of events that have negligible data available.
My interest in Bayes theory arose in part from my curiosity to learn new mathematical prediction models at work. As an Strategy Consultant I frequently make recommendations to organizations on new strategies, changes to their business process, adopting new tools & technologies, in order to optimize their operations by reducing - waste, costs; to gain more market share, to increase revenue, and to grow their business organically and in-organically. I use Bayes at a strategic-level to assess the potential of the short-listed strategies and find the ones that have the highest potential of being successful in future, and also at tactical and operational level. In addition, many organizations themselves use Bayes tools / software’s for their daily operations, such as to predict future sales of their products. Predicting Sales of products might seem easy if it’s a small corner store, but today's corner store has been replaced by supermarkets that sell products from far corners of the world and not surprisingly will surely carry that particular model of Nike shoes you just saw someone wearing today morning, even before you would have realized the need for that new Nike shoe. We take it for granted but for that supermarket to stock that specific Nike shoe for your size took a lot of planning and dollars. Without getting in to the details if we just took a superficial look at the stores intricate web of supply chain network we can see that it would have hundreds of similar stores around the nation, with each store serving thousands of customers, and each stores sells millions of products most of which get replaced by new products within weeks or at the most couple of months. In such a volatile, intricate, high-competition, short product life-cycle environment a single stock-out means not only the loss of that one sale but more probably loss of future revenue from that customer, which means the actual loss is several times the price of that one shoe. But, there are solutions such as Oracles Demantra Forecasting solution that uses Bayesian-Markov model to predict future sales to the most detailed-level and even without the need of any historical data such as for newly launched products that do not have any past sales data. In today’s global economy product life cycles are shorter than ever and be it any kind of product (new smartphone or car), and by the time you decide to buy the product its next generation would have hit the market.
Sharon’s book provides an excellent biography of Bayes who is well-known for being unknown and how Bayes theory got rejuvenated centuries after his death. It’s an interesting and inspirational story of a theory that most assumed to be dead-weight and yet arose to heights that even its originator Thomas Bayes or its biggest advocated Laplace wouldn’t have imagined. The Reverend Thomas Bayes lived during the early period of 1700s in England and was a Presbyterian minister, amateur mathematician and he developed the Bayes rule for forecasting & predicting events that are unknown, since they are events yet to occur in future or have already occurred but remain unknown in their nature. The irony of his life has been that his reputation as a mathematics genius came to light centuries after he died. On the other side were those for whom probability was the objective study of relative frequency, dubbed as the 'Frequentists' or 'Statisticians', they thought Bayes was subjectivity gone mad. As such, the Bayesian Theorem had a painfully slow rate of acceptance amongst academicians and scientists, since there was more skepticism than confidence in Bayes theory.
His story though an irony has its own interesting features, the very theorem called as Bayes Rule was discovered by Bayes during an inflammatory religious controversy launched by the Scottish philosopher David Hume. The controversy was on the question of whether scientists or others could use evidence about the real world to make rational conclusions about God, the creator. It was called as “God, the cause - God the primary cause or just the cause�. Though it’s not clear whether Bayes wanted to prove the existence of God - the cause, but one thing is sure that he tried to deal with the “cause and effect� issue mathematically, and in doing so produced the simple one line theorem, that sets the initial idea and indeed is the core concept of solving any prediction problem using Bayes Theorem. In trying to answer the God question mathematically Bayes stated that if you don't have enough reason to guess one way or the other, guess 50/50. It seems surprising but Bayes actually used the word "guess". On the face of it the solution of 50/50 looks like an obvious derivation of the problem itself, but the beauty and simplicity of Bayes lies in the second step of the solution, wherein Bayes went further with this simplistic approach and suggested that having applied the appropriate weights to the outcomes and having created the baseline idea in the first step, the moment we receive any new objective information about the problem we immediately commit to modifying the initial idea (i.e. modifying the weights to the outcomes) based on the new information that’s received. This looks simple enough but the really tough part is changing our mind in the face of fresh new data. Unfortunately, Bayes himself wasn’t convinced enough about his theorem to publish it and filed away this theorem in his notebook and died a decade later without adding any further analysis or research to it.
Pierre-Simon Laplace derived what we today call Bayes's Rule independently � and then for the next two centuries, whether cracking Enigma codes, locating lost submarines or finding the causes of cancer, scientists and statisticians have fought over a deep philosophical divide about probability, which Sharon Bertsch McGrayne explores with great clarity and wit.
I found the book entertaining as Simon not only presents Bayesian theory but personifies the originator of the theory to its utmost and rightfully deserved glory, giving it a flavor of an inspirational and motivational novel, where good triumphs evil although a bit late and the world is able to see and leverage the true value of a hidden treasure in the form of Bayes Theory. To many it would feel like a philosophical discourse, and you wouldn’t be far away from the truth, since Bayes Theory is much philosophy than science and when two people solve a same problem using Bayes Theory they can very well end up with different answers. This doesn’t mean that the theory is flawed rather it shows the sensitivity and breadth of the solution which gets impacted even with slightest of wrong assessment of the information retrieved during the problem-solving period.
She reveals why respected academics rendered it professionally taboo for decades � while decision-makers relied on it to solve crisis involving great uncertainty and scanty information. A prime example is how Alan Turing used Bayes to break the German U-boats� Enigma code during World War II. She also explains how the advent of off-the-shelf computer technology in the 1980s was a game-changer. Today, Bayes� rule is used everywhere, from DNA decoding to Homeland Security.
Bertsch McGrayne is an entertaining writer. She makes this obscure theory come alive and presents the battle of wits as a gripping historical tale with her easy to read style.
Hope keeps us alive. There's never a time in our life when we are not hoping, expecting or anticipating. Although we can control our actions, we cannot yet control our outcomes. The reason is simple, for we live in an interconnected world where an event is not just the result of a single factor, but interplay of several internal/external factors that are connected to each other in a complex network of relationships.
You sleep hoping that your alarm will buzz at the right time in morning, you wake up hoping that hot water is running in your shower and you will be ready by your daily scheduled time, you enter your car hoping that it will start smoothly and you will have enough gas so as to not make you late to office, you reach your office parking hoping to find that precious spot you always like to park your car, you reach your desk and hope not to see any high priority emails and have a smooth sailing at office all day, you have your performance review with your manager today and hope that you will get that long awaited raise and credit for your hard work, you then check the ebay website hoping that the diamond ring you plan to gift your spouse is still at the same price and within your expected budget, you head out for lunch and feel tempted to eat your favorite chicken cheese sandwich hoping that you will control your diet for the next one week to offset for the sandwich. Just within one day you have hoped, guessed, assumed, predicted your future, and you didn’t even realize it. It all went without much thought for most decisions were made on intuition that has been toned through repetition and similar experiences in the past. So, even after being part of such an intricate web of factors that influence our daily lives, we humans have not only survived but have prospered, progressed and have become more skilled and adaptive to the world around us. Though there's still a lot that we don't know about the world and what we do know is akin to scratching the surface, yet our ignorance hasn't deterred or stopped us from performing our actions.
How did we do that? With time we have learned to assign value to every action & non-action and thus continued to live our life by making frequent choices, predictions, decisions and assumptions. The next obvious question that arises is - "How did we compute and assign values to our actions/non actions and measure their effectiveness to an event?" The answer is a single word: "Statistics". This mathematical computation method of predicting future (i.e. developing forecasts) requires at least 2 prerequisites to be met:
1. There is enough past historical data available on the actions and the effects they caused (i.e. Lessons learned)
2. Each action is repeatable and will consistently reproduce the same level of potency every-time it’s performed (i.e. Reliability and Frequency)
Statistics works well with events that are known, common-place and recurring in nature.
Paradoxically events that can be predicted using statistics aren't catastrophic, as events that cannot be predicted using statistics are mostly the ones that are dangerously potent and catastrophic. In addition, many of the non-predictable events are of an unimaginable nature and hard even to be conceived.
So how do we prevent or defend ourselves from such events? To find a solution for such a problem, the first step would be to understand, dissect and state the problem clearly. Upon further analysis it can be divided into a 2-fold problem:
1. The first and high priority problem - To find a solution for predicting the occurrence of an event (Example: Columbia Shuttle Accident)
2. The second and medium priority problem - To establish a standard procedure to control the damage (i.e. mitigate the risks) in the aftermath of an event (Example: The biggest and longest search effort that US government undertook during the crisis of the missing nuclear war head supposed to be lost when flying above the Atlantic Ocean).
As standard statistical forecasting cannot help in such situations, this kind of a problem requires qualitative forecasting. Such a qualitative model was first introduced by Thomas Bayes in the 18th century which is capable of predicting the rarest of events that have negligible data available.
My interest in Bayes theory arose in part from my curiosity to learn new mathematical prediction models at work. As an Strategy Consultant I frequently make recommendations to organizations on new strategies, changes to their business process, adopting new tools & technologies, in order to optimize their operations by reducing - waste, costs; to gain more market share, to increase revenue, and to grow their business organically and in-organically. I use Bayes at a strategic-level to assess the potential of the short-listed strategies and find the ones that have the highest potential of being successful in future, and also at tactical and operational level. In addition, many organizations themselves use Bayes tools / software’s for their daily operations, such as to predict future sales of their products. Predicting Sales of products might seem easy if it’s a small corner store, but today's corner store has been replaced by supermarkets that sell products from far corners of the world and not surprisingly will surely carry that particular model of Nike shoes you just saw someone wearing today morning, even before you would have realized the need for that new Nike shoe. We take it for granted but for that supermarket to stock that specific Nike shoe for your size took a lot of planning and dollars. Without getting in to the details if we just took a superficial look at the stores intricate web of supply chain network we can see that it would have hundreds of similar stores around the nation, with each store serving thousands of customers, and each stores sells millions of products most of which get replaced by new products within weeks or at the most couple of months. In such a volatile, intricate, high-competition, short product life-cycle environment a single stock-out means not only the loss of that one sale but more probably loss of future revenue from that customer, which means the actual loss is several times the price of that one shoe. But, there are solutions such as Oracles Demantra Forecasting solution that uses Bayesian-Markov model to predict future sales to the most detailed-level and even without the need of any historical data such as for newly launched products that do not have any past sales data. In today’s global economy product life cycles are shorter than ever and be it any kind of product (new smartphone or car), and by the time you decide to buy the product its next generation would have hit the market.
Sharon’s book provides an excellent biography of Bayes who is well-known for being unknown and how Bayes theory got rejuvenated centuries after his death. It’s an interesting and inspirational story of a theory that most assumed to be dead-weight and yet arose to heights that even its originator Thomas Bayes or its biggest advocated Laplace wouldn’t have imagined. The Reverend Thomas Bayes lived during the early period of 1700s in England and was a Presbyterian minister, amateur mathematician and he developed the Bayes rule for forecasting & predicting events that are unknown, since they are events yet to occur in future or have already occurred but remain unknown in their nature. The irony of his life has been that his reputation as a mathematics genius came to light centuries after he died. On the other side were those for whom probability was the objective study of relative frequency, dubbed as the 'Frequentists' or 'Statisticians', they thought Bayes was subjectivity gone mad. As such, the Bayesian Theorem had a painfully slow rate of acceptance amongst academicians and scientists, since there was more skepticism than confidence in Bayes theory.
His story though an irony has its own interesting features, the very theorem called as Bayes Rule was discovered by Bayes during an inflammatory religious controversy launched by the Scottish philosopher David Hume. The controversy was on the question of whether scientists or others could use evidence about the real world to make rational conclusions about God, the creator. It was called as “God, the cause - God the primary cause or just the cause�. Though it’s not clear whether Bayes wanted to prove the existence of God - the cause, but one thing is sure that he tried to deal with the “cause and effect� issue mathematically, and in doing so produced the simple one line theorem, that sets the initial idea and indeed is the core concept of solving any prediction problem using Bayes Theorem. In trying to answer the God question mathematically Bayes stated that if you don't have enough reason to guess one way or the other, guess 50/50. It seems surprising but Bayes actually used the word "guess". On the face of it the solution of 50/50 looks like an obvious derivation of the problem itself, but the beauty and simplicity of Bayes lies in the second step of the solution, wherein Bayes went further with this simplistic approach and suggested that having applied the appropriate weights to the outcomes and having created the baseline idea in the first step, the moment we receive any new objective information about the problem we immediately commit to modifying the initial idea (i.e. modifying the weights to the outcomes) based on the new information that’s received. This looks simple enough but the really tough part is changing our mind in the face of fresh new data. Unfortunately, Bayes himself wasn’t convinced enough about his theorem to publish it and filed away this theorem in his notebook and died a decade later without adding any further analysis or research to it.
Pierre-Simon Laplace derived what we today call Bayes's Rule independently � and then for the next two centuries, whether cracking Enigma codes, locating lost submarines or finding the causes of cancer, scientists and statisticians have fought over a deep philosophical divide about probability, which Sharon Bertsch McGrayne explores with great clarity and wit.
I found the book entertaining as Simon not only presents Bayesian theory but personifies the originator of the theory to its utmost and rightfully deserved glory, giving it a flavor of an inspirational and motivational novel, where good triumphs evil although a bit late and the world is able to see and leverage the true value of a hidden treasure in the form of Bayes Theory. To many it would feel like a philosophical discourse, and you wouldn’t be far away from the truth, since Bayes Theory is much philosophy than science and when two people solve a same problem using Bayes Theory they can very well end up with different answers. This doesn’t mean that the theory is flawed rather it shows the sensitivity and breadth of the solution which gets impacted even with slightest of wrong assessment of the information retrieved during the problem-solving period.
She reveals why respected academics rendered it professionally taboo for decades � while decision-makers relied on it to solve crisis involving great uncertainty and scanty information. A prime example is how Alan Turing used Bayes to break the German U-boats� Enigma code during World War II. She also explains how the advent of off-the-shelf computer technology in the 1980s was a game-changer. Today, Bayes� rule is used everywhere, from DNA decoding to Homeland Security.
Bertsch McGrayne is an entertaining writer. She makes this obscure theory come alive and presents the battle of wits as a gripping historical tale with her easy to read style.
October 31, 2019
I like mathematical history books and this book has a lot of behind the scenes stories of how Bayes Rule came to be what it is right now and the statistical community's aversion to using it. What I found really fascinating was how instrumental Laplace was in the development of Bayesian theory, and how Turing applied it during the war. I found the usages mentioned later for finding the nuclear bomb to be a bit of a stretch at the time. Although it has eventually found its use. The book makes me think that since a lot of applications didn't call themselves bayesian and came to the solution from different angles, just painting them retroactively as bayesian isn't a fair assessment.
Broadly speaking this book treats Bayesian methods and inference as the process of taking assumptions of priors and updating beliefs. This was deemed to be too subjective in Rev. Bayes' era by the broad scientific community and I was surprised to find that Bayes' interest comes from David Hume's skeptical essay on the cause and effect relationships, i.e., God being the first mover. This was further championed by Richard Price. Further practical work done by Laplace.
I found Laplace's story and his intersection with french politics to be fascinating, apparently he had launched Napoleon on his military career by giving him a passing grade in military school.
The second world war, and Turing's usage of Bayesian methods in detecting russian subs and protecting the allies ships was as much inspiring as heartbreaking. Despite doing a lot of work that should've garnered him recognition, all the work was classified and he didn't have the status that could've protected him from persecution. The author suggests that had Turing been knighted after the war for his efforts, it would've given him a lot more protection and ability to work on what he wanted to.
A lot of internal strifes within statistical community wasn't that fascinating to me. Partly because it was mostly people outside of the community that saw the practical usages and were able to utilize Bayesian methods to make them popular. The book would likely be incomplete without that and it provides intrigue. But anyone who has worked in scientific community knows that these factions are all too common.
Overall this is a solid book and highly recommend it as a filler for understanding the historical underpinnings for the rise of Bayesian theory.
Broadly speaking this book treats Bayesian methods and inference as the process of taking assumptions of priors and updating beliefs. This was deemed to be too subjective in Rev. Bayes' era by the broad scientific community and I was surprised to find that Bayes' interest comes from David Hume's skeptical essay on the cause and effect relationships, i.e., God being the first mover. This was further championed by Richard Price. Further practical work done by Laplace.
I found Laplace's story and his intersection with french politics to be fascinating, apparently he had launched Napoleon on his military career by giving him a passing grade in military school.
The second world war, and Turing's usage of Bayesian methods in detecting russian subs and protecting the allies ships was as much inspiring as heartbreaking. Despite doing a lot of work that should've garnered him recognition, all the work was classified and he didn't have the status that could've protected him from persecution. The author suggests that had Turing been knighted after the war for his efforts, it would've given him a lot more protection and ability to work on what he wanted to.
A lot of internal strifes within statistical community wasn't that fascinating to me. Partly because it was mostly people outside of the community that saw the practical usages and were able to utilize Bayesian methods to make them popular. The book would likely be incomplete without that and it provides intrigue. But anyone who has worked in scientific community knows that these factions are all too common.
Overall this is a solid book and highly recommend it as a filler for understanding the historical underpinnings for the rise of Bayesian theory.
February 1, 2019
Bayes (including MCMC) had been called “arguably the most powerful mechanism ever created for processing data and knowledge�
“The Theory That Would Not Die� is well written, informative and an engaging read. From its roots in the "principle of inverse probability�, Bayes rule and Laplacean probability now have numerous newer references and more nuanced meanings. Bayesian probabilities, Bayesian statistical inference (or more simply Bayesian inference), probability based statistics, probability of causes, Empirical Bayes, equal priors, and the Bayes factor are all enough to make your head spin like a Monte Carlo casino wheel.
Bayes use of uniform priors and subjectivity changes the way we view the world. Keynes called it “a smack of astrology, of alchemy.� Today, we know better. One of its adherents, Chris Fossenbeck stated emphatically that all of statistics is subjective. Whether its the choice of the frequentist beloved p-values, which test statistic to use, which data to collect, or even in the design of the experiment, we make subjective choices. “There is no such thing as objective statistics, end of story!�
The history of Bayes is one of intrigue and secrets. Its successes emerged from Turing’s cryptography castle, in the hunt for U-2 submarines during WWII and later in the development of the hydrogen bomb. McGrayne runs us seamlessly through history, moving from Laplace, Newton and Leibniz to Karl Pearson, Ronald Fisher and eventually to Turing and von Neumann. The development in 1906 of Markov Chain Monte Carlo (MCMC) models coupled more recently with powerful computers have once again resurrected this irrefragable theory.
“The Theory That Would Not Die� is well written, informative and an engaging read. From its roots in the "principle of inverse probability�, Bayes rule and Laplacean probability now have numerous newer references and more nuanced meanings. Bayesian probabilities, Bayesian statistical inference (or more simply Bayesian inference), probability based statistics, probability of causes, Empirical Bayes, equal priors, and the Bayes factor are all enough to make your head spin like a Monte Carlo casino wheel.
Bayes use of uniform priors and subjectivity changes the way we view the world. Keynes called it “a smack of astrology, of alchemy.� Today, we know better. One of its adherents, Chris Fossenbeck stated emphatically that all of statistics is subjective. Whether its the choice of the frequentist beloved p-values, which test statistic to use, which data to collect, or even in the design of the experiment, we make subjective choices. “There is no such thing as objective statistics, end of story!�
The history of Bayes is one of intrigue and secrets. Its successes emerged from Turing’s cryptography castle, in the hunt for U-2 submarines during WWII and later in the development of the hydrogen bomb. McGrayne runs us seamlessly through history, moving from Laplace, Newton and Leibniz to Karl Pearson, Ronald Fisher and eventually to Turing and von Neumann. The development in 1906 of Markov Chain Monte Carlo (MCMC) models coupled more recently with powerful computers have once again resurrected this irrefragable theory.
February 14, 2021
This is a science history book about the Bayes Theorem, full of interesting details and insights about how human beings applied this equation to almost all areas of human lives.
In the essence, the topic of the book is about a mathematical theory. And usually, any theme related to math will be dry, hard to understand, and average readers will shun away from it. But this book is exceptional. On the contrary, with the extensive research done by the author, the science history of Bayes Theorem is turned into interesting stories with superb and fluent writings from the author. The reading is never boring. And when the reader finishes the book, she or he will understand how this theory was invented, what controversies are associated with it, and what real problems it has solved. From hunting Nazi U-boats to finding the missing American H-bombs, as I said, the stories in the application of Bayes Theorem are never dry and tedious.
The author has done a great job to introduce the full background details about the Bayes equation to readers with minimal statistics knowledge. This is a solid and excellent popular science book for anyone who is interested in Bayes methods. Another thing worth mentioning is, the author's language and vocabulary are unique. They are different from daily newspapers or magazines. Readers can learn the new words but the whole book is still quite enjoyable.
In the essence, the topic of the book is about a mathematical theory. And usually, any theme related to math will be dry, hard to understand, and average readers will shun away from it. But this book is exceptional. On the contrary, with the extensive research done by the author, the science history of Bayes Theorem is turned into interesting stories with superb and fluent writings from the author. The reading is never boring. And when the reader finishes the book, she or he will understand how this theory was invented, what controversies are associated with it, and what real problems it has solved. From hunting Nazi U-boats to finding the missing American H-bombs, as I said, the stories in the application of Bayes Theorem are never dry and tedious.
The author has done a great job to introduce the full background details about the Bayes equation to readers with minimal statistics knowledge. This is a solid and excellent popular science book for anyone who is interested in Bayes methods. Another thing worth mentioning is, the author's language and vocabulary are unique. They are different from daily newspapers or magazines. Readers can learn the new words but the whole book is still quite enjoyable.
December 11, 2021
Disclaimer: I have graduate degrees in mathematics and electrical engineering, in which I studied and used Bayes rule and other tools of statical inference, so perhaps I don’t qualify as the typical reader of a book of this type.
I vacillated between a rating of 3 or 4 but have decided to go with the former. On the one hand, the author gives interesting historical background on Bayes rule that I was not aware of, including Bayes� own formulation, the documentation by Price, and LaPlace’s analysis and definitive quantification. She also presents extensive anecdotes about the use of the rule and the controversies surrounding it.
The issues I have with it are related to the above. First, there are too many anecdotes, without enough real analysis. I’d like to have seen some worked out examples in simple mathematics of how, for example, Bayes was employed to track German U-boats, or whatever. To be fair, there is such an example in one of the appendices.
Secondly, I’m not convinced of the seriousness of the controversy between Bayes rule and the frequentist interpretation, perhaps because I was taught the rule in engineering classes with no mention of a controversy. The author stresses how different the frequentists and Bayesians see the world of statistics, but in my understanding the applications are often different. That is, depending on the application, there may be no conflict in ideology.
In any case, I did learn a lot about the personalities of the people involved, especially Bayes himself, LaPlace, Fischer, Pearson (all of whom I had read in my studies), and others that escape me now. It is worth a read.
I vacillated between a rating of 3 or 4 but have decided to go with the former. On the one hand, the author gives interesting historical background on Bayes rule that I was not aware of, including Bayes� own formulation, the documentation by Price, and LaPlace’s analysis and definitive quantification. She also presents extensive anecdotes about the use of the rule and the controversies surrounding it.
The issues I have with it are related to the above. First, there are too many anecdotes, without enough real analysis. I’d like to have seen some worked out examples in simple mathematics of how, for example, Bayes was employed to track German U-boats, or whatever. To be fair, there is such an example in one of the appendices.
Secondly, I’m not convinced of the seriousness of the controversy between Bayes rule and the frequentist interpretation, perhaps because I was taught the rule in engineering classes with no mention of a controversy. The author stresses how different the frequentists and Bayesians see the world of statistics, but in my understanding the applications are often different. That is, depending on the application, there may be no conflict in ideology.
In any case, I did learn a lot about the personalities of the people involved, especially Bayes himself, LaPlace, Fischer, Pearson (all of whom I had read in my studies), and others that escape me now. It is worth a read.
June 28, 2023
DNF. The problem with this book is one that I have ran across before. When it comes to technical, math-related topics, authors have two choices. They can either stay very high level and not even try to explain the concept (an attempt to not scare people away with actual equations or complex examples), or they can dive into the details, and trust that their writing skills will make the deep dive enjoyable and rewarding. Too many authors take the first option, including this author.
McGrayne's book is about how Bayes Rule was controversial yet very useful, but she does not go deep enough to explain truly why it was controversial and how it was used. She just continually refers to Bayes rule almost like people were using it to cast a spell. Bayes Rule - problem solved! In doing so, all aspects of its genius are eliminated and it just becomes a confusing mess.
I've been introduced to Bayes Rule several times (even in school a long time ago), and picked this book up to get a fuller appreciation. Nope, not here. If this book is enjoyable to anybody, its people who are already well versed in Bayes Rules and just wanted a review of its history.
McGrayne's book is about how Bayes Rule was controversial yet very useful, but she does not go deep enough to explain truly why it was controversial and how it was used. She just continually refers to Bayes rule almost like people were using it to cast a spell. Bayes Rule - problem solved! In doing so, all aspects of its genius are eliminated and it just becomes a confusing mess.
I've been introduced to Bayes Rule several times (even in school a long time ago), and picked this book up to get a fuller appreciation. Nope, not here. If this book is enjoyable to anybody, its people who are already well versed in Bayes Rules and just wanted a review of its history.
December 5, 2021
I really liked the book and it was a quick and fun read. I knew only a bit about Bayesian analysis prior to the book and I didn’t know what exactly bayes rule was. I found some of the anecdotes in the book very fascinating including the part about Turing during ww2 and when they searched for submarines on the bottom of the ocean. Where I felt like the book was lacking was that maybe I just don’t know much about frequentists, but it just railed against them for 200 pages. It’s very possible that they are unequivocally evil and bad, but the author really portrayed them as stubborn and inflexible. Their methods are portrayed as entirely inferior to bayes because they either produce the same results or if less data is available then bayes wins 100% of the time. I dunno I just get skeptical when something is sold that hard to me.
Bayes rule was portrayed as infallible and right in every circumstance. I am very fascinated in it and actually want to study and learn more about it, I just feel like a counter could have given me a greater perspective.
Bayes rule was portrayed as infallible and right in every circumstance. I am very fascinated in it and actually want to study and learn more about it, I just feel like a counter could have given me a greater perspective.
Displaying 1 - 30 of 346 reviews
Can't find what you're looking for?
Get help and learn more about the design.