Frustrating Exchange on Ockham’s Razor

Let me share my pain. In the talk section of the Occam’s Razor entry of Wikipedia a debate was raging between AceMyth and Gkochanowsky regarding the value of Ockham’s Razor to science. AceMyth ended the exchange as follows:

I’d continue this discussion, but running discussions in talk pages in Wikipedia is a time-consuming hobby that consumes time I just don’t have available these days. So I’m swallowing this frog and letting you have the last word, though I’m left with the impression that a lot of what I said has been misunderstood (for one, I think that “actual data from observations” is absolutely necessary for any conclusion outside the domain of philosophy and mathematics, so I’m slightly puzzled by your implication that I ever as much as hinted otherwise.) —AceMyth 16:53, 19 February 2007 (UTC)

Gkochanowsky’s response:

My point is and always has been that if you have data then who needs Ockham’s? As far as I can tell the use of Ockham’s in science is nothing more than a cover for personal preference. But one has to start somewhere so why not start with the explanation one most prefers. But it doesn’t help anyone one bit by pretending that their preference for the “simplest” is based on some lame principle. Gkochanowsky 17:03, 19 February 2007 (UTC)

Deciding to throw my hat in the ring, I wrote the following, within which Gkochanowsky embedded his responses, and so on:

Gkochanowsky, Have you bothered to read the references on the Ockham’s Razor page? There is a rich literature establishing Ockham’s razor as a quite useful methodologically, and not a “lame principle” (See Simplicity at Stanford Encyclopedia of Philosophy, esp.). If you have data, you still have underdetermination of theory by data. Consider the problem of model selection— of the theories (functions, or distributions) compatible with the data (points), you must select one for predictive purposes. Which do you choose and why, if not using OR?

As I have stated before, I have no doubt that it is useful for philosophy. After all, it is not as if philosophers are bothered all that much by data. And in the end the only thing available to philosophers to prefer one philosophy over another is personal preference. Might as well pretend it is for more significant reasons and call it “parsimony”. As for the “theory of model selection” it is mathematics not science. In science the method of model selection is experiment on nature and I hope for the sake of science that it never changes. Gkochanowsky 17:28, 22 February 2007 (UTC)

Wow. you have managed not to address one substantive point and merely repeat yourself, as if that were an justification for your views. Moreover disparaging philosophers as you do throughout your replies is not an argument, but simply a cheap ad hominum distracting form actual debate. It is perhaps useless to address you, but, here it goes…

Never said dispairing philosophers was an argument. Just an explanation for why the lame concept of Ockham’s Razor still lives when it should have been cast onto the junk heap of history along with philogiston. Gkochanowsky 22:06, 22 February 2007 (UTC)

Model selection, in case you are unaware, is not just a philosophical problem. And, again, why is underdetermination not a problem? How do we select from competing, empirically adequate theories, if not (in part) from simplicity? Actual responses to these questions would be helpful. Johnny Logic 18:56, 22 February 2007 (UTC)

Explanation selection is a scientific problem, however modern science figured out how to do that long after Ockham’s and it was not Ockham’s principle that was chosen. I repeat myself over and over again because you insist it was Ockham’s when it is common knowledge that explanations in science are prefered based on how well they do at fidelity and prediction as demonstraited by experiment on nature. Any theory of model selection that does not do this is not science. It may be mathemeatics or philosophy, but science is what it is today because of the criteria of preference of fidelity and prediction. Probablity, simplicity, complexity, Ockhams, parsimony, “Truth” or what ever other philosophical criteria you may want to toss in there has little to do with it.

As for probability and simplicity, that is a complex subject. Suffice to say that the assignment of probabilities need not as you say “know all the possible outcomes and their frequency of occurrence”. Rather, a prior probability can be assigned and true probabilities can converged upon over time (See A Better Bayesian Convergence Theorem for some details).

As I have stated earlier, you can presume, assume, dream, concoct, [place your mechanism of imagination here] or anything you like for a candidate scientific explanation. If you want to impress people with it you can even claim it was come by using Ockham’s razor. But if it is actual science, as opposed to philosophy there is only one way to actually prefer it, and that is to see how well it accounts for known phenomena and how well it predicts phenomena before the fact as compared to competing explanations. Ockham’s has nothing to do with it when it comes to science. Perhaps philosphy but not science. My comments have been directed at Ockham’s purported role in science. I could care less what philosophers do with it. Gkochanowsky 17:28, 22 February 2007 (UTC)

Again you conveniently ignore my point, heap abuse, and restate your assumptions, which are the very point of contention. No modern version of OR as a methodological principle in theory selection is taken to be an a priori acceptance of simple theories– that is a strawman argument. Actually, the balance of simplicity and fit, it has been argued, tends to increase predictive accuracy (See Malcolm Forster’s The New Science of Simplicity for an excellent overview of this argument). Johnny Logic 18:56, 22 February 2007 (UTC)

Do not point out straw man arguments when you insist that Ockham’s razor is to be restated as a principle that William of Ockham never wrote nor would he recognize it as anything like what he wrote. Gkochanowsky 22:06, 22 February 2007 (UTC)

Further, OR does not imply that scientific theories become simpler over time. They only need be as complex as needed to explain the evidence, but no more. The transition from Newtonian physics to Einsteinian physics is an excellent example of a transition from, you might say, a simpler to a more complex theory, necessitated by anomalies.

Gosh, why do you think that happened? Was it because of Ockham’s or because it had nothing to do with Ockham’s? Gee did they go look at the data and compare that with what the explanations predicted? Where was Ockhams? The goal of science has never been to come up with the “simplest” explanation (whatever that is). But to come up with the explanations that best statisified the preferences of science. Which is not simplicity, but fidelity and predictablity before the fact. Science is very pleased if it can satisfy those criteria in any way it can. For science simplicity will just have to take care of itself. It is hard enough satisfying fidelity and predictablity. Or to quote Feynman who was an actual scientist, not some presumptious philosopher, “Shutup and calculate.” Gkochanowsky 17:28, 22 February 2007 (UTC)

This “presumptious [sic] philosopher” would like your arguments rather than your categorical assertions about the nature of science. OR works in conjunction with these other goals, many philosophers and scientists have argued. Johnny Logic 18:56, 22 February 2007 (UTC)

And many have argued that they do not. So what? Feynman is a notable modern example. If you think I have been hard on philosophy you should find out what he thought. Gkochanowsky 22:06, 22 February 2007 (UTC)

True, there are competing definitions of simplicity, and even a tenable rejection of the existence of a language invariant notion of simplicity, but this plurality does not imply that simplicity has no role in inductive inference. I guess my main point is that while I see your snipes, I do not see your arguments. Johnny Logic 18:32, 20 February 2007 (UTC)

It may indeed have a role in inductive enterprises, but so what? Science is not restricted to induction, and inductive conclusions backed by Ockhams are not acceptable reasons to prefer a conclusion by any scientist. Gkochanowsky 22:06, 22 February 2007 (UTC)

Who claimed that science is restricted to induction and that OR alone is sufficient for theory-choice? This is a non sequitur.Johnny Logic 18:56, 22 February 2007 (UTC)

I didn’t bring up induction. You did. Gkochanowsky 22:06, 22 February 2007 (UTC)

The final arbiter is reality itself, not some lame philosophical principle. Gkochanowsky 17:28, 22 February 2007 (UTC)

You have had your sarcasm, allow me mine: Oh, I see. Ockham’s razor is a “lame philosophical principle” and reality will sort it out. QED. Pack your bags philosophers, statisticians, machine learners, and other methodologists, we have an answer to our centuries-long inductive problems. Johnny Logic 18:56, 22 February 2007 (UTC)

Argh. It is like he was a broken record. He summed up his point (insofar as he had one), as follows:

Perhaps it has never occurred to you but Ockham’s is a statement of criteria of preference for explanations. Its advocates like to assert that it is used by science. However science has no need for it because it has much better and more effective criteria of preference. I can understand why philosophers might find it compelling because philosophy is an ancient and historic tradition that does not use the criteria of modern science and without some principle like Ockham’s they would be forced to admit that all they have is personal preference (As if that was not what they were actually doing anyway.). Science uses criteria which I have restated often enough, it has experiment on reality and it prefers explanations that win in fidelity and predictability, not simplicity. Simplicity is nice, but hardly a criteria of preference in scientific explanations. A striking example is Einstein’s aversion to QM. He seemed to think that reality should conform to some notion he had of what was “simple” but was not able to produce any explanation that worked any better. That is why QM is still the current preferred explanation. Few that have studied it would say it was “simple”, but it has the annoying property of predicting reality before the fact very well. And if anyone were to come up with an explanation that did a better job, no matter how “simple” or “complex” it would then become the preferred explanation. Ockham’s is as much use to science as screen doors on a submarine. Yes there are a few instances were it may come in handy but it is hardly necessary. As such it would be refreshing if advocates of Ockham’s would demonstrate the honesty and fortitude to apply it to Ockham’s. At least when it comes to claims that it is an important criterion of preference for science. Gkochanowsky 19:45, 22 February 2007 (UTC)

My last reply:

Your characterization of philosophy is ridiculous (again a straw man argument). Do you forget that the origin of some of our sciences is in philosophy? Adam Smith, father of modern economics, philosopher; Gottfried Leibniz, co-creator of the calculus, philosopher; William James, pioneering psychologist, philosopher; Bertrand Russell, great logician aiding in the production of a logical foundation for mathematics and much more (paving the road for computer science), philosopher, etc. I anticipate that you will say that their contributions are science and their failings are philosophy, by definition– a cheap tautological victory that shields you from ever having to learn anything about philosophy.

Science uses criteria which I have restated often enough, it has experiment on reality and it prefers explanations that win in fidelity and predictability, not simplicity. Simplicity is nice, but hardly a criteria of preference in scientific explanations.

I’m afraid that what you list is insufficient for uniquely identifying a theory; underdetermination is a specter that cannot be banished that easily. There are quite literally an infinite number of theories/models/inferences logically consistent with any finite number of data points in a possibility space equivalent to the reals. Curve fitting is an excellent example to illustrate this point. How do we determine which curve wins-out in “fidelity and predictability”? I can tell you why a particular one wins in terms of simplicity and fit, and, better, how it enables important epistemic goals (reliability, predictive accuracy, etc.). Again, please look at the literature.

From whence does your monolithic understanding of scientific methodology come? Any references? Does it change your mind that scientists have also written on the significance of simplicity as a theoretical virtue (e.g. Zellner, A., Keuzenkamp, H. & McAleer, M. (eds.) (2001) Simplicity, Inference and Modelling: Keeping It Sophisticatedly Simple, Cambridge: Cambridge University Press.): The editors of [Simplicity, Inference and Modelling] “sent out surveys to 25 recent Nobel laureates in economics. Almost all replied that simplicity played a role in their research, and that simplicity is a desirable feature of economic theories (Zellner et al. 2001, p.2).” (from the article on Simplicity at Stanford Encyclopedia of Philosophy).

My efforts are better spent editing this entry. Like AceMyth, I will swallow a frog and leave you to your ad hominems and categorical statements; maybe (hope beyond hope) you will actually look at the resources that I have provided and try to see what use OR might be in theory selection, and thus, science. Johnny Logic 23:23, 22 February 2007 (UTC)

Your thoughts?