Paradigm shifts: positivism, realism and the fight against apathy in the quantum revolution

Jim Baggott reviews Escape from Shadow Physics: the Quest to End the Dark Ages of Quantum Theory by Adam Forrest Kay

Science can be a messy business. Scientists caught in the storm of a scientific revolution will try to react with calm logic and reasoning. But in a revolution the stakes are high, the atmosphere charged. Cherished concepts are abandoned as troubling new notions are cautiously embraced. And, as the paradigm shifts, the practice of science is overlaid with passionate advocacy and open hostility in near-equal measure. So it was – and, to a large extent, still is – with the quantum revolution.

Niels Bohr insisted that quantum theory is the result of efforts to describe a fundamentally statistical quantum world using concepts stolen from classical physics, which must therefore be interpreted “symbolically”. The calculation of probabilities, with no reference to any underlying causal mechanism that might explain how they arise, is the best we can hope for.

In the heat of the quantum revolution, Bohr’s “Copenhagen interpretation” was accused of positivism, the philosophy that valid knowledge of the physical world is derived only from direct experience. Albert Einstein famously disagreed, taking the time to explore alternatives more in keeping with a realist metaphysics, with a “trust in the rational character of reality and in its being accessible, to some extent, to human reason”, that had served science for centuries. Lest there be any doubt, Adam Forrest Kay’s Escape from Shadow Physics: the Quest to End the Dark Ages of Quantum Theory demonstrates that the Bohr–Einstein debate remains unresolved, at least to anybody’s satisfaction, and continues to this day.

Escape from Shadow Physics is a singular addition to the popular literature on quantum interpretations. Kay holds PhDs in both literature and mathematics and is currently a mathematics postdoc at the Massachusetts Institute of Technology. He stands firmly in Einstein’s corner, and his plea for a return to a realist programme is liberally sprinkled with passionate advocacy and open hostility in near-equal measure. He writes with the zeal of a true quantum reactionary.

Like many others before him, in arguing his case Kay needs first to build a monstrous, positivist Goliath that can be slain with the slingshot of realist logic and reasoning. This means embracing some enduring historical myths. These run as follows. The Bohr–Einstein debate was a direct confrontation between the subjectivism of the positivist and the objectivism of the realist. Bohr won the debate by browbeating the stubborn, senile and increasingly isolated Einstein into submission. Acting like some fanatical priesthood, physicists of Bohr’s church – such as Wolfgang Pauli, Werner Heisenberg and Léon Rosenfeld – shouted down all dissent, establishing the Copenhagen interpretation as a dogmatic orthodoxy.

Historical scientific myths are not entirely wrong, and typically hold some grains of truth. Rivals to the Copenhagen view were indeed given short shrift by the “Copenhagen hegemony”. Pauli sought to dismantle Louis de Broglie’s “pilot wave” interpretation soon after it was presented in 1927. He went on to dismiss its rediscovery by David Bohm in 1952 as “shadow physics beer-idea wish dreams”, and “not even new nonsense”. Rosenfeld dismissed Hugh Everett III’s “many worlds” interpretation of 1957 as “hopelessly wrong ideas”.

But Kay is not content with the myth as it is familiarly told, and so seeks to deepen it. He confers on Bohr “the charisma of the hypnotist, the charisma of the cult leader”, adding that “the Copenhagen group was, in a very real sense, a personality cult, centred on the special and wise Bohr”. Prosecuting such a case requires a selective reading of science history, snatching quotations where they fit the narrative, ignoring others where they don’t. In fact, Bohr did not deny objective reality, or the reality of electrons and atoms. In interviews conducted shortly before his death in 1962, Bohr reaffirmed that his core principle of “complementarity” (of waves and particles, for example) was “the only possible objective description”. Heisenberg, in contrast, was much less cautious in his use of language and makes an easier target for anti-positivist ire.

It can be argued that the orthodoxy, such as it is, is not actually based on philosophical pre-commitments. The post-war Americanization of physics drove what were judged to be pointless philosophical questions about the meaning of quantum theory to the fringes. Aside from those few physicists and philosophers who continued to nag at the problem, the majority of physicists just got on with their calculations, completely unconcerned about what the theory was supposed to mean. They just didn’t care.

As Bohm explained: “Everybody plays lip service to Bohr, but nobody knows what he says. People then get brainwashed into saying Bohr is right, but when the time comes to do their physics, they are doing something different.” Many who might claim to follow Bohr’s “dogma” satisfy their physical intuitions by continuing to think like Einstein.

Anton Zeilinger, who shared the 2022 Nobel Prize for Physics for his work on quantum entanglement and quantum information science, confessed that even physicists working in this new field consider foundations to be a bit dodgy: “We don’t understand the reason why. Must be psychological reasons, something like that, something very deep.” Kay admits this much when he writes: “Yes, many people think the debate is over and Bohr won, but that is actually a social phenomenon.” In other words, the orthodoxy is not philosophical, it is sociological. It has very little to do with Bohr and the Copenhagen interpretation. In truth, Kay is fighting for attention against the apathy and indifference characteristic of an orthodox mainstream physics, or what Thomas Kuhn called “normal science”.

As to how a modern-day realist programme might be pursued, Kay treats us to some visually suggestive experiments in which oil droplets follow trajectories determined by wave disturbances on the surface of the oil bath on which they move. He argues that such “quantum hydrodynamic analogues” show us that the pilot-wave interpretation merits much more attention than it has so far received. But while these analogues are intuitively appealing, the so-called “quantization” involved is as familiarly classical as musical notes generated by string or wind instruments. And, although such analogues may conjure surprising trajectories and patterns, they cannot conjure Planck’s constant. Or quantum entanglement.

But the pilot-wave interpretation demands a hefty trade-off. It features precisely the non-local, “peculiar mechanism of action at a distance” of the kind that Einstein abhorred, and which discouraged his own exploration of pilot waves in 1927. In an attempt to rescue the possibility that reality may yet be local, Kay reaches for a loophole in John Bell’s famous theorem and inequality. Yet he overlooks the enormous volume and variety of experiments that have been performed since the early 1980s, including tests of an inequality devised by the Nobel-prize-winning theorist Anthony Leggett that explicitly close the loophole he seeks to exploit.

Escape from Shadow Physics is a curate’s egg. Those readers who would condemn Bohr and the Copenhagen interpretation, for whatever reasons of their own, will likely cheer it on. Those looking for balanced arguments more reasoned than diatribe will likely be disappointed. Despite an extensive bibliography, Kay commits some curious sins of omission. But, while the journey that Kay takes may be flawed, there is yet sympathy for his destination. The debate does remain unresolved. Faced with the mystery of entanglement and non-locality, Bohr’s philosophy offers no solace. Kay (quoting a popular textbook) asks that we consider future generations in possession of a more sophisticated theory, who wonder how we could have been so gullible.

• 2024 Weidenfeld & Nicolson 496pp £25 hb