Quantum mechanics is both the most powerful theory physicists have ever developed and the most confusing. On the one hand, countless experiments have confirmed the predictions; the theory underlies modern technology and enables the electronic devices we use every day. On the other hand, quantum mechanics describes an underlying reality that is completely at odds with the world we perceive. In the quantum realm, a single particle exists in many places at the same time – at least while no one is looking at it. The theory also allows for inexplicable connections: a pair of atoms, no matter how widely separated, can be “entangled”, so that what happens to one atom immediately affects the other. Albert Einstein called the phenomenon “spooky action at a distance”.
These paradoxes have defined—or plagued—the theory since its inception more than a century ago. To this day, physicists still do not agree on what quantum mechanics tells us about the nature of reality. Are there multiple universes? Do things only occur when they are observed? Is consciousness somehow central to the laws of physics? And what if all these mysteries could have been solved right at the birth of quantum mechanics?
. That is the case physicist Antony Valentini, a physicist at Imperial College London, does in his new book Beyond the Quantum: A Quest for the Origin and Hidden Meaning of Quantum Mechanics (Oxford University Press, 2026).
On supporting science journalism
If you like this article, please consider supporting our award-winning journalism by subscribes. By purchasing a subscription, you help secure the future of impactful stories about the discoveries and ideas that shape our world today.
Valentini claims that Louis de Broglie, a French physicist and Nobel laureate, developed a framework for quantum mechanics that eliminated the paradoxes about 100 years ago. In pilot wave theory, as de Broglie’s brainchild is known, particles are guided by accompanying waves. The particles themselves are always in one position and one position only; it is the spatially extended pilot wave that creates the impression that a particle is here at once and there. There is no need for an observer to conjure up that particle. Although de Broglie’s 1924 conjecture about the wave-like nature of matter was quickly confirmed by experiment and became an integral part of quantum theory, the physics community discounted or misrepresented the larger ideas from which he derived his key insights.
Valentini has spent his entire career championing and expanding de Broglie’s views. He recently spoke with Scientific American about his lonely path and why de Broglie may have been onto something.
(An edited transcript of the interview follows.)
In the history of science, has there ever been another situation like this, where there have been such wildly divergent views on what a theory means?
I’m not sure it has. If you go back to (Isaac) Newton’s time, he believed that space was empty and that there was a direct gravitational effect at a distance. And on the continent it was the Cartesians (followers of the mathematician and philosopher René Descartes), who thought, “Oh no, space is full of this material medium, and that explains the gravitational attraction.” But (the debate) didn’t last that long. Certainly in the quantum case, the great variety of interpretations that say such completely different things about the world – I think it’s a pretty safe bet that there is no analogue in the history of science.
One of the most striking things about modern physics is the stark distinction between the macroscopic and quantum worlds, each of which appears to be governed by completely different physical laws. You compare this to the way medieval astronomers divided the cosmos into terrestrial and celestial regions.
I think it’s a useful and valid parallel, this idea that there was a heavenly kingdom that we couldn’t understand; everything above the moon and beyond was eternal and unchanging, completely different from the sublunar world, which was made of ordinary, imperfect matter that was always changing. It is a distinction that goes back to Aristotle. The parallel with quantum mechanics is extraordinary, that the quantum system is something our mind cannot understand. We can only understand the macroscopic.
The Austrian physicist Erwin Schrödinger developed the wave equation of quantum theorywhich describes quantum systems as waves that evolve with time. What role did this equation play in the so-called measurement problem: If a particle exists in different places at the same time, why do measurements find a given particle in only one place?
Schrödinger created the measurement problem by removing the particles from de Broglie’s theory. Mathematically, a (quantum wave) is a superposition of many different positions: a particle can be here and here and here; it could be anywhere. You can have a superposition of a live cat and a dead cat, or a superposition of different energies. They are all just different variations on the same theme. The wave equation contains all possible positions. How then can you explain that we see this tiny point-like object if the only reality is an extended wave?
And this conundrum was recognized early in the development of quantum theory.
Here is Wolfgang Pauli writing to Niels Bohr in 1927: “In the last edition of Journal of Physics, a paper by de Broglie has appeared…. It is very rich in ideas and very sharp, and on a much higher level than the childish papers of Schrödinger, who even today still thinks he can … abolish material points.” And because Schrödinger removed (particles from his equation), we’ve ended up with decades of confusion.
Why do you think de Broglie’s theory was set aside and neglected?
I’m not sure there is a single answer. It is perhaps a mixture of reasons.
By 1923 de Broglie had developed a new theory of motion. It was a complete break, very different from Newtonian or even Einsteinian physics. And yet these people completely passed by. The only thing that entered the collective consciousness of physicists was that de Broglie had shown that a particle can behave like a wave.
Word of de Broglie’s treatise spread, although hardly anyone actually read it. Einstein did it. It was Einstein who really alerted people that de Broglie had done something very important. He encouraged Schrödinger to read it – and he did. Most others, it seems, never read de Broglie’s treatise.
And then there is the sociological point that de Broglie was quite isolated in Paris. De Broglie was a bit of a loner; he mainly worked alone. At that time, in the 1920s, France was really a backwater in theoretical physics. It was strong in experimental physics, strong in mathematics, but not in theoretical physics.
Has your search for pilot wave theory been lonely? Rewarding? Frustrating?
The short answer is all of this and more. Has it been lonely? It has been this particular situation. I have really tried to convey the key points to physicists. And it seems to fall on deaf ears. It’s like people are stuck on repeat – the same wrong arguments, the same historical misconceptions just go round and round and round.
When I first came across pilot wave theory, it seemed so obvious to me. My goodness, pilot wave theory is in principle a broader physics; quantum theory is a special case of something larger. Pilot wave theory has exciting new physics, and maybe we can find evidence for it.
In your book, you describe how the predictions of the pilot wave theory about the physics of matter differ in some cases from the predictions of accepted quantum mechanics. In particular, you mention how the cosmic microwave background (CMB) – the radiation created during the big bang that now permeates the universe – can support some of the predictions of pilot wave theory.
CMB is an excellent and promising path and I have worked extensively on it with various collaborators. There are reported anomalies in the CMB that qualitatively match the type of anomalies that pilot wave theory would predict. There are some tantalizing hints, but the data is just too noisy to draw any firm conclusions. This probably won’t be settled for another 10 years or so.
Is pilot wave theory true? Is it an accurate theory of the world? If I knew it was true, I wouldn’t investigate it. There is always, in the back of your mind, the thought that this could be completely wrong! Or it may be partially correct. In the late 19th century Ludwig Boltzmann modeled (gas molecules) as little billiard balls – little hard balls that bounce around. It turns out that molecules are much more complicated than that. But still, his model contained much truth. It could be that pilot wave theory is a bit like that, an approximate model.
