"The Jahn-Teller effect has long been known to play an important role in the relationship between the structure of molecules and their energy levels, but this is the first time anyone has directly imaged it at the single-molecule level," says Mike Crommie of the University of California at Berkeley and the Lawrence Berkeley Laboratory, leader of the team that saw the effect.

The Jahn-Teller effect occurs in systems that can exist in two or more distinct states that have the same energy. Because such a "degenerate" system is unstable, a molecule will distort itself to split the energy levels. Pure carbon-60 is an insulator because its highest occupied molecular orbital (HOMO) is full of electrons, while the lowest unoccupied molecular orbital (LUMO) is empty. The HOMO state is 5-fold degenerate, while the higher-energy LUMO state is 3-fold degenerate.

If carbon-60 is doped with potassium (K) to produce K3C60, it becomes metallic because the potassium atoms donate three electrons to the carbon-60 molecules, and these electrons go into the LUMO band. However, if an extra potassium atom is added to produce K4C60, the molecule becomes insulating, even though there are electrons in the LUMO band.

To investigate this, Crommie and colleagues studied a gold surface on which monolayers of both materials can exist simultaneously. In particular, they used a low-temperature scanning tunnelling microscope (STM) to both image the monolayer and also map the local density of states for K3C60 and K4C60.

The Berkeley physicists observed different symmetries for the occupied and unoccupied states, which is a telltale sign of a Jahn-Teller distortion. The STM data shows that the K3C60 monolayer exhibits a triangular lattice backbone structure. In contrast, the K4C60 monolayer displays an almost nearly rectangular structure with four molecules per unit cell. If the number of potassium atoms falls between 3 and 4, the system is a mixture of the two phases (see figure).

Moreover, the Berkeley team saw an energy gap of 200 mV in the three-fold degenerate LUMO band of K4C60. This band splits up into a group of two-fold degenerate levels at lower energy and one non-degenerate level at higher energy, which explains why the molecule becomes insulating.

"The beauty of seeing this physics in a monolayer is that it is so accessible: we can image the individual molecules with our STM and look to see where the electrons go at different energies," says Crommie. "This is a freedom that does not exist for a bulk insulator, which you cannot image with a STM."