A collaboration of physicists and computer scientists has shown that artificial intelligence (AI) can help control the delicate process of confining an ultra-hot plasma within a fusion reactor. Members of the team say that their demonstration, which exploits a technique known as reinforcement learning to operate the magnetic coils in a tokamak, could enable more speedy development of reactors with novel geometries.
Tokamaks are doughnut-shaped chambers designed to produce energy by fusing light nuclei in the form of a plasma. The nuclei are heated to hundreds of millions of degrees to overcome their mutual repulsion, while the plasma is held in place using the fields from a series of magnetic coils. Those fields keep the plasma away from the chamber’s walls, where it would otherwise lose heat and damage the tokamak.
One aim of fusion scientists is to understand how the spatial distribution of a plasma within the tokamak chamber influences its stability and spatial confinement. This task is complicated by the need to design a new feedback scheme for each configuration, so that the magnets can be tuned appropriately in response to the plasma’s highly nonlinear behaviour. The design process usually involves calculating an initial set of coil currents and voltages, then using a combination of plasma-reconstruction algorithms and feedback controllers to adjust the voltages – and with them the magnetic field. The result is effective control over a plasma’s vertical and radial positions as well as its current, but only with significant effort.
The right rewards
In the latest work, scientists from Google’s London-based DeepMind subsidiary and the École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland swapped these multiple feedback controllers for a single controller based on reinforcement learning (RL). An RL algorithm is a form of machine learning that is designed to arrive at an optimum set of output values by adjusting the weights on its nodes in a step-like fashion.
In this case, the algorithmic “agent” receives measured and target values of the plasma parameters – a long list, including numerous spatial dimensions, electrical currents and magnetic fluxes within the tokamak chamber – as its inputs. After processing these values using several layers of nodes, it issues outputs corresponding to the voltage levels on each of the magnets. Once the magnets are adjusted and the new feedback from sensors is received the cycle starts again – being repeated some 10,000 times a second.
This process relies on a computational object known as a reward function, which compares the measured and target values of the plasma parameters at each step and combines the disparity among all parameters into a single value. It is this value that determines how great a “reward” the agent receives for getting closer to the desired values. Conversely, if the plasma ends up crashing into the chamber wall the agent receives a penalty and the process comes to a halt.
Working out this reward function is the first stage of setting up a new RL-based controller, since each function corresponds to a specific plasma configuration. Next, the agent is trained by exposing it to input data from a simulated tokamak, which must realistically describe the plasma’s changing shape and current while limiting its computational demand to keep the learning process manageably short. Finally, the agent’s optimized control scheme is tested on a real tokamak.
Testing the AI
To carry out the experimental stage of the work, the collaboration turned to the EPFL’s Variable-Configuration Tokamak. Initially, members of the team used traditional feedback controllers to create and maintain the plasma in an initial state. At a certain pre-agreed time, they switched to their own control scheme, adjusting 19 separate magnetic coils to tune the plasma so that it ended up with the correct shape and current while keeping the neural network’s weights fixed.
The researchers then showed they could keep control over the plasma’s current and shape – with deviations of no more than a few percent from the intended values – while making the kind of changes needed for a full plasma discharge. They were also able to manipulate the plasma into a variety of shapes, including one similar to that proposed for the ITER reactor being built in the south of France and another known as a snowflake configuration that helps to spread a tokamak’s heat and particle exhaust over a larger surface. In addition, they showed it was possible to set up two separate plasmas within the same tokamak, which they say is a first step to study more advanced plasma configurations.
Improving tokamak performance
The researchers say that their new AI-based scheme could improve tokamak performance, with its open-ended nature perhaps allowing power output to be maximized. More broadly, they say, the technology might lead to new reactor designs by allowing the joint optimization of several device parameters – including plasma shape, wall design and heat load.
First results from UK tokamak offers a STEP towards commercial fusion
Other scientists are enthusiastic about the new work, but caution that its widespread adoption will depend on improving the simulator used to train the AI agent. According to Karel van de Plassche of the Dutch Institute for Fundamental Energy Research and the University of Technology of Eindhoven in the Netherlands, “essential components” of more accurate simulation will include detailed physics on turbulence and magnetohydrodynamics. Ghim Young-chul of the Korea Advanced Institute of Science and Technology (KAIST) notes that it remains to be seen how well the new approach can be applied to tokamaks based on superconductors rather than copper coils, given that the former introduce time delays when regulating current. Another uncertainty, he adds, is the control of stability. While the RL algorithm enables steady-state operation, he notes that “tokamaks have other important unstable events that must be controlled”.
The research is published in Nature.
