Every chemical reaction is a journey over a mountain range: the molecules start in one valley (the reactants) and must cross to another (the products), and the only way over is the lowest pass through the ridge. The top of that pass is the transition state. It lasts less than a trillionth of a second, yet it decides almost everything. How high the pass sits sets how fast a reaction goes, and its shape determines which product forms. Find it, and you can predict and design chemistry, leading to better catalysts, cleaner processes, and new medicines. The catch is that transition states are notoriously hard to locate, often costing days of supercomputer time for a single reaction.
Recently, generative machine-learning models learned to sketch these structures in seconds, reaching near chemical accuracy, but only on the small organic molecules they were trained on. The reactions we most want to speed up involve transition metals and heavier elements the models had never seen and for which large datasets for model training aren't readily available. So, we asked what happens there, and the answer was sobering. On benchmarks we built for exactly this test, the models produced unphysical geometries and large energy errors, even for reactions that looked, to a chemist's eye, much like ones they should be able to handle perfectly. They had memorized the chemistry they were shown rather than learning chemistry itself.
The fix came from changing how we thought about the problem. Transition states are rare and expensive, so there will never be many examples of exotic chemistry to train on. But stable molecular structures, the calm resting shapes molecules sit in, called equilibrium conformers, are cheap and abundant, even for the tricky elements. So before asking the model to do the hard thing (find a pass in unfamiliar terrain), we let it do an easy thing first: wander that terrain and learn what it looks like. This self-supervised pretraining teaches the model what physically sensible structures look like for new elements, such as how far apart their atoms sit and how they bond, from data that costs almost nothing. Only then do we fine-tune it on the few precious transition-state examples we have.