A key question on many people’s minds is how to leverage recent advances in both large language models (LLMs) and domain-specific generative models. Fortunately, LLM-to-Flow Distillation (LFD) proposes a succinct solution: (1) ask an LLM how to solve your specific problem using a generative model for your scientific domain, (2) use the generative model to design some new data, (3) assess that new data with the LLM and create subgoals for the generative model to reach, (4) use direct preference optimization (DPO) to steer the generative model’s latent representations into a promising direction (according to the LLM’s feedback), and (5) rinse and repeat until goal convergence.
This is, of course, a simplification of the process, but at a high level, LFD modifies domain-specific generative models to produce data increasingly in alignment with the user’s specified goal through iterative subgoals and DPO. Since this blueprint is domain-agnostic, one can imagine how it will be applied across scientific disciplines in the coming years. If this work piques your interest, its source code can be found on GitHub.
The Baker Lab at the University of Washington recently released their latest version of the popular protein design model RFdiffusion, titled RFD4-Proteina (as a collaborative nod to the Proteina family of protein models from NVIDIA). In contrast to previous editions of RFdiffusion, RFD4-Proteina features all-atom latent space generative modeling for protein design tasks. Similar to Zatom-2, its architecture features physics-based supervision to improve the fidelity of its generated samples.
Most notably, RFD4-Proteina supports a large number of protein binder design tasks that have been validated by wet-lab experiments, e.g., targeting nucleic acids, ligands, complexes, and more. Such promising experimental results suggest that RFD4-Proteina’s new intrinsic and extrinsic reasoning capabilities, alongside its diversely pretrained generative latent space, have endowed it with rich biomolecular design prowess compared to existing protein design models. As usual for the Baker Lab, its source code can generously be found on GitHub.
How do you think the interplay between ambient space generative modeling and latent space generative models will play out? Are there fundamental capabilities that latent space generators can achieve which ambient space models cannot? And how will domain-specific generators like RFD4-Proteina best be paired with large language models for scientific discovery? Using the LFD template, or via some other means?