A team of researchers from MIT and beyond recently proposed Atomistic Language Models (ALMs) as a new paradigm for generating and understanding properties of 3D (periodic) materials with a single model. The benefit of this approach is that it leverages the large amount of research effort that has been invested in each area of material science independently (e.g., energy and force prediction, crystal structure prediction) to create a unified method mutually beneficial to each area through the magic of backpropagation and parameter-efficient fine-tuning.
ALMs perform well across a range of tasks in material science, including unconditional (or text-guided) crystal generation, structure prediction, and property regression. Luckily for the research community, ALMs are publicly available on HuggingFace, with a new benchmark for text-guided material generation and optimization (ALM-Bench) also available there!
Complementary to ALMs is BioMatrix, a comprehensive approach to modeling 3D molecules and proteins. Like ALMs, it relies on bespoke tokenizers for each data domain (i.e., MolStructTok for molecules and GCP-VQVAE for proteins) to equip its base language model (Qwen3) with knowledge of their continuous (geometric) features.
BioMatrix has been benchmarked on 80 prediction or generation tasks involving molecule/protein sequences, structures, or textual descriptions, reaching state-of-the-art for 77/80 tasks. This suggests that multimodality and scaling, as well as pretraining on natural language data, can advance many scientific tasks with minimal task-specific assumptions.
Many recent works have pitched a vision of AI for Science models becoming less specialized over time. Do you agree with this vision, or do you think reality will be (a bit) messier and less linear in its simplification trajectory?