NVIDIA: EvolutionaryScale Unveils ESM3 Generative AI Model for Protein Design
Joerg Hiller Jun 25, 2024 12:00
EvolutionaryScale launches ESM3, an advanced AI model for protein design, leveraging NVIDIA's H100 GPUs to accelerate biological research.
Generative AI has revolutionized software development with prompt-based code generation, and now protein design is the next frontier. EvolutionaryScale has announced the release of its ESM3 model, the third-generation ESM model, which simultaneously reasons over the sequence, structure, and functions of proteins, providing protein discovery engineers with a programmable platform, according to the NVIDIA Blog.
The startup, which emerged from the Meta FAIR (Fundamental AI Research) unit, recently secured funding led by Lux Capital, Nat Friedman, and Daniel Gross, with investment from NVIDIA. EvolutionaryScale is at the forefront of programmable biology, assisting researchers in engineering proteins that can target cancer cells, find alternatives to harmful plastics, drive environmental mitigations, and more.
EvolutionaryScale's ESM3 model used NVIDIA H100 Tensor Core GPUs, resulting in the most compute ever put into a biological foundation model. The 98 billion parameter ESM3 model uses roughly 25x more FLOPs and 60x more data than its predecessor, ESM2. The company has developed a database of over 2 billion protein sequences to train its AI model, offering technology applicable to drug development, disease eradication, and understanding human evolution at scale.
Accelerating In Silico Biological Research With ESM3
With significant advancements in training data, EvolutionaryScale aims to accelerate protein discovery with ESM3. The model was trained on nearly 2.8 billion protein sequences sampled from various organisms and biomes, allowing scientists to prompt the model to identify and validate new proteins with increasing accuracy.
ESM3 offers substantial updates over previous versions. The model is inherently generative and follows an “all to all” approach, meaning structure and function annotations can be provided as input rather than just as output. Once publicly available, scientists can fine-tune this base model to create purpose-built models based on their proprietary data. ESM3’s large-scale generative training across vast amounts of data offers a revolutionary tool for in silico biological research.
Driving the Next Big Breakthroughs With NVIDIA BioNeMo
ESM3 provides biologists and protein designers with a generative AI boost, improving their engineering and understanding of proteins. With simple prompts, it can generate new proteins with a provided scaffold, self-improve its protein design based on feedback, and design proteins based on the user-indicated functionality. These capabilities can be used in any combination to provide chain-of-thought protein design, akin to messaging a researcher fluent in the intricate three-dimensional meaning of every known protein sequence.
“In our internal testing, we’ve been impressed by ESM3’s ability to creatively respond to complex prompts,” said Tom Sercu, co-founder and VP of engineering at EvolutionaryScale. “It solved an extremely challenging protein design problem to create a novel Green Fluorescent Protein. We expect ESM3 to help scientists accelerate their work and open up new possibilities — we’re excited to see its contribution to future research in life sciences.”
EvolutionaryScale will be opening an API for closed beta today, with code and weights available for a small open version of ESM3 for non-commercial use. This version will soon be accessible on NVIDIA BioNeMo, a generative AI platform for drug discovery. The complete ESM3 family of models will be available to select customers as an NVIDIA NIM microservice, run-time optimized in collaboration with NVIDIA, and supported by an NVIDIA AI Enterprise software license for testing at ai.nvidia.com.
The computing power required to train these models is growing exponentially. ESM3 was trained using the Andromeda cluster, which employs NVIDIA H100 GPUs and NVIDIA Quantum-2 InfiniBand networking. The ESM3 model will be available on select partner platforms and NVIDIA BioNeMo.
Image source: Shutterstock