When you spin an image of a molecular configuration, a person can discern that the rotated image represents the same molecule, while a machine-learning algorithm might interpret it as a distinct data point. In computer science terminology, the molecule is termed “symmetric,” indicating that the core configuration of that molecule remains unchanged when subjected to specific transformations, such as rotation.
If a drug discovery algorithm fails to grasp symmetry, it could lead to erroneous predictions regarding molecular attributes. Yet, in spite of some practical achievements, it remains uncertain whether an efficient computational approach exists to train a robust model that inherently respects symmetry.
A novel investigation by MIT scholars addresses this query and unveils the first methodology for machine learning that incorporates symmetry, which is demonstrably efficient in terms of both computation and the volume of data required.
These findings clarify a fundamental inquiry and may assist researchers in advancing more sophisticated machine-learning models capable of managing symmetry. Such models could prove beneficial across a spectrum of applications, from the exploration of new materials to the identification of celestial irregularities to deciphering intricate climate trends.
“These symmetries are crucial as they convey information that nature reveals about the data, and we ought to consider it in our machine-learning models. We have now demonstrated that it is feasible to conduct machine-learning with symmetric data efficiently,” remarks Behrooz Tahmasebi, an MIT graduate student and co-lead author of this research.
He is joined in the publication by co-lead author and fellow MIT graduate student Ashkan Soleymani; Stefanie Jegelka, an associate professor of electrical engineering and computer science (EECS), and a member of the Institute for Data, Systems, and Society (IDSS) and the Computer Science and Artificial Intelligence Laboratory (CSAIL); along with senior author Patrick Jaillet, the Dugald C. Jackson Professor of Electrical Engineering and Computer Science, who is a principal investigator at the Laboratory for Information and Decision Systems (LIDS). The findings were recently presented at the International Conference on Machine Learning.
Examining symmetry
Symmetric data exists across numerous fields, particularly in natural sciences and physics. A model proficient in recognizing symmetries can identify an object, such as a vehicle, regardless of its position in an image.
Unless a machine-learning model is formulated to accommodate symmetry, it may exhibit lower accuracy and be more susceptible to errors when confronted with novel symmetric data in real-world contexts. Conversely, models that leverage symmetry can be quicker and necessitate less training data.
However, training a model to analyze symmetric data is a complex endeavor.
A widely used methodology is known as data augmentation, where researchers manipulate each symmetric data instance into multiple variations to enhance the model’s generalization to new data. For example, one could rotate a molecular configuration numerous times to generate new training data, but if researchers aim to ensure the model respects symmetry, this could become computationally overwhelming.
An alternative strategy is to embed symmetry within the model’s design. A prominent example of this is a graph neural network (GNN), which naturally handles symmetric data because of its architecture.
“Graph neural networks are swift and effective, managing symmetry efficiently, but the underlying learning process of these models remains elusive. Gaining insight into GNNs is a primary impetus for our work, prompting us to begin with a theoretical analysis of what occurs when data exhibit symmetry,” Tahmasebi states.
They investigated the statistical-computational tradeoffs in machine learning involving symmetric data. This tradeoff indicates that methods requiring fewer data might be more computationally demanding, necessitating a balance.
Building on this theoretical exploration, the researchers devised an efficient algorithm for machine learning utilizing symmetric data.
Mathematical combinations
To achieve this, they borrowed concepts from algebra to compress and streamline the issue. Subsequently, they reformulated the challenge using geometric principles that effectively encapsulate symmetry.
Ultimately, they merged algebra and geometry into an optimization problem that can be resolved efficiently, culminating in their novel algorithm.
“Most theories and applications have concentrated on either algebra or geometry. Here, we amalgamated the two,” notes Tahmasebi.
The algorithm requires fewer data samples for training compared to traditional methods, thereby enhancing a model’s precision and adaptability to new applications.
By demonstrating that efficient algorithms for machine learning with symmetry can be developed, and illustrating the process, these findings may pave the way for new neural network architectures that are more accurate and less resource-intensive than existing models.
Researchers could further utilize this analysis as a foundation to scrutinize the operational mechanisms of GNNs and how their processes differ from the algorithm devised by the MIT team.
“With a deeper understanding, we can create neural network architectures that are more interpretable, robust, and efficient,” adds Soleymani.
This research is partially supported by the National Research Foundation of Singapore, DSO National Laboratories of Singapore, the U.S. Office of Naval Research, the U.S. National Science Foundation, and an Alexander von Humboldt Professorship.