“`html
Artificial intelligence systems such as ChatGPT deliver seemingly credible responses to any inquiries you may have. However, they do not always disclose the shortcomings in their understanding or the areas where they lack certainty. This issue can have significant ramifications as AI technologies are increasingly implemented in fields like drug development, information synthesis, and autonomous driving.
Currently, the MIT spinout Themis AI is assisting in quantifying model uncertainty and rectifying outputs before they escalate into larger issues. The company’s Capsa platform is compatible with any machine-learning model, enabling it to identify and amend unreliable outputs within moments. It operates by adapting AI models to recognize patterns in data processing that signify uncertainty, incompleteness, or bias.
“The concept is to take a model, integrate it with Capsa, pinpoint the uncertainties and failure modes, and subsequently enhance the model,” states Themis AI co-founder and MIT Professor Daniela Rus, who also leads the MIT Computer Science and Artificial Intelligence Laboratory (CSAIL). “We are enthusiastic about providing a solution that can enhance models and guarantee the model’s correctness.”
Rus established Themis AI in 2021 along with Alexander Amini ’17, SM ’18, PhD ’22 and Elaheh Ahmadi ’20, MEng ’21, two former research associates from her laboratory. Since then, they have assisted telecommunications firms with network planning and automation, aided oil and gas companies in leveraging AI for seismic image analysis, and published research on creating more reliable and trustworthy chatbots.
“We aim to empower AI in the most critical applications across every sector,” Amini remarks. “We’ve all encountered instances of AI fabricating information or committing errors. As AI is rolled out more extensively, those errors could have catastrophic consequences. Our software can enhance the transparency of these systems.”
Assisting models in recognizing their knowledge gaps
Rus’ lab has been investigating model uncertainty for several years. In 2018, she secured funding from Toyota to examine the reliability of a machine learning-based autonomous driving solution.
“That is a safety-critical area where understanding model reliability is crucial,” Rus explains.
In another study, Rus, Amini, and their team developed an algorithm capable of identifying racial and gender biases within facial recognition systems and automatically reweighting the model’s training data, demonstrating it successfully eliminated bias. The algorithm worked by pinpointing the unrepresentative segments of the foundational training data and generating new, analogous data samples to achieve balance.
In 2021, the co-founders demonstrated that a similar method could assist pharmaceutical firms in using AI models to forecast the characteristics of drug candidates. They established Themis AI later that same year.
“Guiding drug discovery could potentially yield substantial cost savings,” Rus notes. “That was the application that made us realize how powerful this tool could be.”
Today, Themis is collaborating with businesses across a diverse array of industries, many of which are developing large language models. By utilizing Capsa, these models can assess their own uncertainty for each output.
“Numerous companies are eager to utilize LLMs that are tailored to their data, but they have concerns regarding reliability,” points out Stewart Jamieson SM ’20, PhD ’24, Themis AI’s technology lead. “We assist LLMs in self-reporting their confidence and uncertainty, facilitating more dependable question answering and flagging unreliable outputs.”
Themis AI is also in talks with semiconductor firms creating AI solutions on their chips that function independently of cloud environments.
“Typically, these smaller models operating on mobile devices or embedded systems fall short in accuracy compared to what can be processed on a server, but we can achieve the best of both worlds: low latency and efficient edge computing without compromising quality,” Jamieson elaborates. “We envision a future where edge devices handle most tasks, yet whenever they are uncertain about their outputs, they can relay those tasks to a central server.”
Pharmaceutical firms can also deploy Capsa to enhance AI models aimed at identifying drug candidates and predicting their performance in clinical trials.
“The predictions and outputs of these models are incredibly complex and difficult to interpret — experts invest significant time and effort trying to decipher them,” Amini states. “Capsa can offer insights right from the start to clarify whether the predictions are substantiated by evidence within the training data or merely conjecture lacking robust grounding. That can speed up the identification of the most convincing predictions, which we believe holds immense potential for societal benefit.”
Research for meaningful impact
Themis AI’s team is confident that the company is well-positioned to advance the forefront of ever-evolving AI technology. For instance, the organization is investigating Capsa’s capacity to enhance accuracy in an AI technique known as chain-of-thought reasoning, where LLMs articulate the steps they take to arrive at a conclusion.
“We’ve observed indications that Capsa could assist in steering those reasoning processes to pinpoint the highest-confidence chains of reasoning,” Amini asserts. “We believe this has monumental implications for improving the LLM experience, minimizing latencies, and lessening computational demands. It represents an extremely high-impact opportunity for us.”
For Rus, who has co-founded multiple firms since joining MIT, Themis AI represents an opportunity to ensure that her research at MIT has a meaningful impact.
“My students and I have grown increasingly passionate about taking the extra step to make our work relevant to the world,” Rus comments. “AI has tremendous potential to transform industries, but it also raises concerns. What inspires me is the possibility of facilitating the development of technical solutions that address these issues while also fostering trust and understanding between people and the technologies that are becoming integral to their everyday lives.”
“`