The majority of Americans are now familiar with ChatGPT, a computer program powered by artificial intelligence, or AI, that generates text and dialogue in response to text prompts provided by a user. Its service has an estimated 180 million registered users with about 1.6 billion website visitors per month.
A groundbreaking collaboration with the tool’s maker, OpenAI, is underway at Arizona State University. As part of the AI Innovation Challenge, students, researchers and faculty members across the university are developing more than 100 projects in 14 schools using ChatGPT Enterprise to explore diverse topics including ethics, scientific advancement and data analysis.
But while people all over the world are adjusting to this new technology and putting it to work enriching their daily lives, researchers in the Ira A. Fulton Schools of Engineering at ASU are looking to the future, driving innovation in the AI arena.
Dimitri Bertsekas, professor of computer science in the School of Computing and Augmented Intelligence, part of the Fulton Schools, has been awarded a provisional patent for the development of new algorithms that improve the predictive features of tools like ChatGPT. The goal is to generate the most accurate, helpful results for the user. The new work builds on the famed mathematician’s prior research in reinforcement learning.
Predicting what comes next
The acronym GPT refers to generative pretrained transformer. GPTs are a type of artificial intelligence inspired by the human mind. A software program is given access to large amounts of text that describe a broad range of subjects. This process, referred to as pretraining, provides the AI system with context to help it understand and process the input it receives. The program then combines its pretraining with specific information it gathers from the user, transforming the data and generating something new.
“Currently, software like ChatGPT use a ‘next word prediction’ principle,’” Bertsekas says. “Based on the text that the GPT has seen, it generates a set of likely next words. It expands the text it has generated one word at a time, looking exclusively to the past.”
Bertsekas notes that many people already have experience with this technology using the predictive text features of their smart phone’s text messaging program, even if they have not specifically interacted with a chatbot.
Working with Yuchao Li, a postdoctoral research scholar in the School of Computing and Augmented Intelligence, Bertsekas has developed algorithms that harness the power of complex mathematics to enhance the current predictive process by looking not only to the past but also to the future.
“Our algorithms predict next words by looking at the preceding text but also by anticipating future word choices to improve metrics of quality, such as coherence and grammar,” Bertsekas says.
Read the full story on Full Circle.