ChatGPT doesn’t always get it right, but the future of AI is ‘so scary and so exciting,’ says AU prof
What does ChatGPT know about Dr. Jon Dron, and what does that tell us about the potential impacts of this technology? We asked the Faculty of Science and Technology researcher to share his insights about ChatGPT, AI, and what comes next.
What does ChatGPT know about Dr. Jon Dron?
The Athabasca University (AU) computer science researcher in the Faculty of Science and Technology, who specializes in learning technologies, asked the artificial-intelligence-based platform this question to see what it came up with. The results were fairly accurate—and then they weren’t.
“The first thing I asked was, ‘Tell me about Jon Dron,’ and it was pretty smart,” Dron said.
The response contained a small error about where he had worked in the past but was otherwise fairly accurate. It was when he started pressing for more specific information that the results became a lot less reliable.
When asked an open-ended question, “Tell me more,” ChatGPT embellished and overstated his accomplishments in some cases and seemed to outright fabricate things in others.
“It's completely hallucinating at this point. I mean it's got this vague idea that I'm doing something in computing and then it's just inventing stuff.”
– Athabasca University professor Dr. Jon Dron, speaking about ChatGPT
Dron has never released albums of experimental and ambient music as ChatGPT suggested, although he is a jazz singer. He does not have an undergraduate degree in mathematics, but rather in philosophy. ChatGPT also stated he has a master of science from the University of Calgary and a PhD from the Open University, when neither of those are true.
“It’s completely hallucinating at this point,” Dron said. “I mean it’s got this vague idea that I’m doing something in computing and then it’s just inventing stuff.”
While Dron was driven mostly by curiosity to go through this exercise, he said he also thought it was a good way to introduce the topic of ChatGPT’s capabilities and limitations.
The results showed that while there is tremendous potential for this kind of technology, there are also many unanswered questions about its reliability, equity in access to this kind of technology, and the potential impacts it could have on our lives.
How does ChatGPT work?
While the process is complicated, Dron said it comes down to this piece of software being trained on a big chunk of the product of human intelligence—basically what’s publicly available on the internet within a specific time frame.
Those inputs are then run through a type of computational model inspired by the human brain, known as a neural network, which is capable of learning and performing tasks by processing data through interconnected layers of artificial neurons.
“Essentially what it’s doing is tokenizing all the stuff that we write, so all those words turn into numbers,” he said.
The numbers go through the neural network and in the case of ChatGPT, eventually get translated back into plain English, although other AI systems may instead output images, sound, and video.
“This is what makes generative AI so smart,” Dron said. “You can feed anything digital into it and get anything digital out of it. The tokens work the same no matter what kind of data they’re dealing with.”
Why did you ask ChatGPT about yourself?
Dron said he was inspired to ask ChatGPT about himself due largely to “narcissism and a hunch” after he was invited to speak at a forum about ChatGPT in education. He likened the exercise to putting one’s own name in a search engine.
“People having been Googling themselves for years,” he said. “Given that I was going to be talking about this, I thought what a great way to introduce myself: have ChatGPT do it,” he said.
The results, of course, were a mixed bag. But in the process of asking about himself, Dron found that he could get it closer to the truth by asking it more specific questions. This idea that the right question will yield the right answer is the basis of a new and growing area of interest known as prompt engineering.
“This is again a massive oversimplification and trivializing what it’s actually doing, but essentially what it’s doing is averaging stuff. It’s got the contents of all the libraries in the world many times over, and you’re just getting kind of an average of what people would be most likely to write. ”
– Dr. Jon Dron
Why does ChatGPT lie?
Dron said it’s not necessarily that ChatGPT lies; it’s more accurate to say it just doesn’t know what’s true.
He explained that when the AI gets these written prompts from users as inputs, there may not be anything in its source material in the same form that was fed into it by a user. It creates an answer despite this limitation by breaking down the question, or “tokenizing” it, then forming the answer in a way like how predictive text works on a cellphone.
“What that means is that it’s always just trying to guess what word comes next, statistically. That’s not to trivialize it, and it shouldn’t be trivialized. It’s a much more complex process and a much more interesting process than that,” he said.
“It only ever knows about the next word, and the next word is the one what would be most likely to be said if you were asked this kind of question.”
What does this exercise tell us about how reliable ChatGPT is?
It is apparent from its fabrications about Dron’s life and accomplishments that ChatGPT isn’t necessarily reliable—and in fact, what it produces can be “kind of boring.”
“This is again a massive oversimplification and trivializing what it’s actually doing, but essentially what it’s doing is averaging stuff,” Dron said. “It’s got the contents of all the libraries in the world many times over, and you’re just getting kind of an average of what people would be most likely to write.”
But despite being fairly unreliable, and in Dron’s opinion boring, that doesn’t mean it’s not a useful tool. It just shouldn’t be relied upon to provide information that’s accurate and true.
What impact could AI technology have on society and culture?
Dron said the impact of AI technologies like ChatGPT will be, in short, revolutionary—and not necessarily in a way that will benefit everyone.
“This is a crazy moment that is going to revolutionize everything—in the older sense of the word, it’s turning things over. And many things are going to be turned over.”
“There are jobs that will not exactly disappear but will certainly become rarer than they used to be. There are jobs that will be created. There are ethical dilemmas that we will face that we’ve never had to face before.”
It’s not just ChatGPT that will have this impact, but other similar technology that is using AI to do things differently. For example, Dron pointed to AutoGPT, a technology similar to ChatGPT that can actually do simple tasks for you like finding and booking a hotel room.
And assuming this kind of technology doesn’t “get legislated out of existence,” Dron said we haven’t really even begun to imagine the potential.
“It’s such a fast development, and meanwhile we’re also heading into a whole different realm of things like prosthetics and cyborg things,” Dron said. “You can already interface directly with a computer that can generate language from your thoughts, and so forth.”
He said he expects it won’t be long before it will be common for users to have access to generative AI systems far more advanced than ChatGPT, with non-intrusive interfaces that will silently take input and provide output to users. You won’t necessarily need electrodes or the kind of physical plug-ins popularized in science fiction stories like The Matrix.
“A small portion of the world’s population may have this remarkable superpower, and it is terrifying what this means in terms of the divisions that will be created and the difference in power and capability.”
– Dr. Jon Dron
Who will have access to this technology?
Dron said this question is important, but nearly impossible to answer. There is the possibility that this powerful technology will only be accessible to an elite few.
“A small portion of the world’s population may have this remarkable superpower, and it is terrifying what this means in terms of the divisions that will be created and the difference in power and capability,” he said.
But on the other hand, Dron said while there may be inequalities and they may be significant, right now developments from the open-source community are happening very quickly—within timelines measured in weeks or months rather than years. And the open-source models being developed can be on par with the capabilities of models funded with billions of dollars of investment, like ChatGPT.
“And that’s what makes it so scary and so exciting: the adjacent possibles are exploding exponentially so the future is quite unpredictable,” Dron said. “The pace of change makes it fiendishly difficult to look more than a few weeks or months ahead.”
Learn more about artificial intelligence with computer science programs and courses in the Faculty of Science and Technology, and with the Artificial Intelligence Ethics Micro-Credential from PowerED™ by Athabasca University.
