The Wisdom of ChatGPT: Confederacy and Ficino

A couple of months ago, a version of GPT3, called ChatGPT, was rolled out as a free service to the general public. (And if you don't know that, you are living under a rock.) GPT3 is designed to craft written texts in English. ChatGPT lets the average individual with an internet connection see the results of its work for free. The popular press has been making extravagant predictions about the threat posed to the jobs of all white collar workers, who might be replaced by these AI systems. While the threat is non-trivial, the leap from ChatGPT to the end of human managerial employment assumes that ChatGPT is actually coming up with innovative ideas, rather than just putting non-innovative truisms into readable texts.

I was aware that AI had made large strides in recent years. In 2015, the system AlphaGo, developed by the company Deep Mind, defeated a professional Go player for the first time in a five game match, and in 2016, an updated version dominated the human world champion of Go four games to one. Commentators were struck not only by the victories but by the computer's playing style. Earlier game-playing AI systems, such as Deep Blue in chess, simply out-computed the human opponent. AlphaGo's style showed creativity and flair. It didn't just beat humans, it played like a human, only better. I was not shocked by this development, but I was shocked in December of 2017 when Deep Mind released AlphaZero. AlphaZero was given the rules of chess, was not given any human games as models to learn from, and within a short span of playing games against itself, was able to defeat the computer program that at that time held the title of world chess champion. That speed of learning by the new algorithm was truly astonishing.

I am not an AI expert, but I do have a masters degree in computer science and in 1991, I had studied precursors to the current systems, then called Expert Systems, in a graduate course. Around the year 2000, I was aware that computer scientists had made breakthroughs in machine learning algorithms, which accelerated AI development. With the rise of massive data warehouses generated by traffic on the Internet, these more powerful AI systems had tasks to perform which could generate revenue for the developers, specifically tasks involving language.

With the release of ChatGPT this year, there has been active discussion about the ability of AI systems to write papers for college students. The assumption is that ChatGPT is not only writing flowing prose, it is also formulating ideas and arguing in their favor. The journal Nature has had articles discussing whether it is ethical for scientific researchers to use AI systems to help them write their papers. In this case, the scientist would provide the hypothesis, the data, and the conclusions, and the AI would provide the English prose. Such a development would no doubt help non-English-speaking researchers.

As for myself, I was slow to try ChatGPT, because I wasn't sure how to evaluate it. Enter John Kennedy Toole and his novel A Confederacy of Dunces. I have spent years studying this novel and have read nearly all of the scholarly work published on it. I could ask ChatGPT about Confederacy. On this topic more than any other, I would know how well the system was generating ideas. If it was borrowing ideas from others, I would know where those ideas were coming from.

Thinking along these lines, it also occurred to me that ChatGPT and other systems have a major problem with intellectual property. If the test set of documents contain ideas (which they mostly do), then the output of the AI might also parrot those ideas in the texts it generates. If the documents in the learning set have been copyrighted, the parroted ideas may violate copyright. To test this, I specifically prompted ChatGPT to talk about a subject I have researched in depth. I asked it about the influence of Marsilio Ficino on the novel. Here are the results.

Prompt submitted at 6:25 a.m. (Central time), Thursday, March 30, 2023: In his novel A Confederacy of Dunces, did John Kennedy Toole use the ideas of Marsilio Ficino?

Answer: There is no clear evidence to suggest that John Kennedy Toole used the ideas of Marsilio Ficino in his novel A Confederacy of Dunces. Ficino was a prominent Renaissance philosopher and scholar who wrote extensively on a wide range of subjects, including Platonic philosophy, astrology, and magic.

While A Confederacy of Dunces does contain philosophical and intellectual themes, there is no direct reference to Ficino or his ideas in the novel. The main focus of the book is on the eccentric and dysfunctional characters who populate the city of New Orleans.

It is possible that Toole was indirectly influenced by Ficino's ideas through the broader intellectual and cultural milieu of the time, which was characterized by a revival of interest in classical philosophy and literature. However, without specific evidence from Toole's own writings or statements, it is difficult to say for certain whether Ficino had any direct influence on the novel.

As you can see, the prose is well-crafted. There are even ideas behind the prose. ChatGPT argues that there is no obvious, demonstrable connection. The answer is technically correct, and the computer wisely declines to speculate on something that is not explicitly provable. My arguments about Ficino having influenced Toole are out there in cyberspace, so it could have put forward my ideas and cited my paper. Still, ChatGPT did not violate my copyright by presenting my ideas about Ficino and Toole as its own. I am both comforted and relieved. The system gave a good answer that an articulate human who was not a scholar in the field could give, and it did not steal my ideas or my role as an investigator. It did settle on a thesis, though, so you can argue that it wasn't just handling English words, it was formulating ideas and articulating them.

With all the discussion of how well ChatGPT crafts prose, it has occurred to me that, just as with chess and go, computers may soon be much better than most humans at writing texts. If that becomes the common expectation, then the phrase "passing the Turing test" could flip from being a compliment to the programmers of the computer system to being an insult toward a text. "This essay is so poorly written, it passes the Turing test. It's so bad, a human could have written it."

I will end this blog entry with my second query to ChatGPT and its response. Again, it argues for a thesis, and it does it well.

Prompt submitted at 6:30 a.m. (Central time), Thursday, March 30, 2023: Could chimps at typewriters eventually write the works of William Shakespeare?

Answer: It is theoretically possible for chimpanzees at typewriters to produce the works of William Shakespeare, but it is incredibly unlikely to the point of being practically impossible.

The famous thought experiment of the infinite monkey theorem states that given an infinite amount of time and an infinite number of monkeys randomly typing on an infinite number of typewriters, they would eventually produce the complete works of Shakespeare by chance. However, in reality, the amount of time required for this to occur is so vast that it is effectively impossible.

Even if we consider a more realistic scenario, such as a finite number of monkeys typing for a finite amount of time, the probability of producing even a single coherent sentence is extremely low. This is because the number of possible combinations of letters is astronomically large, and the vast majority of them would be meaningless or nonsensical.

Furthermore, writing is not simply a matter of randomly arranging letters. It requires a level of skill, creativity, and linguistic knowledge that is beyond the capacity of chimpanzees, who do not possess the complex cognitive abilities necessary for language and literary expression.

In short, while the idea of chimpanzees writing Shakespeare may make for an interesting thought experiment, in reality, it is highly unlikely to occur.

This answer shows that ChatGPT does possess "a level of skill, creativity, and linguistic knowledge that is beyond the capacity of chimpanzees." It can also formulate and defend ideas.