First, I would like to acknowledge that a number of people offered advice on this topic, and they generally know more than I do about philosophy and education. I would name them, but then I would want to have their approval, and I take ultimate responsibility for this text.
On October 22, 2025, I was a co-presenter of a talk at Winona State University to the informal AI Discussion Group along with Professor Malgarzata Plecka. I billed the talk as "The Reverse Turing Test." The basic thesis of the talk was not groundbreaking in the least: if you are trying to discern whether an academic text was generated by AI rather than written by a person, then one strategy is to check the footnotes and references. AI systems will sometimes fabricate references to articles that don't really exist. If a submitted paper has fabricated references, a reviewer can reject it not because it was generated by AI, but because it is a flawed paper. It was probably generated by AI, but that is almost beside the point.
As part of that talk, I digressed onto the philosophical issues pertaining to the situation. Namely, I touched on philosophical theories of Truth. In last month's entry for this blog, I compared "truth as factual accuracy" with "truth as loyalty." A friend of mine who is more familiar with these issues pointed me toward a philosophical position called "The Correspondence Theory of Truth." Does a given statement correspond to a state of affairs in the world?
I sent this paragraph to a philosophy professor:
Some schools of thought deprecate the belief that there is an external, objective, observable world at all, much less one that can be measured. I am reminded of the joke about the drunk at the bar during the philosophy conference who said, "Solipsism works for me, but that’s just one man’s opinion."
That professor pointed out that there are a variety of philosophical schools that don't fall into this dichotomy of "correspondence theory of truth" or solipsism. There are deflationist theories and pragmatist theories which still believe in the external world. Further, one can deny the existence of the external world without being a solipsist, such as in Bishop Berkeley's philosophy of idealism.
Circling back to factual accuracy, though, even if you don't buy a theory similar to the correspondence theory in general, it is neverthesless the case that a research library is part of a scholarly correspondence theory of truth. Even for scholars who cast a jaundiced eye toward the value of telling the truth, they do largely adhere to a different commitment to factual accuracy: within their own scholarly texts, they try to accurately quote and reference other writings.
For example, I have read a book by Melvin Pollner called Mundane Reason. Pollner develops a theory of ethnomethodology, which he then uses to question the existence of the objective, determinate world. Truth is socially constructed, and a community will develop a unitary world of social truth.
Even if I am a scholar who is a follower of Pollner or Bishop Berkeley, I will still adhere to a correspondence theory of truth in my production of my own scholarly texts. If I claim a quotation from one of their texts or a text that discusses them, I will make a citation that will correspond to the location of that statement in the objective, observable universe of texts managed by libraries.
These musings are relevant to a discussion of AI LLMs, because LLMs generally string statements together without worrying whether they are factually accurate statements about the universe of scholarly texts. ChatGPT will often just make crap up. Some commentators politely refer to made up quotes and references as hallucinations. Others rely on Harry Frankfurt’s definition and refer to the act of making up stuff, including references, as bullshitting. Some library researchers have created a metric for testing LLM systems called the Reference Hallucination Score. See citations below.
The scenario that we presented on October 22nd was this: a researcher has claimed to have authored a paper that was submitted to an academic journal. The editors of the journal have a policy of not allowing texts that are generated by AI. The basic justification for detecting AI-generated texts is that they should be identified and rejected. Why should they be rejected, though?
Obviously, if a student is assigned to write a paper without the assistance of AI, and the professor can show that the paper was generated by AI, that is a violation of the assignment. But if the paper is a genuinely brilliant and well-executed paper, perhaps an academic journal should consider publishing it, even if it is generated by an AI LLM. Of course, if the references in the paper are fabricated, then the paper is by definition flawed, regardless of who or what generated it.
During the discussion after the presentation, a professor argued that he uses AI LLMs so extensively that a text under his name is almost always a fusion of his own words and those of the AI LLM, and that he himself might not know which of the two crafted any one passage. The basic motive for detecting AI-generated texts is that human-generated texts are preferred. The basic point of the other professor's claim of blending is that the AI-generated text is superior to the human-generated text and should be utilized. Those two positions are not necessarily opposed to one another.
The core of the problem is whether the human claiming authorship is actually the author of the work. Here I offer a modified definition of author, namely: the human is the author if that human is an authority on the content in the work, and the human takes responsibility for what it says. An AI LLM might have assisted with the wording, or even some of the ideas, but the human should have both a command of the content and editorial control of the final text. After all, human ghost writers have been around for millennia; this is not a new concept.
If a paper has fabricated references, then it is likely that the purported human author left the generation of the text entirely up to the AI system. The human might not understand what the paper says or what it contains. For a paper that is entirely composed by an AI system, the human is not the author of the paper under this definition.
If, on the other hand, the human is fully in control of the subject matter and is using the AI system as a tool to improve the final product, then that case probably falls on the acceptable end of the spectrum. That assertion of authorship, though, would involve actual work by the human.
For instance, if the AI system offers up references, the human would be responsible for actually tracking down those texts (and in the process verifying their existence), reading them, and establishing that they are relevant to the argument. This problem of AI-fabricated references is related to the long-standing problem that arises when human authors borrow citations from earlier human-written texts without verifying them. If the earlier paper has a mistake in the reference, that mistake will be perpetuated by later authors who fail to verify the reference and read the cited work.
You can even couch this discussion within a larger context of education: To me, the point of the university is 1) to educate students, 2) to expand the realm of knowledge through research and scholarship and creative works, and 3) to preserve that realm of knowledge and share it with the broader community.
Educating students can be thought of as the process of building neural networks. However, they are the biological networks made of actual neurons and located between the ears of the students, not neural networks located in massive data centers (which in the industry are sometimes called "multi-layer perceptrons"). If the students (or faculty members) rely solely on an artificial neural network, they are often not building the biologically-based neural network in their own brains, and we are failing to educate them.
A complicating aspect of today’s world is that AI is ubiquitous. One of the things our students—and our faculty—will have to learn is how to delve into AI systems to leverage their own knowledge while still retaining intellectual control and responsibility. ( "Delve" and "leverage" are two words commonly used by AI LLMs when writing prose—as are m-dashes.) That authorship requires work, and AI systems make it easy to skip the work and masquerade as the author. Worse, employers may require workers to give up their authorship to increase productivity (that is, increase the quantity of product, not its quality).
NOTE: None of the above text was generated by an AI LLM. It is all the product of the wetware inside my skull, plus the broader collective knowledge held by the community of people with whom I consulted and written texts which are part of my culture (and the product of cultural evolution).
Bibliography
Aljamaan, Fadi, et al. (2024). "Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study." JMIR Medical Informatics 12 (1): e54345. doi: 10.2196/54345
Hicks, Michael Townsen, James Humphries, and Joe Slater. (2024). "ChatGPT is Bullshit." Ethics and Information Technology 26: 30. doi: 10.1007/s10676-024-09775-5
Pollner, Melvin. (1987). Mundane Reason. Cambridge UP.
No comments:
Post a Comment