In the last seven months, I have posted four times on the results of AI chatbot tests. For each test, I used the same corpus of documents of my own related to John Kennedy Toole. I then tested by asking the same four questions to each system. The first test (reported in September) ran this standard battery against a simple, out-of-the-box ChatGPT chatbot with no additional prompt engineering. The second test (reported in October) ran that same battery against an out-of-the-box chatbot with no prompt engineering built with Google's NotebookLM. For the third test (reported in November), I went back to the ChatGPT chatbot but applied a bit of prompt engineering. The fourth test (reported last month) used the same corpus of documents, the same battery of questions, and the prompt from test three, and it ran them against a chatbot created in Microsoft's Co-Pilot.
As I said last month, the test I have conducted for all of these chatbots is a bit particular: I want the chatbot to digest a set of documents and then use the "facts" in those documents to answer questions. One might complain that this is not the only use case for an AI system. Nevertheless, it is a use case that has a wide application in education: to use a chatbot as a learning assistant to guide a student's learning of the content held within the corpus of documents.
The question I want to pose this month is: why create an AI learning assistant? Why not just assign the student to read a chapter in a textbook? My cynical answer is that students need to be motivated to engage with the corpus of resources in a discipline. AI chatbots are the latest technology; therefore, for students to get the texts on the subject through a chatbot piques their interest.
If you consider the brain of the student to be a neural network (call it NI, natural intelligence), and the course to be the mechanism for training that NI network, then the textbook and other course readings are part of a method for training. Students need to be motivated to spend many hours training their NI systems. In theory, using an AI system to query the domain knowledge might actually be detrimental to training the NI, because the students don't have to do the work to build their own network of weighted neural connections.
Students might see an AI system as a way to have a command of a subject without having to do all the work to train their own brains with the disciplinary content. However, students may spend hours querying and interacting with the chatbot. Building a chatbot as a learning assistant who supposedly does all the work for the student is a way to trick the students into being motivated to spend the necessary time to build their own NI networks, that is, to learn.
