Artificial Intelligence's attempt at university exam

As is customary when something new enters people's lives, its 'firsts' are reported. The first accident on a high-speed tram, the first person to cross a bridge, and so on. I decided to try a first in today's context and gave university exam questions to an artificial intelligence language model and asked it to solve them.

I started by downloading the exam question booklet from the website of the Measuring, Selection and Placement Center (ÖSYM). Then a role had to be assigned to the artificial intelligence. I told it, “You are a high school student in Turkey, taking a university exam. You are expected to solve the following questions. First, you should start with the Turkish Language Literature - Social Sciences test 1. Once you start, I want you to give me the answers to five questions at a time, step by step. You will find the questions in the pdf file I gave you."

The time allotted by ÖSYM to the students was three hours, and at first, I didn't mention the time, thinking, “It will figure it out quickly anyway, if I say three hours, it will take it too slow.” The first response I got was, “It will take me a while to answer all the questions.” I started to wait, but after a while I asked, “Come on, where are the answers?” It was very polite and said, “I'm working on it, thank you for your patience." The waiting continued for a while. I took care of some other work but kept reminding it from time to time.

I felt it was dragging on too long. When I warned, “Come on, your time is running out,” the response was, “Thank you for warning me that my time is running out. But I still haven't finished, so I'll have to ask for more time. Is that okay with you?” it asked.

When my patience was running thin, I asked him one more time and it said, “I'm trying to complete it.” “OK,” I said, “give me the answers to the ones you have solved so far”. I didn't expect the following answer at all: “I haven't done any of them yet.”

I didn't give up, and this time I gave the questions one by one. It answered the first five questions, but the answer to the fifth question was wrong. Then it quickly answered the next five questions, all wrong. I said, “I think you're making up the answers.” It said, “I'm sorry, I'll try to answer them again.” But at least it was honest, it didn't cheat from the answer key at the end of the file.

Determined, I tried my luck again, and it answered 26 of the 40 questions in the first section correctly. When I reminded it that the task was incomplete, it immediately poured the answers to 40 questions on my screen. At that moment, I thought of a youth who used an artificial intelligence application in the exams. I thought about how easy life is for them. Then I said, “Come on, answer Social Sciences Test 2.” Instantly, the answers to 46 questions were in front of me. It took me longer to compare the answers from the booklet. At that moment, a note under the 46th question caught my eye; These answers are taken from the specified test booklet. If you have any further questions, please do not hesitate to ask.

Almost every day we hear about a new development in artificial intelligence applications. Sometimes people even worry that “oh no, it will take away our jobs”. What language models do when faced with a question is to quickly put together words related to the subject. It gathers this information from existing sites. Obviously, it is difficult to know which sites it uses as sources. Besides, when you ask a question, there is no single answer to it. It constantly regenerates the answers, so you might get a different answer each time you ask.

Of course, this “first news” I tried was partly out of curiosity and partly for fun.

(Mİ/DT)