Hinder and detect prohibited use of AI

AI is constantly and rapidly developing, which makes it necessary to continuously monitor developments. Some advice and proposals can be found below, divided up into two categories:

Those that can be carried out fairly quickly and without complications, even by an individual teacher, and which revolve around how written homework assignments are formulated.
Those that require a major overhaul of assessment practice, and which may entail more comprehensive and resource-intensive measures.

Finally, a few lines are dedicated to the option of using AI tools to detect texts created by AI tools.

Three general recommendations are particularly important:

Focus on quality, not on cheating! Focus on measures that truly enhance the quality of examination for all students and whose primary focus is not simply on preventing cheating among a small minority.
Take a preventive approach too. The individual proposals below all need to be supplemented by broader, preventive efforts described on the previous page.
Work collaboratively and collegially. This is not an area in which all of the responsibility can be placed upon individual teachers. Procedures are required, along with support from management and the administration, collegial discussions and a systematic exchange of experiences.

1) Formulation of take-home assignments

The following two pieces of advice are relatively uncomplicated to implement and relate to how written take-home assignments are formulated.

2) Review of examination practice

The following proposals, with the exception of the final one, are based on relatively extensive elements of feedback or follow-up. It goes without saying that this requires resources, and each subject area needs to produce its own assessment here of what is reasonable – but on a lot of courses it should be possible to shift more contact hours over to feedback.

Note that in no case is it permitted to introduce new forms of examination on a course if they are not covered by the wording in the current syllabus, so this is a process that can take a relatively long time.

Proposals

Require even short texts to contain references

Start by reviewing the instructions for take-home exams, essays, etc. and require even short texts to state correct references, with correctly formulated source references of the literature used. This should be placed within the text and directly connected to the areas of relevance. Emphasise in the instructions that the references form part of the answer and will also be assessed and, where necessary, followed up.

Fortunately, the essence of this requirement is not about getting help in detecting possible cases of cheating. In actual fact, it can only be a positive thing for students to learn at an early stage to underpin their reasoning with careful references and to recognise the central role this part of the academic craft plays in crafting a scholarly discourse.

But in addition, the AI tools – as things stand, the picture is changing rapidly – often seem to have difficulty producing such references. The answers produced by ChatGPT (not to be confused with a search engine, as they are in fact a tool for generating comprehensible text based on instructions) are not always transparent, making it difficult and laborious to attempt to cheat by manually adding references subsequently to an AI-generated answer. Tools can invent entirely false references, but even in cases where the references really do lead to websites on the internet or articles that truly exist (Perplexity can produce such links), they are conspicuously imprecise and lack necessary page references, for example. For teachers, substandard or strange references can relatively easily offer a hint that a submitted take-home exam at least needs to be followed up more closely in an appropriate way. But even if everything looks good, teachers should get into the habit of taking a few random samples to check the references (and students must of course be informed that this will be done).

The instructions need not stop at merely requiring references:

Specify in the assignment instructions that the students must justify the selection of references they have included, for example
You could also ask the students to discuss the content of their references in detail.

Of course, this is not a solution to every single problem since not all written take-home assignments are suited to including references; other solutions may be needed. [Back]

Formulate questions to which the answer is to be either partially or fully based on, or applied to, local, specific contexts

Formulate assignments in which the students are to analyse, or in some other way relate to, activities carried out on that specific course based on course literature, etc. Alternatively, ask them to base or apply the reasoning in their answers on/to specific situations or circumstances that an AI tool could not easily know about. [Back]

Review which goals are assessed via take-home assignments and other potential forms of assessment

The selection of suitable forms of assessment is normally carried out based on the course’s learning outcomes, and for certain goals, take-home assignments are not the most suitable form of examination. For example, take-home assignments should have less focus on knowledge-based questions and more requirements relating to methodology and skills, such as the ability to select and justify what information and which source material is relevant to answering the assignment, as well as an independent, critical ability to analyse. Therefore, to be on the safe side, start by reviewing your examination practices. Perhaps there are take-home assignments that can very reasonably be replaced by supervised exams, or where examination can instead be conducted in connection with some other type of activity? [Back]

Oral examinations?

Another option that should be considered is oral examination. This may well seem overwhelming purely in terms of time, but it is often possible to assess several students at once – and after all, it also takes a lot of time to produce written exam questions and to mark piles of papers. The point at which it takes an equal amount of time to complete a series of oral exams does not have to be unrealistically far away – and sometimes an oral exam can do more justice to the students' knowledge. Note that relevant syllabi may need to be adjusted unless it is stated that oral examinations may be a possible form of examination.

If considering oral examinations, it is wise to first discuss at the faculty – but also subsequently at the department board and any other bodies that include student representatives – what guidelines should apply to such examinations so that they can be implemented fairly and securely. In the case of oral examination, it is particularly important to have clearly stated grading criteria that the students are aware of in advance and which can help teachers formulate appropriate questions. Above all, it is important to inform the students about the process and how the examination will be held at an early stage of the course, as oral examinations are perceived to be extremely stressful by many students. The Swedish Higher Education Authority’s document Fair Examination also proposes that oral examination can be documented to facilitate any grading reviews.

One option is to hold the oral examination as stage 2 of an assessment whose first stage is the submission of a written assignment. This would give the oral examination a tighter structure. Two such scenarios are described below.[Back]

Give feedback on various versions of the assignment

Essays in the degree project format are almost always produced as a series of versions that are discussed individually or in a group of students. When this type of process is used, whereby texts are produced gradually and their structure is discussed and adjusted continuously, it can be harder to rely on AI-generated texts. Of course, not all assignments are sufficiently extensive in scope to justify using this kind of process, but even for many short essay assignments, elements of discussion of drafts would make it harder to stand by a text that a student has not written themselves.

One option is to allow this type of discussion to be held partially or fully through some form of peer feedback, which in itself is an important element of learning. Especially early on in a programme, it can be important to provide students with some kind of support materials and examples that explain which components are important to enquire about.

Regardless of whether draft discussions are conducted with the teacher or fellow students, it is crucial that they are not carried out as, or perceived to be, a way of preventing cheating, but for them to form part of teaching and provide support for the students. If they make it less attractive to cheat in the process, that is merely a positive side-effect. [Back]

Consider phased examination

Not all take-home assignments consist of running text. Answers that largely consist of more formulaic text, such as programming code or mathematical or other formulae, cannot be underpinned by references in the same way. There are perhaps two paths to go down here: either move the examination to some kind of hall-based, invigilated exam (see further down on this page), or consistently invest resources in oral feedback, whereby the teacher meets the student – alone or in small groups – and has a discussion about the answers, with the student needing to orally justify and explain their choices. Naturally, in this case the students need to know in advance that the examination is not complete until after this second, oral stage: in this case, the feedback is given in connection with the examination itself. Check that the formulations concerning examination in the syllabus allow for this.

This would require resources in terms of contact hours, but it need not be a negative thing to redistribute contact hours from teaching before the exam to feedback on submitted work. It is an age-old problem that teachers have too little time for feedback and/or that feedback is not always sufficiently taken into consideration by students, despite it representing one of the most valuable elements of teaching. [Back]

Use the University’s examination halls in new ways

The University’s examination halls are often relatively empty. By opening up the examination halls on certain days for independent writing, we can take advantage of that opportunity to fully or partially check access to online aids. The tools are in place for allowing students to come at a time of their choosing to the examination hall during its opening hours and take exams in several different ways:

Using locked computers whereby the students log in as during an ordinary exam and have a limited timeframe to write answers using permitted resources they have brought from home (books, lecture notes and so forth) as well as approved, permitted online resources.
As above, but with the opportunity for students to return on multiple occasions to the examination hall and receive a new log-in for the same assignment with help from the invigilators. This would allow the assignment to be completed over the course of several days, yet still in a controlled environment.
Using open computers where the students have full access to the internet and the resources they bring from home, but with limited opportunities to collaborate with other students. The possibility of logging each student’s web history should be investigated, as an entirely open computer offers the chance to use chat rooms and social media.

Whenever an assignment is to be submitted, students would be required to submit them from the examination hall. This can be controlled, as a second log-in to the same assignment requires assistance from an invigilator, who gives the student access using an administrator password. It would thus be impossible for the student to start working on an assignment in the examination hall and complete it at a later date from another location using this method. However, it would be possible for students to return to the examination hall and receive a new log-in with help from the invigilators’ administrator password.

A written assignment can therefore be configured for students in several different ways:

As an assignment to be completed during a limited but flexibly structured timeframe. Settings can be configured in a way that gives students a limited time to respond from the time they begin the assignment, but they can choose when to start.
As an assignment that can be completed across multiple sittings. In this case, students can return to the examination hall another day or time to continue their work. They would not have an assigned computer or need to use a specific hall. For example, the assignment could be started at Bergsbrunnagatan and continued at Råbyvägen, but students would not be able to log in independently from home or from another computer room at the University.

To discuss possible options, contact tentamensamordning@uadm.uu.se [Tillbaka]

Can we produce new tools that detect AI-generated material?

Trials have already been carried out to see if AI tools can be used to check whether a text has been generated using AI. Several tools are available. For example, OpenAI has released its own AI Text Classifier. It is not perfect, however: it does not capture all AI-generated texts, it can also misclassify authentic texts as being written with help from AI, and only small changes in an AI-generated text are needed to pass the review.

These tools will undoubtedly be developed too, but so far there is no tool that allows teachers to carry out such checks systematically. Instead they need to carry out random checks, and in the end it still needs to be a question of the teacher's judgment, as is the case with the text-matching system. As technological developments are moving so rapidly, the most secure option is to not put our trust solely in technical solutions. Instead, modifying examination practice is a more sustainable and long-term solution that could also lead to other qualitative advantages for students’ learning. [Back]