You are invited to participate in a research study to improve our understanding of the psychophysiology of students doing AI-assisted software development. The project is conducted by Mihaela Hristova (PhD student at ITU) as part of her PhD project.
The Research Study
With the increasing use of artificial intelligence (AI) in the software engineering industry, the amount of research on adoption and effects has increased. However, the impact of AI on development is an evolving research topic, and there are still areas that remain mostly unexplored. One of such area is the cognitive effect of using AI assistants while writing code.
In recent years, research has found that prompting an AI tool in a correct way enhances the interaction with the tool. We therefore hypothesise that there is a greater cognitive similarity between writing code and writing prompts for an AI than between writing code and working with the output of an AI. Additionally, we hypothesise that working with the output of an AI is cognitively more similar to reading and understanding code than creating new code.
Our study aims to compare the neurological activity of developers while using AI assistants to complete coding tasks and those without such assistants for the same kinds of tasks.
The Procedure
Your participation in the study consists of an experiment of 4 parts, which will take approximately 90 minutes:
- Briefing: In the first part, you will be asked to sign an information statement, and you will be able to ask any questions you may have related to the study, prior to the experiment. You will also fill out a demographics questionnaire.
- Setup: In the second part, we will set up the experiment, calibrate the measurement devices and collect a baseline for our analysis.
- Experiment: In the experiment, you will solve 2 programming tasks, one with the use of AI assistance, and one without (the order is random). You will answer a short questionnaire after each task. During the experiment, you will be monitored with an electroencephalographic (EEG) device, an eye-tracker, a mouse, and a key logger, and the screen will be recorded.
Debriefing: We will have a short conversation about the experiment.
Risks and Adverse Effects
The monitoring devices used in this experiment are non-intrusive, painless and safe. There is a very small risk that EEG can cause seizures if you have a seizure disorder. To ensure the safety of all participants, we exclude people with such disorders from the experiment. Each participant will also receive a coffee mug as a token of appreciation for their help.
Outcomes
The data will be anonymised and used for multiple scientific studies on the topic of AI and programming.
Further Information About the Project
If you have any questions about the study, you are welcome to contact Mihaela Hristova at mihr@itu.dk.
Experiment Booking
By booking the experiment using the following link, you declare that:
- I can write simple projects in Java with multiple classes.
- I’m familiar with Visual Studio Code.
- I’m familiar with simple UML, specifically class, activity and sequence diagrams.
- I’m comfortable writing code on a Danish keyboard.
- I have experience with AI, including chat-based AI assistants (ChatGPT, Claude etc.).
- I have used AI for coding assistance.
- I do not have any vision impairment that cannot be corrected with glasses or contact lenses.
- I do not have a history of epilepsy and seizures.
- I’m comfortable sitting still and focusing for 2×20 minutes.