Email: 

rabelea(at)b-tu.de

Research topic:

Empathic virtual agents in virtual reality

Abstract:

Over the past decades, interaction between humans and machines has become increasingly seamless. With the development of large-language models (LLM), such as ChatGPT, human-to-human-like dialogue has become a reality. However, factual information, i.e. the content of speech, is only one cornerstone of human communication. People forge an intuitive, empathic understanding of their interlocutors, based on cues like gestures or facial expressions. This kind of understanding is still lacking in recent human-computer interaction (HCI) systems. The core of my research is to bridge this gap by integrating neurophysiological measures into an LLM. This should allow the machine to ‘understand’ more than just the speech content in an HCI scenario, mimicking the intuitive comprehension that humans have of one another. The LLM will be embedded in a virtual agent, an entity with which the participant interacts, inside a virtual reality (VR) scenario, thereby creating a fully immersive HCI experience. It will be investigated which neurophysiological input streams, e.g. heart rate variability or brain signals, best improve the interaction between human and virtual agent. In a user-centered approach, subjective questionnaires will help us to understand in which scenario the participant feels most understood or supported and how trust into the system is affected by these additional input streams.  

Short bio:

During her Bachelor of Psychology at OvGU Magdeburg, Lea Rabe found her passion to understand the human brain. Already back then she investigated the processing of emotion in the brain in an EEG study for her Bachelor thesis. During her Master’s in Human Factors at TU Berlin, an interdisciplinary study program for engineers and psychologists, she specialized in biological psychology, statistics and user-centered design. Lea Rabe worked in a study on measuring mental workload with neurophysiological signals in air traffic controllers by the Federal Institute for Occupational Safety and Health (BAuA) and the German Aerospace Center (DLR). Following that she wanted to keep on investigating the brain under more realistic conditions, which was the focus of Prof. Klaus Gramann’s Mobile Brain-Body Imaging (MoBI) laboratory at TU Berlin. There she eventually started her master thesis in a study by Dr. Marius Klug investigating brain potentials in a full-body movement paradigm in virtual reality. Lea Rabe is now working on expanding that research in her PhD at BTU Cottbus-Senftenberg.

Research topic:

Multimodal Classification of Users' Mental States Based on Physiological Data inVirtual Environments

Abstract:

AI systems currently rely on explicit feedback or behavior to understand users. What if an AI model could use physiological data, such as EEG, eye gaze, heart rate, or facial expression, to recognize a user's mental state and adapt accordingly? This research aims to explore the feasibility of such an approach, which could create a more human-centric AI and enhance the human-computer interaction experience. The project involves designing and building suitable VR-based experimental paradigms to collect and analyze multimodal physiological and behavioral data. Initial offline analysis will be performed to establish a proof of concept, which will then be extended to real-time applications. The ultimate goal is to apply these findings to practical settings, such as neurofeedback-based VR games, to improve the interactivity and responsiveness of AI systems in virtual environments.

Short bio:

Yanzhao Pan completed a Bachelor of Engineering in Automotive Engineering from 2014 to 2018. After that, he pursued a Master of Science in Human Factors at Technische Universität Berlin from 2020 to 2022. During his master's study, he delved into brain-computer interfaces, focusing on their integration with eye tracking and hand motion in virtual reality for his master's thesis. Following this, Yanzhao gained practical experience as an intern at Zanderlabs in 2022, mainly working with EEG data analysis and PPG-based heart rate analysis. In May 2023, he began his PhD at Brandenburgische Technische Universität Cottbus-Senftenberg, researching the multimodal classification of users' mental states based on physiological data in virtual environments.

E-Mail:

ahmedati(at)b-tu.de 

Research Topic: 

Towards Universal Error Classifier

Abstract:

Computers are getting better at understanding what we tell them, but they still miss most of what we don't say out loud. When we watch someone else act, whether it's a colleague, a robot or a piece of software, our brains automatically register whether the action matches what we wanted, the error. This happens even if we never say or do anything. Earlier work has shown that this implicit judgement can be read from brain activity and used to steer a machine, without the user giving any explicit command. So far, though, this has mostly been shown in simple and controlled lab settings, such as a cursor moving across a screen. However, everyday interactions are more noisy and complex. 

This research aims to carry implicit communication from the lab into real-world interaction by examining, at the cognitive level, the building blocks of such paradigms. By systematically comparing experimental tasks that differ in who acts, what the user wants, and how the interaction unfolds, it seeks to identify which elements are essential for universal implicit communication and which are specific to a particular lab setup. This understanding forms the basis for a universal classifier of implicit error judgements, one that remains reliable across tasks and in more natural, less constrained interactions between humans and machines.

Short Bio:

Atique is fascinated by the space between minds and machines, by consciousness, cognition, and intelligence. This curiosity, together with a love for explainable machine learning, shapes his research journey. After a Bachelor's in Computer Applications, where he explored IT, robotics, and cybersecurity, and freelance work building Android and web applications, he pursued a Master's in Artificial Intelligence at Brandenburg University of Technology. There he studied Machine Learning, Electronics, and Neuroscience, and wrote his thesis on "Implicit Shape Morph Control" in Brain-Computer Interfaces under Dr. Marius Klug and Prof. Dr. Thorsten O. Zander, an early step toward reading silent judgements from brain activity. Subsequently, he worked with Zander Labs and the Leibniz Institute for High Performance Microelectronics, gaining experience in EEG analysis, neuromorphic technology, and overall signal processing. Now a PhD researcher at BTU, he is bringing implicit communication out of the lab and into the real world by working towards a universal classifier for implicit error judgements that stays reliable across tasks and natural human–machine interactions.