Cumhur Erkut, Dr. Sc. (Tech)

Room: 2.0.013a

Aalborg University

A.C. Meyers Vænge 15

DK-2450 Copenhagen SV, Denmark

Edge AI, Active and Perceptual Inference, Quantum Computing (QC) and Quantum Machine Learning (QML) will have a huge transformative potential and impact in the immediate future. My mission is to bridge X-verse, AI, and QC/QML through Active and Perceptual Inference and academic/industrial ML / AI engineering practices within the Spatial Web Standard, developed by the Spatial Web Foundation and the IEEE P2874 Spatial Web, Architecture and Governance Working Group. I work towards this mission within my lab through scientific projects.

I have a PhD on acoustics and audio signal processing from Aalto University, Finland, with minor studies on Information Systems (Machine Learning) and general studies on Extended Reality. I have participated in national (Finland, Denmark, and Turkey) and international projects (EU Framework Programs 5-7, and Horizon programs). Between 2007 and 2012, I conducted independent research as an Academy Research Fellow on sonic interaction design (Schema-SID, #120583, Academy of Finland), together with the FP7 IC601 SID project on the same theme. At Aalborg University, I worked on embodied interaction, neural sound and motion synthesis. and AI Engineering.

I am affiliated with the Pioneer Center for AI, served at Danish Data Science Academy as a mentor in 2023, participate to the steering commitee of Movement and Computing (MOCO) and to the program commitee of EAAI Mentored Undergraduate Research Challenge.

selected publications

IEEE

Exploring the Impact of AI-Generated Speech on Avatar Perception and Realism in Virtual Reality Environments

Anders R. Bargum, Emil Sønderskov Hansen, Cumhur Erkut, and Stefania Serafin

In IEEE Conf. Virtual Reality and 3D User Interfaces - Abstracts and Workshops (VRW). Video, audio, and data can be found at the html link , Mar 2025

DOI Supp
Reimagining speech: a scoping review of deep learning-based methods for non-parallel voice conversion

Anders R. Bargum, Stefania Serafin, and Cumhur Erkut

Frontiers in Signal Processing, Mar 2024

Abs DOI arXiv

Research on deep learning-powered voice conversion (VC) in speech-to-speech scenarios are gaining increasing popularity. Although many of the works in the field of voice conversion share a common global pipeline, there is considerable diversity in the underlying structures, methods, and neural sub-blocks used across research efforts. Thus, obtaining a comprehensive understanding of the reasons behind the choice of the different methods included when training voice conversion models can be challenging, and the actual hurdles in the proposed solutions are often unclear. To shed light on these aspects, this paper presents a scoping review that explores the use of deep learning in speech analysis, synthesis, and disentangled speech representation learning within modern voice conversion systems. We screened 628 publications from more than 38 venues between 2017 and 2023, followed by an in-depth review of a final database of 130 eligible studies. Based on the review, we summarise the most frequently used approaches to voice conversion based on deep learning and highlight common pitfalls. We condense the knowledge gathered to identify main challenges, supply solutions grounded in the analysis and provide recommendations for future research directions.
ACM

Multimodal Looper: A Live-Looping System for Gestural and Audio-visual Improvisation

Pelin Kiliboz, and Cumhur Erkut

In Proceedings of the 9th International Conference on Movement and Computing, May 2024

Abs DOI Video

We present the Multimodal Looper, an embodied digital interface that connects body movements to audiovisual forms for musical improvisation. It extends the performative practice of the musical looper, which enables musicians to record and play back multiple layers of sound in real-time. With the multimodal looper, we explore music cognition from an embodied perspective, considering cross-modal correspondences in order to achieve an intuitive method suitable for collaboration. Our goal is to create a live-looping system with multimodal objects that are gesturally activated and visually represented. Our first prototype focused on the essential modules of a live-looping system: for gesture recognition, we used a depth camera and a decision tree; for visuals that correspond to distinct categories of sounds, e.g., sustained, iterative, and impulsive, we employed procedural generation techniques such as animated noise, feedback loops, instancing, and various mathematical operations. The system’s effectiveness in establishing cross-modal correspondences for a multisensory experience was evaluated through user testing