Brain Computer Interface for Neuro-rehabilitation With Deep Learning Classification and Virtual Reality Feedback

Tamás Karácsony, John Paulin Hansen, Helle Klingenberg Iversen, Sadasivan Puthusserypady

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

621 Downloads (Pure)


Though Motor Imagery (MI) stroke rehabilitation effectively promotes neural reorganization, current therapeutic methods are immeasurable and their repetitiveness can be demotivating. In this work, a real-time electroencephalogram (EEG) based MI-BCI (Brain Computer Interface) system with a virtual reality (VR) game as a motivational feedback has been developed for stroke rehabilitation. If the subject successfully hits one of the targets, it explodes and thus providing feedback on a successfully imagined and virtually executed movement of hands or feet. Novel classification algorithms with deep learning (DL) and convolutional neural network (CNN) architecture with a unique trial onset detection technique was used. Our classifiers performed better than the previous architectures on datasets from PhysioNet offline database. It provided fine classification in the real-time game setting using a 0.5 second 16 channel input for the CNN architectures. Ten participants reported the training to be interesting, fun and immersive. "It is a bit weird, because it feels like it would be my hands", was one of the comments from a test person. The VR system induced a slight discomfort and a moderate effort for MI activations was reported. We conclude that MI-BCI-VR systems with classifiers based on DL for real-time game applications should be considered for motivating MI stroke rehabilitation.

Original languageEnglish
Title of host publicationProceedings of the 10th Augmented Human International Conference 2019
Number of pages8
PublisherAssociation for Computing Machinery
Publication date2019
Article numberArticle No. 22
ISBN (Print)978-1-4503-6547-5
Publication statusPublished - 2019
Event10th Augmented Human International Conference 2019 - University of Reims Champagne-Ardenne, Reims, France
Duration: 11 Mar 201912 Mar 2019
Conference number: 10


Conference10th Augmented Human International Conference 2019
LocationUniversity of Reims Champagne-Ardenne


  • Motor Imagery
  • Brain Computer Interface
  • Deep learning
  • CNN
  • Virtual Reality
  • Online EEG classification

Fingerprint Dive into the research topics of 'Brain Computer Interface for Neuro-rehabilitation With Deep Learning Classification and Virtual Reality Feedback'. Together they form a unique fingerprint.

Cite this