Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry

Flavia D. Frumosu, Nicole N. Lønfeldt, A. -R. Cecilie Mora-Jensen, Sneha Das, Nicklas Leander Lund, A. Katrine Pagsberg, Line K. H. Clemmensen

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

62 Downloads (Pure)


Observation is an essential tool for understanding and studying human behavior and mental states. However, coding human behavior is a time-consuming, expensive task, in which reliability can be difficult to achieve and bias is a risk. Machine learning (ML) methods offer ways to improve reliability, decrease cost, and scale up behavioral coding for application in clinical and research settings. Here, we use computer vision to derive behavioral codes or concepts of a gold standard behavioral rating system, offering familiar interpretation for mental health professionals. Features were extracted from videos of clinical diagnostic interviews of children and adolescents with and without obsessive-compulsive disorder. Our computationally-derived ratings were comparable to human expert ratings for negative emotions, activity-level/arousal and anxiety. For the attention and positive affect concepts, our ML ratings performed reasonably. However, results for gaze and vocalization indicate a need for improved data quality or additional data modalities.
Original languageEnglish
Title of host publicationProceedings of Workshop on Interpretable ML in Healthcare at International Conference on Machine Learning
Number of pages7
Publication date2022
Publication statusPublished - 2022
Event38th International Conference on Machine Learning - Virtual event
Duration: 18 Jul 202124 Jul 2021
Conference number: 38


Conference38th International Conference on Machine Learning
LocationVirtual event
Internet address


Dive into the research topics of 'Interpretability by design using computer vision for behavioral sensing in child and adolescent psychiatry'. Together they form a unique fingerprint.

Cite this