- This event has passed.
A holistic measure of inter-annotation agreement with continuous data
February/21/2022 @ 12:00 - 13:30
Rachid Riad (École Normale Supérieure – Inria – Inserm)
Abstract: Inter-rater reliability/agreement measures the degree of agreement among raters to describe, code or assess the same phenomenon. Most coefficients (ex: α, κ) measuring these agreements in psychology and natural sciences focus on the categorization of events. Yet, the annotations of speech and especially conversational spontaneous speech represent a complex continuous phenomenon to annotate. There is not only categorization but also the localization of events that is asked from annotators, referred to as unitizing. In this presentation, we will describe the gamma agreement γ introduced by Mathet et al. 2015 and our work to extend this measure with the python package ‘pygamma-agreement’. We illustrate the use of this measure with corpora coming from (1) daylong recordings to study language acquisition, and (2) interviews at the hospital to study speech pathologies.
Where: Zoom link https://univ-amu-fr.zoom.us/j/2515421853