Sonifying handwriting movements for the diagnosis and the rehabilitation of movement disorders by Jérémy DANNA (BLRI)
Except for the slight scratching of the pen, handwriting is a silent activity. Transforming it into an audible activity might sound curious. However, because audition is particularly appropriate for the perception of fine temporal and dynamical differences, using sounds to gain information about handwriting movements seems judicious. We use the sonification that consists in adding synthetic sounds to silent movements in order to provide support for information processing activities. The idea is to associate a melodious sound, which flows, to a fluent handwriting, and a dissonant sound, which squeaks, to a jerky handwriting. By sonifying the relevant variables of handwriting in dysgraphic children or in Parkinsonian patients, it could be possible to detect their handwritings troubles 'by ear' only.
My talk will be organized in two parts. First, I will expose an experiment showing that adding relevant auditory information is sufficient for discriminating the handwriting of dysgraphic children and the skilled handwriting of proficient children 'by ear' only. I will also present an experiment in progress in which real-time auditory feedback are supplied to help dysgraphic children to improve their handwriting movements. Secondly, I will present the BLRI project that consists in using computerized analysis and sonification of handwriting movements for the early diagnosis of Parkinson Disease.
