The MIDIator is a software tool that has been developed at the Piano Lab to analyze a student’s piano performance. With the MIDIator, researchers, teachers and students have an objective measurement of a performance that can be used as a basis for studying the accuracy of the playing and the quality of the expressive elements. This tool can also be useful to follow progress over time (Shirmohammadi, Khanafar, & Comeau, 2006 ).
An online digital video library of piano teaching was developed and a multiple case study methodology was employed to examine the experiences of five piano teachers as they interacted with this tool for four weeks. Findings from this study indicated that viewing the online videos was a beneficial professional development activity, which facilitated teacher learning and which could be immediately carried over into their teaching situation resulting in increased student success (Brook, 2007).
Following the success of this initial study, more than 700 piano teaching video clips were filmed and categorized at the Piano Lab, and an online video database created. These videos show multiple teaching strategies, various approaches to piano technique, activities in music reading, presentations of period instruments and examples of technology usage in the piano studio. The value and the benefits of this resources for research and teacher-training purposes is now being evaluated.
(The online digital video library webpage)
In a first project, we looked at the design and implementation of a flexible and easy-to-use multi-camera acquisition setup for markerless human gesture monitoring that could provide non-invasive observations of a pianist’s fingers, hands and body movement. We were then able to highlight the advantages and limitations of such a system for assisting in piano pedagogy research (Cote, Payeur, & Comeau, 2007).
Using three-dimensional motion tracking, we developed two innovative applications to explore how 3D visual feedback could help with maintaining a good posture at the piano. First, through basic motion-capture techniques, we were able to create a 3D skeletal reconstruction of a student performing at the piano. This reconstruction could then be displayed through an interactive user interface that permits the visualisation from any angle, making it possible to observe the performer from any possible view. In a second project, we reconstructed in three dimensions the posture and body motions of a professional piano player so that it could be compared to the posture and movements of various students, by simply overlaying the reconstructed model over two-dimensional videos of students playing the piano. We were then able to study how this visual feedback can help a student noticing potential postural problems when playing the instrument. The possibility of using motion-capture technology to record piano posture and then to impose the 3D reconstruction above a range of available video performances proved to be an innovative application than can enhance and offer an additional teaching aid for the instructor (Mora, Lee, Comeau, Shirmohammadi, & EI Saddik, 2007).
We studied the Microsoft Kinect sensor to find out if it is able to track and quantify the characteristics of the upright posture of pianists. We wanted to find out under what condition Kinnect would be a useful tool in tracking the characteristics of the neutral postures specific to different performers. Up to now, results indicates that Kinect does not have a high enough resolution for measuring posture variables involving head, shoulder, and spine positions for the purpose of comparing the measurements over multiple trials. Presently, the technology is best suited for gaining qualitative information about the position of the head and shoulder, or general movement trajectories that involve large range of motion (Payeur et al., 2014).
We are also exploring whether Dartfish, video-based motion-tracking software used in elite and professional sports, could provide an effective tool for quantitatively tracking and measuring a pianist’s body position during live performance in order to compare posture variables before and after somatic training interventions. We have tested the reliability and repeatability of distance and angle measurements in a controlled lab situation. So far, the results indicate that Dartfish measurement error is small enough to permit quantitative measurement of pianists’ body postures and could be used effectively in studies with live subjects (Beacon, 2015).
Future project: We have examined Dartfish in its capacity as a quantitative measurement tool to analyse piano performance. We now want to investigate its suitability as a feedback tool for teaching somatic training in piano pedagogy. Since webcams, smartphones and portable video cameras provide widely available devices for collecting video date quickly and easily, Dartfish could be used to help piano teachers instruct students about body positioning during piano lessons.