VirtualSign Game Evaluation

Abstract

This paper presents the evaluation methods and techniques applied to the serious game developed within the VirtualSign project as well as the results achieved through those methods. VirtualSign is a Portuguese sign language bi-directional translator with three main components. The components are a gesture to text translator, a text to gesture translator and a serious game. The serious game aims to make the process of learning sign language easier and enjoyable using the VirtualSign bi-directional translator. The method used to evaluate the VirtualSign was the Quantitative Evaluation Framework (QEF). The translator undertakes a rigid validation process by both sign language experts and the deaf community. The evaluation process was also supported by questionnaires. The precision of the translator has a direct impact on the game performance. To evaluate the precision of the automatic translator we have used QEF and 10-fold cross validation to estimate the accuracy of the translator. The experimental results show a precision above 90%. In the identification of the Portuguese Sign Language terms As for the game the quality rate is 88% based on the QEF evaluation.