Statistical comparison of differences in usability feedback across ethnic groups was verified using the Kruskal non-parametric procedure. Groups means were used for the comparison.
The comparison of three distinct ethnic user groups indicated that differences in usability feedback and attitudes exist.
The Indian/Pakistani group of users was consistently more positive and less critical about the prototype tested.
The American group presented the highest number of problems detected among all three groups and was the most critical group.
The Chinese/Korean group presented an intermediary result, if the number of problems and answers given in questionnaires is considered. In terms of qualitative answers, the Chinese/Korean group presented the least usable results, perhaps due to the difficulty of verbal, expontaneous spoken skills (which could be due to a combination of language and cultural differences). The American Group was the most usable group, in this regard.
The lack of good interaction between the observer and the Chinese/Korean group in this study limits the validity of the above results. The amount and quality of feedback generated in evaluations of this nature is dependent on the interaction between observers and participants.
Overall, having three ethnic user groups in the study allowed for a wider range of issues and perspectives to be considered that would not have become apparent if only one user group was targeted. This issue is particularly important when developing educational software for international audiences.