Mandryk, R. L., Inkpen, K. M., & Calvert, T. W. (2006). Using psychophysiological techniques to measure user experience with entertainment technologies. Behaviour & Information Technology, 25(2), 141-158.
This paper describes two experiments designed to test the efficacy of physiological measures as evaluators of user experience with entertainment technologies. The authors described two experiments that were designed to test the two main conjectures:
- Physiological measures can be used to measure a player’s experience with entertainment technology.
- Normalized physiological measures of experience with entertainment technology will correspond to subjective reports.
The First Experiment
The aim of Experiment One was understanding how psychology can be used to measure user experience with entertainment technology. Eight male participants in this study played NHL 2003 in the four different levels of difficulty, beginner, easy, medium, and difficult. The authors collected two categories of measurements. The first one was a questionary which asked participants about boredom, challenge, frustration, and fun in each difficulty level. Participants rated each subjective measure on a scale of one (low) to five (high). The physiological measures were the second category consisting of:
- EKG (Cardiovascular measures)
- HR (Heart Rate)
- GSR (Galvanic Skin Response)
- EMG (Electromyography)
- Resp Rate (Respiratory Rate)
- RespAmp (Respiratory Amplitude)
The authors’ hypothesis for this experiment was that participants would prefer playing in the condition that was best matched to their level of expertise, experiencing the most enjoyment, satisfaction, and engrossment in this condition.
Based on subjective responses, the mean ratings for challenge were significantly different amongst different difficulty levels in spite of boredom, frustration and fun, which were no significantly different.
Based on physiological measures, authors found out that there were no main effects of difficulty level on any of the physiological measures.
Regarding the correlation of physiological measures to subjective results, they correlated the mean of each physiological measure to the subjective ratings for each participant individually because unlike subjective ratings, there are large individual variations in physiological data. They said that a relationship between a physiological measure and a subjective rating would be evidenced by a significant number of the participants showing the correlation between the physiological measure and the subjective rating. They found out that although there were correlation for most individuals, these correlations were not consistent across participants.
The Second Experiment
The authors’ aim in Experiment Two was further understanding how body responses can be used to create an objective evaluation methodology. In this experiment Ten male participants age 19 to 23 played the same game as Experiment One in two different conditions:
- Against the computer
- Against another co-located player
Their main suppositions for Experiment Two were that participants would be more excited, and prefer playing against a friend than against a computer.
Like Experiment One, they collected two categories of measurement. The first one was questionary asking about Boring, Challenging, Easy, Engaging, Exciting, Frustrating, and Fun. The participants again rated these measures on a five-point scale. The second category measurements were psychological measurements similar to Experiment One.
Based on subjective responses, they found out significant differences in Boring, Engaging, Exciting, and Fun between two play conditions.
Based on physiological measures, the authors reported that mean GSR and mean EMG were significantly higher when playing against a co-located player. However, the MANOVA, the technique authors used to analyze physiological measures, showed no significant differences in HR, RespAmp, or RespRate between the two play conditions.
Regarding the correlation of physiological measures to subjective results,they normalized both categories. They reported significant correlations between subjective responses and physiological measures as follows:
- Fun significantly correlated with GSR.
- Boredom significantly correlated with EMG.
- Challenge significantly correlated with RespAmp and EMG.
- Ease significantly correlated with RespAmp and EMG.
- Frustration significantly correlated with GSR and RespRate.
Finally authors reported that all of participants except one participant prefer to play against their friend than playing against computer in the future.
All in all, this paper is really amazing. The authors work was really interesting and they explained it very well. They clearly explained their works and stated what they wanted and how they measured, evaluated them, and how concluded the results. In this part of my discussion about the paper, at first I explain the good points of this paper and then enumerate the negative points.
First of all, I should mention that the paper is really well structured and clear. Every thing is understandable and explicit. This paper is really easy to read and follow, very well structured. They used the same structure for explaining both experiments. The experience design, procedure, every details were mentioned, so for novice researchers in this field, it is really useful. The level of details in describing procedures, data collection, and data analyzing was really amazing. With this level of explanation, this study can be used as a template for other similar studies.
But the most important strength of this paper in my opinion is its results. Based on their results in Experiment Two, the other researcher can identify which physiological measure is suitable for which subjective measurement in studying collaborative situations. For example, based on their results showed in the Table 1, it is better to measure GSR, and RespRate in order to studying Frustration instead of measuring HR and EMG.
Besides all good points, the paper has some negative points that I discuss them in this section. The paper in the section 2 described all of the physiology measurements that the authors used in studying experiments. They described them in details, I think it is not necessary to talking about them in this level of details. They could easily reference these measures instead of describing them again in details.
Regarding Experiment One the number of participants was really low. They recruited only eight male participants that one of them did not completed the task.
Regarding Experiment Two, they just studied ten male participants that were really low for the experiment. Another issue about this experiment was that in playing against co-located partner, each participant played against her friend. The authors mentioned that “We recruited the participants individually, but asked them to bring their own partner. We didn’t want the participants playing against strangers, which may have discouraged people who prefer playing alone from signing up.”, but we can criticize that playing against a friend may not be equal to playing against a stranger, and you cannot generalize your findings about playing against friends to a wider subjects, playing against co-located partner. May playing the game against your friend be enjoyable because they have fun and enjoy being in collaboration, not because of playing against a co-located partner. Therefore, this condition might had some effects on responses and physiological measures. The authors could eliminate this effect in that way that half of players play the game against their friends, and half of them play the game against strangers.
As I mentioned before, this paper is interesting and in explaining every thing was really clear and in details. However, the paper had some negative points, but at last we can say that the positive points do compensate them.