Cooperating with fellow players by representing tangible objects in a virtual world!
Huynh, D., Raveendran, K., Xy, Y., Spreen, K. & MacIntyre, B. (2009). Art of Defense: a collaborative handheld augmented reality board game. Sandbox ’09: Proc. of 2009 ACM SIGGRAPH Symposium on Video Games, 135-142. doi: 10.1145/1581073.1581095
The authors of this paper were interested in exploring the affordances, constraints and player experience of mobile (handheld) augmented reality (AR) games. One of the key motivators is the ability to collaborate and socialize while playing such games; thus they analyze such products as social games. To explore all this, the authors created a game called Art of Defense (AoD), a tangible AR game that requires the players to interact with board game elements in the real world to drive the virtual simulation. The authors first discuss the technical details behind the game, the design considerations and, finally (and most importantly) the player experience. To evaluate player experience, the authors conducted a formal study with twelve participants. Participants played the game in pairs and were recorded on video, and observed, with a feedback questionnaire and interview following the session. From these results, the authors present their lessons learned from the participants’ experiences and discuss considerations for new tabletop AR games.
Structure-wise, the paper is very well organized and the writing really flows from part to part. That’s really all there is to say about that. Content wise, the paper provides an excellent introduction to augmented reality, its current uses in gaming, its caveats, and important considerations for AR game designers. The authors walk us through their design process and motivations. They state from the start that their intentions were to explore the experience side of tangible AR gaming.
Let’s cut to the chase; we’re not here to talk about AR gaming or considerations. If you want to know about what they did, read the paper! We’re here to discuss player experience, and that’s exactly what these authors were after in this paper: determining player experience in the domain of AR games. Although the AR content of this paper is wonderful, I have a few concerns with their evaluation of player experience, and I can also identify a consideration concerning player experience specifically in mobile AR gaming.
In several collaborative AR-related papers (Billinghurst et. al, 2001a; Billinghurst et. al, 2001b; Billinghurst & Kato, 2002; Grasset et. al, 2005), the authors set the stage for multiple methods of collaborating with AR-based products. Mark Billinghurst is truly an influential contributor to the field of AR, and he has practically written a slew of gold-mine papers that frequently focus on the collaborative applications of AR and user experience in specific areas of the game. As such, there are plenty of papers that reflect on players’ claimed experiences while playing AR games. So, if you’re looking for a gold-mine of AR design considerations and stuff, there is certainly an abundance of papers to tell you about all this stuff.
Now, what I mean by ‘claimed’ experience is the reliance on a player to describe, using words, his/her experience playing a game. Questionnaires and interviews, as used in the AoD paper and many others, are good for getting a broad idea of how players feel, but we all know that this is horribly inaccurate because players may not say what they mean or tell the whole truth!
This AoD authors rightfully state: “Designing handheld AR games is challenging… because we do not yet understand how to create effective play experiences.” I agree with this statement completely, as I also have first-hand experience in the difficulty of creating an engaging mobile AR experience. I can also try to explain why the authors’ statement is so accurate. The main reason is that everybody is relying on evaluation questionnaires and interviews! The paper contributes to this list of ‘claimed experience’ papers, because, like many others, they did not attempt to read true affective measurements. Thus, the main concern I have with this paper is that there are no affective measurements. Future studies evaluating this kind of product could make use of EMG and skin conductance data to evaluate what players are really feeling. If not exact, this data will still have a much higher confidence level than simply asking players how they feel, again because most people are either not entirely honest or they simply do not know exactly how they feel. This would also be an opportunity for me to go about my evaluations in the same manner; it really is not easy to develop a good mobile AR game because we don’t know how to get players to enjoy them. Additionally, since we are on the topic of collaborative gaming, this would give us the opportunity to find out how players feel about collaboration and competition in AR games! Also, having only twelve participants does not give a broad enough view of the audience; studies of this nature should have more play sessions to acquire more data.
Another important consideration that I would like to point out is the effect of having a handheld device to display the game world. This is as opposed to some other method of delivery, such as a HMD or shared screen to display the virtual component of the AR. It takes a bit of control and concentration to constantly hold your phone or iPad in place; I bring this up because it can be a problem when dealing with a game like AoD, one that uses a tangible interface. The problem arises because players must focus on interacting with the both tangible parts and the display at the same time. This is really speculative, but I think this would have an effect on biosensor data because players have to constantly monitor that their manipulations in reality are having the desired effects in virtual space; looking back and forth can be strenuous. Thus, having a hands-free solution such as a HMD could help. That’s just my opinion on the matter, but I think this brings up a considerable point for mobile AR designers.
I thoroughly enjoyed this paper, because it involves both a high-level aspect of affective game evaluation and my current research interest, AR. The authors raise some interesting points which I find incredibly valuable since I hope to do some AR game development sometime soon. In the augmented reality domain, they chose not to focus on algorithms and tech, which is okay, but rather the evaluating gameplay elements, which bridges their content to affective gaming and player experience. In the AR domain, my opinion is they did a great job.
In the affective and experiential domain, this paper is… a good start. What really needs to happen in order to better understand the impact of AR games, especially ones with tangible interfaces, is that we need to experiment with biosensors to get real affective data and stop relying on peoples’ opinions. The paper definitely provides a good base for a game to study, but also exposes these considerable points.
Overall, a good work, it just could have used better evaluation methods.
Additional Readings & References
Billinghurst, M., Kato, H. & Poupyrev, I. (2001). Collaboration with tangible augmented reality interfaces. Proc. of HCI International, 234-241. Retrieved from http://www.hitl.washington.edu/pubs/
Billinghurst, M., Kato, H. & Poupyrev, I. (2001). The MagicBook: a transitional AR interface. Computers & Graphics, 25(5), 745-753. doi: 10.1016/S0097-8493(01)00117-0
Billinghurst, M. & Kato, H. (2002). Collaborative augmented reality. Communications of the ACM, 45(7), 64-70. doi: 10.1145/514236.514265
Grasset, R., Lamb, P. & Billinghurst, M. (2005). Evaluation of mixed-space collaboration. ISMAR ’05: Proc. of 4th IEEE/ACM International Symposium on Mixed and Augmented Reality, 90-99. doi: 10.1109/ISMAR.2005.30