Toward a Universal Game Design Assessment Tool: Establishing a Game Design Patterns Rubric

Charles Kinzer · Dan Hoffman · Selen Turkay · Caitlin Nagle · Nilgun Gunbas

Thu., June 10, 11:00–12:30, Inn Wisconsin

Educational game development is increasing in popularity and is tapping into a huge, lucrative market, yet educational game design is sometimes more a matter of intuition rather than based on tested design patterns. A reliable, universal evaluation tool to assess educational games’ designs across genres and content areas is not available, though several researchers and designers are trying to identify core design patterns (Holopainen, 2003; Church, 1999) while others provide rubrics around very limited, design elements (CSUS, 2007; SDSU, 2009).

This presentation encourages educators, researchers and designers to evaluate games with a shared vocabulary and professional dialogue about clear criteria and standards by using a rubric created by researchers from the Games for Learning Institute (G4LI). G4LI is a consortium of university researchers, supported primarily by Microsoft Research, studying the educational use of digital games and investigating their socio-cultural, cognitive and emotional impact (http://www.seriousgamessource.com/item.php?story=20553). Part of this effort is to identify supportable design patterns for effective educational games that designers can draw on to assure high quality gameplay and educational value. A rubric with the dual purpose of providing guidance for educational game design as well as an evaluation tool for comparing educational games across play genres is one outcome of this work.

A rubric is several things: a guide for evaluating performance (Danielson, 1997); a descriptive scoring scheme developed to guide the analysis of a product or process (Brookhart, 1999); a written-down version of criteria, with all scoring described and defined (Arter & McTighe, 2001). While rubrics are commonly used in education, creating a rubric for evaluating game design has been challenging, as fixed criteria about what to assess in a game is not agreed upon. Thus, G4LI researchers surveyed the existing literature and identified 17 “design patterns” of effective games. The rubric uses these design patterns as criteria for game design and evaluation.

Along with the game design patterns, three central criteria are used in the rubric. These relate to internal game mechanics that affect gameplay and learning, and are simultaneously important and independent of one another (Danielson, 1997). Each of the 17 individual design patterns are evaluated on 5-point scale in each of the three criteria:

  1. Technical Implementation: The activity of programming and executing a design pattern into a working version of the game. Includes the seamless integration of design elements within game play.
  2. Educational Appropriateness: The ability of the game to address educational/curricular goals and the player(s) knowledge/ability relative to the educational content being addressed.
  3. Overall Integration with goals: The integration of the design pattern being considered with the other elements within the game, and within overall game play and educational goals.

The rubric has been developed and tested as a guidance-for-design tool in the national G4Li Game Design Challenge, already announced (http://g4li.org/archives/680). Authors will report the effectiveness of using the Game Design Patterns rubric, lessons learned from this experience and, by including a sample game evaluation, demonstrate how the rubric is used.

References

Arter, J. & McTighe, J. (2001). Scoring rubrics in the classroom: Using performance criteria for assessing and improving student performance. Thousand Oaks, CA: Corwin Press, Inc.

Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of pedagogy. ASHE-ERIC Higher Education Report (Vol. 27, No.1). Washington, DC: The George Washington University, Graduate School of Education and Human Development.

Church, D. (1999). Formal abstract design tools. Retrieved from www.gamasutra.com.

Danielson, C. (1997). A collection of performance tasks and rubrics: Middle School mathematics. Larchmont, NY: Eye on Education Inc.

Kreimeier, B. (2002). Game design patterns. GDC 2002 roundtable. Retrieved from http://www.oneArrow.org/game/pattern/

CSUS. (2007). Educational electronic games rubric. Retrieved from http://www.csus.edu/indiv/k/kaym/rubric/edgamesrubric.html.

SDSU. (2009). E-Game evaluation rubrics. Retrieved from http://edweb.sdsu.edu/courses/edtec670/e-game-rubric.html .