In traditional sport and entertainment settings, players with mobility disabilities typically do not have opportunities to engage in physical play with their peers without disabilities and vice versa due to physical and social barriers. Addressing these barriers is critical for reducing the risk of developing serious health problems and secondary health conditions, such as pain, fatigue, depression and obesity, as well as for increasing a sense of empowerment, normalcy, and acquisition of social capital.

The key idea behind the AiR Play project is to remove these barriers to play by combining some of the elements and well-known benefits of adaptive sports (e.g., power chair soccer) and sports in general with the whole-body interaction modalities of exergames (e.g., Nintendo Wii or Xbox Kinect) to create an inclusive, projected augmented reality (AiR) game environment that is accessible and enjoyable for people of all abilities.

The AiR Play project builds on prior work on an inclusive play and exercise system, iGYM, developed by a group of faculty and students from four different schools including the Stamps School of Art & Design, EESC, Kinesiology, and School of Information (see U-M and AP news stories). This system is different from existing spatial or projected augmented reality systems and exergame platforms as it levels the playing field in two ways: first, it projects a circle on the floor that visualizes each player’s peripersonal space boundaries with which they can manipulate a virtual target on the floor by body movement, limb extension or pressing a kick button to simulate limb extension; second, it provides adaptable game mechanics using physics simulation to create, for example, a realistic ball game environment, which allows balancing players’ individual differences in response time or mobility by controlling game mechanics and customizing them for each player. Taken as a whole, these two design features allow novel, inclusive co-located play and exercise opportunities that can be provided, for example, with projected augmented reality in the form of interactive floors using ceiling mounted video projectors. Further, similar play opportunities can be provided outdoors with vector-based laser projection technology as well as in the future with further improved head mounted display systems (the latter are currently impractical for co-located play scenarios similar to adaptive sports or sport activities in general due to their limited field of view and form factor).

The student team will play an integral role in the further design and development of various aspects of this system and approach including game development, augmented reality techniques, sound design, user experience design, interaction design, and computer vision. The team will gain hands-on experience in each of those areas and help to shape a new genre of interactive games and systems. The immediate goals of this team will be to:

  • make the existing iGYM system fully operational for end user; implement and refine web-based GUI
  • improve current game design and sound effects; develop 1-2 new games
  • improve player detection and enable player ID tracking (for players using kick-buttons)
  • design and build wireless kick-buttons
  • conduct customer discovery and lead project promotion

The ultimate goal of this project is to build a customizable kit and augmented reality game platform that can easily be installed in different environments.

Meeting Details
Thursdays 4:30-6:30pm (or Fridays 9:30-11:30am)
Modality: In-person (interested in the project but unable to be on campus? Contact us to inquire!)

Students apply to a specific role on team as follows:

Game Development & Sound Design (3 Students)

Preferred Skills: Unity

Likely Majors/Minors: ARCH, ARTDES, CE, CS, EE, PAT

Computer Vision & Software Engineering (3 Students)

Preferred Skills: ROS2, Realtime systems, OpenCV (computer vision), C++, signal processing

Likely Majors/Minors: CE, CS, EE, ROB

UX/UI for Operator Interface & Web Design (2 Students)

Preferred Skills: HTML, CSS, and JavaScript

Likely Majors/Minors: ARTDES, CE, CS, EE, ROB, SI

Interaction & Product Design (2 Students)

Preferred Skills: 3D modeling software such as Rhino or SolidWorks and experience with digital fabrication and prototyping of interactive systems (wireless kick-button).

Likely Majors/Minors: ARCH, ARTDES, CE, CS, EE, ME 

Startup & Entrepreneurship (2 Students)

Preferred Skills: Customer discovery, SWOT analysis, agile development, product planning, project management, marketing

Likely Majors/Minors: BBA, COMM, ENTR

Faculty Project Leads

Roland Graf is a media artist, design researcher, and inventor. He collaborates across disciplines to design platforms, games, and interactive systems that reframe the body and interactivity in the built environment. Graf’s doctoral research focused on tracing a convergence of artistic and ludic (i.e., playful) engineering approaches at the intersection of interactive art and human-computer interaction. His current research explores the roles of play and playful exploration of emerging technologies in prototyping and building more accessible and inclusive futures.

Graf initiated and directs several interdisciplinary research collaborations at the University of Michigan including: Daylight Media Lab, a collaboration with material scientists and engineers exploring sunlight as a medium for interactive outdoor media systems; Internet of Shoes, a sensory networking platform for street-level interactions in collaboration with engineers of embedded wireless technologies. His most recent research with computer engineers, iGYM, is an inclusive augmented reality system that enables people with different abilities to play and exercise together. Funded by the U-M Exercise & Sport Science Initiative, it received the Best Paper Award in the proceedings of CHI Play 2019 and was featured in more than 300 media outlets including The New York Times, The Washington Post, ABC News, Houston Chronicle, and USA Today.

Graf is also a co-founder of the artist collective Assocreation, whose oeuvre includes a wide range of interactive installations, happenings, and urban interventions. Assocreation’s work has been shown internationally in galleries, museums, and at art festivals, conferences, and computer/world expos, including the Bienal de Valencia, The Vienna Künstlerhaus, Museum of Modern Art in Warsaw, CENTRALE for contemporary art in Bruxelles, Ars Electronica Festival in Linz, CeBIT in Hannover, EXPO 2017 in Astana, TEI’15 in Stanford, as well as in the streets of, among others, New York, Detroit, Sao Paulo, Istanbul, Mumbai, Hong Kong, and Kathmandu.

Throughout his career, Graf has often worked simultaneously in different creative domains including art, architecture, and product design. His individual and collaborative work in these domains has received awards, patents, and international recognition including: the Excellence Award at the Japan Media Arts Festival for the street video game Solar Pink Pong, the Prix Ars Electronica Distinction in the Category of Interactive Art for the telematic sidewalk Bump, Winner of the National Play Everywhere Design Challenge with the urban intervention Brightmoor Runway, Feldmark licensed to the German retailer Magazin, Slim Jim exhibited by Fredericia at the Milan Design Week.

Since relocating to the USA in 2011, Graf has also collaborated with communities at the heart of Michigan’s postindustrial society on public art and design/build projects such as “Camp Take Notice” in Ann Arbor, and “Black Men for Social Change” in Flint. Before joining the Stamps School of Art & Design faculty, Graf was an Assistant Professor in the Department of Spatial and Sustainable Design at the Vienna University of Technology.

Michael Nebeling is an Assistant Professor in the School of Information, where he leads the Information Interaction Lab. His lab investigates new methods, tools and technologies that enable users to interact with information in more natural and powerful ways, and also make it easier for designers to create more usable and effective user interfaces.

Previously, he was a postdoctoral researcher and lecturer at the HCI Institute at Carnegie Mellon University and the Department of Computer Science at ETH Zurich, where he also obtained his PhD.

More on his personal web site

Hun-Seok Kim is an assistant professor at the University of Michigan, Ann Arbor. Kim received his B.S. degree from the Seoul National University (South Korea) in 2001, and M.S. & Ph.D. degrees from the University of California, Los Angeles (UCLA), all in Electrical Engineering. His research focuses on system analysis, novel algorithms, and efficient VLSI architectures for low-power/high-performance wireless communication, signal processing, computer vision, and machine learning systems. Before joining the University of Michigan, Kim worked as a technical staff member at Texas Instruments (2010 – 2014). He is serving as an associate editor of IEEE Transactions on Mobile Computing, IEEE Transactions on Green Communications & Networking, and IEEE Solid State Circuits Letters. Kim is a recipient of the 2018 Defense Advanced Research Projects Agency (DARPA) Young Faculty Award (YFA) and the National Science Foundation (NSF) Faculty Early Career Development (CAREER) Award 2019.

Students: 12

Likely Majors/Minors: ARCH, ARTDES, BBA, CE, COMM, CS, EE, ENTR, ME, PAT, ROB, SI

Meeting Details: Thursdays, 4:30-6:30pm or Fridays, 9:30-11:30am

Application: Consider including a link to your portfolio or other websites in the personal statement portion of your application to share work you would like considered as part of your submission.

Summer Opportunity: Summer research fellowships may be available for qualifying students.

Citizenship Requirements: This project is open to all students on campus.

IP/NDA: Students who successfully match to this project team will be required to sign an Intellectual Property (IP) Agreement prior to participation.

Course Substitutions: CoE Honors