Ankle Rehabilitation, the Microsoft Kinect and Gamification

1

Ankle Rehabilitation, the Microsoft Kinect and Gamification

Catherina Lau

Paige Sinker

Travis Zimpfer

With

Aaron McRuer

Executive Summary

For roughly four months, three convergence students from the Missouri School of Journalism and an information technology research assistant worked on every aspect of a video game designed to assist people recovering from ankle injuries. Ideally, this game would come from the idea called “gamification,” the introduction of game elements to a non-game activity or setting to make the pursuance of the activity more desirable, engaging and fun.

In our case, we had a central problem that needed solving presented to us by Dr. Aaron Gray, an expert in rehabilitative medicine. Those undergoing rehabilitation for ankle injuries often fail to do the home exercises required for recovery out of boredom, a lack of discipline and a myriad of other reasons. We were tasked with making a game for Microsoft Kinect that made those exercises fun and interesting enough that injured athletes would do them and proceed with their recovery.

We approached this research project from all sides, executing multiple tasks from the aesthetic (concept art, plot, story, characters) to the commercial (market research, business strategies, finding our customers, management) to the technical (computer coding and working with the Kinect).

Other examples of gamification played a key role in helping us understand what to include content-wise, why those processes worked and how we could expand on those central principles to make a unique gaming experience that would attract an audience. We also looked at Kinect games to understand precisely how we could work within the boundaries of what is possible with this amazing technology.

Through much toil and tinkering, we have a small showcase that will act as a proof of concept for the university or the Missouri School of Journalism to spearhead a larger, more robust and well-funded game. We have the utmost confidence that this project could succeed at the professional level given the time and resources demanded of most games at such a playing field.

Starting the Project

Three students pursued this project from the beginning; Catherina (Catie) Lau, Paige Sinker and Travis Zimpfer. We first met in mid-January simply to share contact information, learn about each other and throw around early ideas for the game. Catie and Travis both turned out to be avid gamers and had an abundance of experience in playing games and understanding their storylines. Paige, on the other hand, lacked video game experience, but had figure skated when she was younger. She also suffered from injuries in the sport, including ankle injuries, which gave her unique, personal insight into the rehabilitation process.

We later met with Dr. Gray, a physical therapist and one of our faculty advisors for the project, early on in the creative process. He telegraphed to us the impetus for the project, namely his interest in “gamification,” the use of a game mindset in a non-game setting. He explained how successful certain smart phone applications, such as Zombies, Run!, had become based on the idea of gamification. He also provided us with a list of exercises that he gave to his patients that would give us blueprints for different motions and exercises we could use in the game.

Gray impressed upon us that he saw a true future in gamification and that he wanted a game that could compel his patients, mostly high school athletes, to do their home exercises and, in doing so, keep their healing process on track. He gave us absolute creative freedom to make that idea a reality.

Gamification: Understanding Our Core Concept

Our game would be designed to offer different types of rehabilitation exercises masked as gameplay. Certain actions in the game would benefit those who had suffered from ankle injuries and needed a way to repair them outside of physical therapy locations and in their own homes. Our game would act like an important tool in the recovery process.

As previously stated, our own introduction to gamification sprang from the mind of Dr. Gray and particularly his enthrallment with Zombies, Run!, a game application for Apple and Android devices. Zombies, Run! adds a gripping story of survival in a zombie-infested universe to the sometimes tedious task of running for exercise. You listen to a narrator inform you of picking up supplies and rescuing survivors through your ear buds or headphones. Those supplies and survivors could then be used to build up a base camp on your device.

While Zombies, Run! is most assuredly a game by any stretch of the imagination, it used storytelling, plot devices and a suspension of reality to encourage those who purchased it to pursue a real life goal – becoming more physically fit. Not only did it succeed at that, it also made quite a bit of money for its progenitors. On Feb. 27, 2012, it became the top grossing fitness application on the Apple App Store and to date, it has sold almost half a million units. Fitocracy, a website/application that tracks fitness growth, has followed a similar system. Depending on how rigorous and how long the duration of physical activity, the app rewards players with experience points like those found in role-playing games and offers achievements towards which “players” can strive. Last, but certainly not least, the Nintendo Wii’s Wii Fit and Wii Sports encouraged gamers to get off the couch using the Wii-mote and Wii Balance Board to exercise in a game setting. Those two games sold upwards of 100 million copies combined, the latter selling over 80 million copies on its own (making it the best selling game of all time).

However, gamification means more than just helping people get fit and active; it also helps the ailing, injured and sick. Stroke victims have used the Nintendo Wii virtually since its release to regain control of lost motor function.[1]Games may also help autistic children focus on learning goals oriented in a more game-like environment.[2]Other examples of gamification have been used in the business world to track employee satisfaction, incentivize consumers to buy certain products and receive feedback from those same consumers. We elected not to follow that line of thinking as much because we wanted to stick to games more closely related to our own enterprise.

Obviously, gamification faces few obstacles since various companies, industries and individuals have displayed the breadth and depth. Instilling a game mentality in consumers to help them achieve real-life goals creates monetary benefits for companies and physical and psychological benefits for consumers. While gamification has some critics, it has also helped people overcome obstacles by simply making them more enjoyable to overcome. By studying these examples, we grew a knowledge base that gave us keen insight into how to make our own game achievable. It also gave us confidence that not only would our product work, but that so long as we found the right niche and put in considerable effort, we could make a worthwhile, lasting game that could stand alongside some of those that we mentioned earlier

Microsoft Kinect’s Capabilities

Let’s examine the groundbreaking technology behind Microsoft Kinect. Originally called “Project Natal” during its development, the Kinect at first appeared like it was an answer to the Nintendo Wii’s innovative Wii-mote controller. However, it turned out to be a completely unique method of playing games that used the body of the player as a controller. It uses dual cameras and a microphone to map players in a 3D space, interpret gestures and facial patterns and recognize voices while cutting down on ambient noise.

We also conducted personal research on what exactly we could accomplish with the Kinect and whether or not it would best fit our needs. The current model of Kinect tracks twenty separate points on the body as illustrated below – four in each arm and leg, one on the head and three on the torso (Figure 1). It works extremely well when all of those points are distinct in Kinect’s view, namely when someone is standing up, facing the Kinect head on with out crossing any of their arms or legs or contorting themselves into uncommon positions. However, break any of those rules, and it fails to register motion nearly as well. We also had minor difficulties tracking smaller, more subtle movements but not to the extent that they would prove imperceptible. These hindrances limited us to choosing certain exercises, but other than those small problems, we found ways to make the technology work for us.

We haven’t walked on untread ground with this idea either. A USC study found in a comparison between the relatively low-cost Kinect and a more expensive, but more precise OptiTrack V100:R2 that the “Microsoft Kinect is a promising VR neurologicalrehabilitation tool for use in the clinic and home environment.”[3] In fact, the amount of research done in the last few years, since Kinect’s inception, has been mind boggling, and few scholarly articles and pieces of academia have discouraged the use of Kinect as a tool for rehabilitation.

Ankle Exercises and Our Own Research

The day we met with him, Dr. Gray also provided us with the same resources he would give to a patient about to go through ankle rehabilitation. Packets filled with helpful illustrations and diagrams showed us precisely how to perform these exercises, and how we would apply those exercises into the atmosphere of the game. Different exercises coincided with a different level of progress in the rehabilitation process. For example, certain, more advanced exercises would have to wait until after the patient had completed simpler, more basic ones. Before hopping up and down on a sprained ankle, one must simply have the ability to bend it first.

Many people are unfamiliar with what an ankle sprain actually entails. Dr. Gray provided us with informative packets on symptoms, treatments, the grades of a sprain and the stages/ levels of rehabilitation. The early symptoms of ankle sprains (credit to Dr. Gray) includemild aching to sudden pain, swelling, discoloration, inability to move ankle properly and pain in the ankle even when no weight is applied. The foot should not have full weight applied to it because this will only cause the injury to get worse. Rehabilitation needs to be activated if someone has these symptoms.

Some treatments include taking medicine to stop inflammation, ice, elevating the foot, wrapping with an ace bandage, using crutches to keep weight off the hurt ankle and strengthening rehab exercises. Some of these exercises can be very redundant but are necessary to strengthen the ankle and develop a full recovery.

There are 3 grades of an ankle sprain. All 3 have different severities that included damage to ligaments and joints in the ankle. Grade 1 being minimal damage and grade 3 being the most severe.

Along with the level of sprain, the level of rehab and stages of rehab are applied. These exercises should be performed 5 days a week. From what Dr. Gray has told us, many of his patients do not practice these exercises at home. We hope that our game will provide a fun way to do rehab exercises, therefore patients will recover from their injuries sooner.

One of the unique challenges of our project sprung from this seemingly simple task. We had to make these exercises fit into a single world and a single narrative on a linear trajectory from least complicated to most. It also had to stay flexible, quick, fun and humorous in order to maintain a user base that would continue playing.

Our group came up with a survey to gather some quantitative data for our project. We combined efforts with Dr. Gray and had him look over our survey, and we continued to make changes until it was approved by him and his team. Dr. Gray agreed to have his patients fill out the survey. We did not want to supply this survey to a random group of people. Instead, we kept our sample to people who were going through some sort of sports related physical therapy program with Dr. Gray. Therefore, our survey sample was small. He had to hand out the survey due to certain IRB rules especially with patients under than 18. Dr. Gray received 7 completed surveys.

Seventy-one percent of respondents said they played sports video games. This high correlation gave us positive feedback for making a sports related physical therapy video game. Some of sports the people who filled out the surveys played were soccer, volleyball, basketball, track and baseball. Another part of this survey that was shocking was when asked about how many ankle injuries people have has in their life to cause them to miss sports or see a doctor the range varied from 0-5.

We had both males and females in our survey and the age range was 20-30 years old. A problem we came across when reviewing the data is that our target audience for the game if younger (teenagers) than the people who were surveyed. We were hoping to get people in the range of 12-22, rather than 20-30. If we were to give this survey out again we would hope to target people in the younger age range that would fit our target audience.

One of the respondents answered, “some exercises are boring and tedious” when asked if exercises were done at home regularly. This is one of our main goals in our game. We wanted to create a fun atmosphere at home for patients to achieve the highest level of physical therapy. If people don’t practice their exercises at home, they won’t recover at a quick rate and can be more prone to another injury in the future.

Sprained

The plot ofSprained is one that we think will appeal to the younger demographic, but still be fun and accessible to older generations. In it, we tried to embrace the fact that many rehabilitation exercises can look rather silly and use it to our advantage. The following scenario entails a brief demo of the game we designed that would highlight a certain exercises from each level of rehabilitation.

The game will open with a short introductory video that will contextualize the story. The character walks up to a rental spaceship dealership with a number of less-than-stellar vehicles to choose from. He talks to a rental agent and makes his choice, which appears to be roughly a lime-green Gremlin with wings. Though it is obviously very close to falling apart, the character is ecstatic and takes off right away. He flies off to begin his intergalactic road trip.

This brings the player to the first stage. In it, the dashboard, steering wheel, and gas pedal is visible, complete with fuzzy dice and a dashboard hula girl. The player has been instructed by the GPS navigator, the Driver Emulating Space Map and Onboard Navigation Doohickey (DESMOND, for short), that in order to keep the vehicle from overheating, he must compress and release the gas pedal. This is the first exercise, resulting in pointing and flexing the foot.

Despite the player’s best efforts, the car still ends up breaking down. After crash landing on a nearby planet and the car all but disintegrating, DESMOND explains where they have landed. They are on a planet ruled by a race of pygmy samurai warrior elephants, whose maximum height is roughly that of an average five-year-old human. DESMOND warns that they are a very proud, very violent race that must first be appeased with a number of greeting rituals, perfectly executed. DESMOND walks the player through the rituals he will encounter. This is the second set of exercises. Matching the elephants, the player must balance on one leg, then swing the other leg forward and back, then side to side. This is done on both sides. As soon as the player knows what must be done, he leaves his broken ship and performs the greeting rituals with the elephants. If he does well, they will accept him as a friendly traveler. They will provide a few necessary parts for your vehicle, and ask that the player help them with a volcano problem that they have. They say that they cannot climb the insides of the volcano in order to turn it off via a large red button at the top, and say that if you are able to, they will fix your car while you are gone. However, if you fail the rituals, you are simply kicked into the next stage.

The final stage we have created is within the volcano. There are rocks of varying sizes that serve as stepping-stones within the river of lava. The player will hop from stone to stone using both feet to make his way up the mountain. Further up, the rocks will be smaller, necessitating the player to balance on one foot and hop from stone to stone. This is one of the most advanced rehabilitation exercises. The player will eventually make his way to the top and stop the volcano.