Stroke Recovery with Kinect

The box-and-block test in Stroke Recovery with Kinect evaluates a patient’s coordination, manual dexterity, and motor skills.Stroke Recovery with Kinect is an interactive rehabilitation system prototype that helps stroke patients improve their upper-limb motor functioning in the comfort of their own home. By using Microsoft Kinect technology, this prototype system recognizes and interprets the user’s gestures, assesses their rehabilitation progress, and adjusts the level of difficulty for subsequent therapy sessions.

Project goals

  • Develop cost-effective Kinect-based virtual reality system that helps in upper-body rehabilitation
  • Enable patients to perform therapy within the privacy of their own homes, where they may be more relaxed and likely to persevere
  • Provide doctors with data on patients’ performance to help measure progress and adjust treatment
  • Provide patients with a fun and enjoyable experience that a rehabilitative setting does not traditionally provide
  • Reduce expenses associated with office visits for rehabilitation, which burden healthcare systems and patients alike

How it works

The system's outer-space game tracks the patient's reflexes and reactions.Built by using the Microsoft Kinect for Windows software development kit (SDK), this prototype system uses the Kinect sensor’s three-dimensional camera to capture the movements of 48 skeletal points on the patient while he or she performs the therapy. Stroke Recovery with Kinect interprets the movement data supplied by the sensor, enabling the system to measure and evaluate the patient’s movements. This information helps the doctor asses the patient’s rehabilitation progress. The system uses the patient’s scores from previous sessions to adjust the level of difficulty for subsequent therapy sessions.

The system programs

Stroke Recovery with Kinect consists of three rehabilitative programs that the patient can perform wherever they can set up the Kinect sensor and a computer on which the system operates. The patient may be instructed to perform any or all of the programs below.

  • A virtual version of the classic box-and-block test (BBT) evaluates patients’ coordination, gross manual dexterity, and motor skills as they (virtually) attempt to pick up blocks one-by-one and put them into a box in a set amount of time. Similar to a computer game, Stroke Recovery with Kinect displays patients’ scores as soon as they finish a session, providing immediate reinforcement when scores improve from session to session.
  • A separate program displays a target pose on the monitor that the patient assumes. As the target moves, the patient attempts to duplicate the target’s position. The patient then receives a Fugl-Meyer Assessment score, based on his or her success.
  • An outer-space game enables patients to exercise their reflex and reaction abilities as they guide a spaceship through space while attempting to avoid oncoming asteroids. Stroke Recovery with Kinect tracks the stroke patient’s hand trajectory—relative to and in conjunction with the movement of the elbow and/or shoulder.

Future features

Long-term plans for the project include the following developments.

  • Integrate social networking into the system so that stroke patients can connect with one another and participate jointly in the rehabilitative programs, building a sense of camaraderie that could offer emotional and psychological support and motivation. Within the community, patients will have the opportunity to communicate about their condition and receive encouragement as they advance toward recovery.
  • Update the system to enable doctors to monitor the patient’s rehabilitation from the hospital or their office, and to communicate with the patient regarding their treatment and progress.
  • As the system becomes more widely used and we acquire more data, we anticipate using Microsoft technology such as Windows Azure for big data analysis, enabling us to incorporate machine learning into the Stroke Recovery with Kinect system.
Eric Chang

Miran Lee

A project sponsored by

Microsoft Research and Seoul National University, with funding from the Korean Ministry of Science, ICT, and Future Planning under the IT/Software Creative Research Program, and supervised by the National IT Industry Promotion Agency