Conceptually, the Gesture Therapy platform is composed of 5 modules:

  • Hardware: Encompasses the hardware platform incorporating a PC, a webcam and the handgrip.
  • Visual Tracking The software bit responsible for tracking the handgrip, proxy of arm movements. The basic algorithm based on particle filters.
  • Simulated Environment Responsible for presenting the game and interacting with the user
  • Trunk Compensation detector Permits detection of the trunk compensation movements by tracking the user head. After trunk compensation movement occur, the system may provide an alarm or block the game.
  • Adaptation System Capable of adjusting the 3D space in which the exercise occurs. It allows to adjust the difficulty of the task and give feedback to the user.
Gesture Therapy Modules

Gesture Therapy Modules

System architecture


Gesture Therapy Architecture

Gesture Therapy Platform Architecture

User profiling capabilities

A postgreSQL supports the user profiling capabilities. The database permits tracking of user activity. From the patient point of view it tracks times spent on the therapy, and in each particular game individually. It further keeps track of the traces of the user avatar at all times and records timestamps and performances associated.
Therapies can be delivered following game selection by the therapist or a pre-dictated plan. The therapist can add clinical annotations to every session played by the patient. The Entity-Relation diagram of the database is shown in the Figure.




Benefits of virtual reality-based motor rehabilitation therapies [AugustK2005, LaverKE2011, HoldenMK2005]

  • Validity: Comply with the 3 key concepts in rehabilitation:
    • Repetition and practice
    • Feedback about performance
      • Real time feedback and “knowledge of results” for the patient and the clinician
      • Different types of feedback and in different degrees: virtual teacher, learning by imitation, rewards, augmented feedback, errorless learning, etc)
    • Motivation to endure practice
  • Initial evidence from other studies suggests that the patient can learn motor skills in virtual environments and transfer those skills to the real world
    • …perhaps even better than doing so directly in the real world (knowledge may be acquired at a more abstract level)!!
  • Adaptable to the user need and progress
    • Progress, dose, frequency, challenge, variabilty, difficulty, etc…
  • Customizable to the therapy requirements
    • Different affections: stroke, cerebral palsy, brain injury, alzheimer, etc…
    • Different target groups; ages, genre, specialized (e.g. military, sports, etc)
  • Low clinical supervision demands
    • Apt for home use
  • Low cost
    • Suitable for low and middle income countries
  • Others:
    • Quick development
    • Less dangerous in certain applications (e.g. coffee making training)
    • Flexible schedule (available 24/7)
    • Distractors may be aliminated or added on demand
    • Allows for complex task decomposition into simpler tasks
    • Wide tersting capabilities (methodologies, feedback forms, timings, etc).

Leave a Reply