Research ArticleNEUROTECHNOLOGY

Illusory movement perception improves motor control for prosthetic hands

See allHide authors and affiliations

Science Translational Medicine  14 Mar 2018:
Vol. 10, Issue 432, eaao6990
DOI: 10.1126/scitranslmed.aao6990
  • Fig. 1 Movement percepts for all participants.

    Schematics representing perceived movements induced by 90-Hz vibration to the reinnervated residual muscles in six amputee participants (Par 1 to Par 6) reported using the intact hand. Participant details are shown in fig. S1. From a start position (Start, gray outlines), participants perceived movement in the direction and relative magnitude indicated by orange arrows to an end position (Finish, black outlines). Digits are specified as follows: D1, thumb; D2, index finger; D3, middle finger; D4, ring finger; D5, little finger.

  • Fig. 2 Active, passive, and intrinsic movement percepts with measures of similarity across days and percept types.

    (A) Kinematic trajectories for the cylinder grip percepts for Par 1, Par 2, and Par 5 with the start (open) and end (closed) positions of the percepts demonstrated using the virtual hand at the top of each column. Graphs of digit average joint angles [n = 30 trials (40 for Par 2 passive, first day)] are ranked in descending order according to average change in angle across all percepts and all participants. Individual plots show the joint with the greatest change in angle for that digit. PIP, proximal interphalangeal; MCP, metacarpophalangeal; Ip, interphalangeal; Op., opposition. Plots include active (teal), passive (gray), and intrinsic (magenta) percepts measured on the first experimental day (solid line), last experimental day (dashed line), and after the first speed game (ASG) (ASG1, dotted line; see fig. S8, B and C). (B) Aggregate measures of similarity in grip dynamics between each percept pair for each individual participant quantify percept stability across days and similarity between active, passive, and intrinsic conditions. The darker the shade, the greater the similarity between average percept joint trajectories [root mean square differences (RMSDs) averaged across the joints in each digit with the greatest change in angle (n = 5)]. (C) Overall movement similarity by participant for all percepts across all days to quantify global percept stability. The farther the marker to the right, the greater the average correlation [Pearson correlation coefficients averaged across all of an individual’s percepts for the joints in each digit with the greatest change in angle (top; n = 5 joints) and all joint movements (bottom; n = 22 movements)].

  • Fig. 3 Performance in functional tasks with and without vibration-induced kinesthetic illusory feedback.

    (A) Three participants’ (Par 1, Par 2, and Par 5) ability to accurately reach proportioned intervals in a grip conformation task (25, 50, 75, and 100% hand closed) while receiving no (0 Hz; orange), 20-Hz (purple), and 90-Hz (teal) vibratory feedback. Actual intervals between targets (colored rectangles) are compared to the ideal intervals fit to each participant’s actual performance times (black open rectangles). The actual times to target position are shown as circles with error bars indicating 95% confidence intervals (CIs; n = 20), which specify the height of the colored rectangles. The ideal change in time to target position between percent close positions (black open rectangles) is specified by the linear regression with intercept set at zero (black dotted line). Alignment between the black open rectangles and the colored rectangles is an indication of the participant’s ability to reach the proportional degrees of closure. (B) Line graph showing degree of alignment with ideal proportional performance in the grip conformation task shown in (A) for amputee participants (Par 1, Par 2, and Par 5) and in an analogous task for an able-bodied cohort (AB Avg, n = 5; fig. S4B). The black dashed line indicates the average performance of able-bodied (AB Avg) participants ± 2 SDs (gray shaded area). (C) Bar graph showing average adaptation rate to self-generated error for Par 1 and Par 2 in different feedback conditions (vis. + kin., vision and kinesthesia; kin. only, kinesthesia only; vis. only, vision only; sham only, 20-Hz vibration). Error bars represent 95% CIs (n = 75 to 95 trials). (D) Bar graph showing the SD of the overall system noise (see Materials and Methods for details) for Par 1, Par 2, and Par 5, for different feedback combinations. (E) Graph of cumulative EMG control signal trajectories for Par 5 using an agonist-antagonist muscle pair (biceps, hand close; triceps, hand open). Average cumulative control signal trajectories [n = 4 trials, time the participant provided a close signal (negative) plus the open signal (positive)] for each feedback condition [90-Hz (teal line), 20-Hz (purple line), or no vibration (orange line)] compared to the target trajectory (black line).

  • Fig. 4 Measures of agency and embodiment for combinations of intent, visual feedback, and illusory movement sensation.

    (A) Average agency and embodiment questionnaire (fig. S7D) responses across Par 1, Par 2, and Par 5 under different conditions (baseline, illusory percept matches the hand visualization; no vibration, hand visualization closes without illusory percept; too fast, hand visualization closes faster than the illusory percept; too slow, hand visualization closes slower than the illusory percept; onset delay, hand visualization closes 1 s later than the illusory percept; opposite movement, illusory percept closes while the hand visualization opens; passive, experimenter controlled the hand closing visualization and illusory percept; fig. S7, A and C). Error bars represent SD. The dagger indicates a significant main effect (P < 0.001) for question type (agency/embodiment versus control) from full-factorial linear mixed models (fixed effects: condition and question type). The asterisk indicates significant Bonferroni-corrected post hoc t tests (P < 0.05) between pairs of conditions within a question type. (B) Average agency responses (n = 16) compared to average estimated intervals relative to the baseline condition (n = 3 intervals, 20 trials each) both averaged by condition across Par 1, Par 2, and Par 5. The horizontal dotted line denotes no difference in estimated interval from the baseline condition, and the vertical dashed line indicates the +1 cutoff for an experience of agency (see Fig. 4A). Error bars represent estimates of average SD calculated as the square root of the average variance within a condition averaged across participants.

  • Fig. 5 Application of kinesthetic illusory feedback within a bidirectional neural-machine interface.

    (A) Schematic representation of the movement feedback paired to a real-time functional prosthetic hand clinically fitted to the participant with illusory feedback locked to their volitional control, which was used to explore clinical feasibility. Feedback pathways are represented in blue (VCLM, voice coil linear motor). Prosthesis control pathways are represented in red (participant control). Participants matched the perceived sensation with their intact hand, and prosthetic hand closing speed was timed to the demonstrated perceptual illusion. (B) Graph showing the average start/stop times of the control signal (n = 32; EMG-activated prosthetic hand closing, blue) and the average start/stop times of the concurrently demonstrated percept movement (n = 32; matching hand, red) superimposed over the ideal start and thumb–index finger contact times of the physical prosthetic hand under continuous drive (black crosshairs, radius = 250 ms). The 5-s progression of movement from fully open to thumb–index finger contact for the physical prosthetic hand is our approximation of the participant’s demonstrated movement of the illusory percept (gray dashed lines). All events are plotted along time-linked axes. The raw matching hand movement start times (solid circles) and stop times (x’s) are colored according to relative position within the experimental timeline (first, blue; last, red). Gold stars represent the intersection of the average movement start and stop points for the EMG-control and the demonstrated movement. (C) Graph showing average percept speeds (n = 30; error bars represent SD) in Par 1, Par 2, and Par 5 measured before (Pre), after one, two, three, four, and five conditioning games (ASG1 to ASG5) designed to increase percept speed (fig. S8, B and C), and after a washout period (Post). The gray area represents the range of hand close speed at the lowest speed setting in a common commercially available prosthetic hand (OttoBock Speed 0 range).

Supplementary Materials

  • www.sciencetranslationalmedicine.org/cgi/content/full/10/432/eaao6990/DC1

    Materials and Methods

    Fig. S1. Overall participant demographic, surgical and experimental details with comparisons of cutaneous touch percepts in the skin with underlying movement percepts in the muscle.

    Fig. S2. Perceived magnitude of the kinesthetic illusion.

    Fig. S3. Virtual or prosthetic hand movement and linked vibration-induced kinesthetic percept, controlled either actively (by the amputee) or passively (by the experimenter).

    Fig. S4. Grip aperture experimental setup and percent close grip values for able-bodied cohort on a single degree of freedom task.

    Fig. S5. Falling block experimental setup and example analyses used in adaptation and just-noticeable difference experiments.

    Fig. S6. Clinical implementation with two-site (agonist-antagonist) kinesthetic feedback.

    Fig. S7. Setup for intentional binding experiments, conditions tested, and questionnaire statements.

    Fig. S8. Matching complex hand percepts to the dexterous robotic prosthetic hand.

    Fig. S9. Muscle activity (EMG) relative to baseline when virtual hand movement and linked vibration-induced kinesthetic percept were controlled passively (by the experimenter) or actively (by the amputee).

    Movie S1. Demonstration of active and passive percepts.

    Movie S2. Clinical implementation of two-site agonist-antagonist kinesthetic feedback.

    Movie S3. Volitionally controlled, clinically fit robotic hand matching perceived movement.

    Movie S4. Speed game demonstration.

    Reference (72)

  • Supplementary Material for:

    Illusory movement perception improves motor control for prosthetic hands

    Paul D. Marasco,* Jacqueline S. Hebert, Jon W. Sensinger, Courtney E. Shell, Jonathon S. Schofield, Zachary C. Thumser, Raviraj Nataraj, Dylan T. Beckler, Michael R. Dawson, Dan H. Blustein, Satinder Gill, Brett D. Mensh, Rafael Granja-Vazquez, Madeline D. Newcomb, Jason P. Carey, Beth M. Orzell

    *Corresponding author. Email: marascp2{at}ccf.org

    Published 14 March 2018, Sci. Transl. Med. 10, eaao6990 (2018)
    DOI: 10.1126/scitranslmed.aao6990

    This PDF file includes:

    • Materials and Methods
    • Fig. S1. Overall participant demographic, surgical and experimental details with comparisons of cutaneous touch percepts in the skin with underlying movement percepts in the muscle.
    • Fig. S2. Perceived magnitude of the kinesthetic illusion.
    • Fig. S3. Virtual or prosthetic hand movement and linked vibration-induced kinesthetic percept, controlled either actively (by the amputee) or passively (by the experimenter).
    • Fig. S4. Grip aperture experimental setup and percent close grip values for able-bodied cohort on a single degree of freedom task.
    • Fig. S5. Falling block experimental setup and example analyses used in adaptation and just-noticeable difference experiments.
    • Fig. S6. Clinical implementation with two-site (agonist-antagonist) kinesthetic feedback.
    • Fig. S7. Setup for intentional binding experiments, conditions tested, and questionnaire statements.
    • Fig. S8. Matching complex hand percepts to the dexterous robotic prosthetic hand.
    • Fig. S9. Muscle activity (EMG) relative to baseline when virtual hand movement and linked vibration-induced kinesthetic percept were controlled passively (by the experimenter) or actively (by the amputee).
    • Legends for movies S1 to S4
    • Reference (72)

    [Download PDF]

    Other Supplementary Material for this manuscript includes the following:

    • Movie S1 (.mp4 format). Demonstration of active and passive percepts.
    • Movie S2 (.mp4 format). Clinical implementation of two-site agonist-antagonist kinesthetic feedback.
    • Movie S3 (.mp4 format). Volitionally controlled, clinically fit robotic hand matching perceived movement.
    • Movie S4 (.mp4 format). Speed game demonstration.

Navigate This Article