Research ArticleENGINEERING

Supervised autonomous robotic soft tissue surgery

See allHide authors and affiliations

Science Translational Medicine  04 May 2016:
Vol. 8, Issue 337, pp. 337ra64
DOI: 10.1126/scitranslmed.aad9398

Hands-free

The operating room may someday be run by robots, with surgeons overseeing their moves. Shademan et al. designed a “Smart Tissue Autonomous Robot,” or STAR, which consists of tools for suturing as well as fluorescent and 3D imaging, force sensing, and submillimeter positioning. With all of these components, the authors were able to use STAR for soft tissue surgery—a difficult task for a robot given tissue deformity and mobility. Surgeons tested STAR against manual surgery, laparoscopy, and robot-assisted surgery for porcine intestinal anastomosis, and found that the supervised autonomous surgery offered by the STAR system was superior.

Abstract

The current paradigm of robot-assisted surgeries (RASs) depends entirely on an individual surgeon’s manual capability. Autonomous robotic surgery—removing the surgeon’s hands—promises enhanced efficacy, safety, and improved access to optimized surgical techniques. Surgeries involving soft tissue have not been performed autonomously because of technological limitations, including lack of vision systems that can distinguish and track the target tissues in dynamic surgical environments and lack of intelligent algorithms that can execute complex surgical tasks. We demonstrate in vivo supervised autonomous soft tissue surgery in an open surgical setting, enabled by a plenoptic three-dimensional and near-infrared fluorescent (NIRF) imaging system and an autonomous suturing algorithm. Inspired by the best human surgical practices, a computer program generates a plan to complete complex surgical tasks on deformable soft tissue, such as suturing and intestinal anastomosis. We compared metrics of anastomosis—including the consistency of suturing informed by the average suture spacing, the pressure at which the anastomosis leaked, the number of mistakes that required removing the needle from the tissue, completion time, and lumen reduction in intestinal anastomoses—between our supervised autonomous system, manual laparoscopic surgery, and clinically used RAS approaches. Despite dynamic scene changes and tissue movement during surgery, we demonstrate that the outcome of supervised autonomous procedures is superior to surgery performed by expert surgeons and RAS techniques in ex vivo porcine tissues and in living pigs. These results demonstrate the potential for autonomous robots to improve the efficacy, consistency, functional outcome, and accessibility of surgical techniques.

INTRODUCTION

Despite increased adoption of robot-assisted surgery (RAS), the execution of surgical tasks on soft tissue remains entirely manual under a human-controlled paradigm (15). Functional outcomes, including complication rates, have remained highly variable owing to human factors, such as a surgeon’s hand-eye coordination and experience. With more than 44.5 million soft tissue surgeries in the United States each year (6), autonomous soft tissue surgery promises substantial benefits through improved safety from reduction of human errors, increased efficiency due to procedure time reduction, and potential access to optimal surgical techniques and consistent outcomes independent of surgeon training, condition, or experience. Previous work on automating soft tissue surgeries includes demonstration of knot tying, needle insertion, and execution of predefined motions in laboratory settings, but a comprehensive preclinical evaluation of those systems in human-like surgical scenarios has not been performed (79).

According to the International Organization for Standardization, autonomy requires “an ability to perform intended tasks based on current state and sensing without human intervention” (ISO 8373:2012; www.iso.org). Supervisory functions for autonomous systems include planning, teaching, monitoring, and intervening (10). Supervised autonomy based on preoperative model acquisition of static rigid tissue anatomy and computer-assisted surgical planning has been applied to improve surgical outcomes in people (1114). Because of the fidelity between the preoperative model and the anatomy during surgery, precise execution of planned surgical tasks is possible (15). In contrast, unpredictable, elastic, and plastic changes in soft tissues pose prohibitive challenges when adapting surgical plans made before the operation, for instance, during linear vaginal cuff closure after hysterectomy or end-to-end anastomosis for reconstructive bowel surgery. Unlike rigid tissue surgery, autonomous decision support and execution of surgical tasks in soft tissue must constantly adjust to unpredictable scene changes, including nonrigid deformations as a result of cutting, suturing, or cauterizing.

To determine whether a complex in vivo surgical task, currently done only by humans, could be performed by a supervised autonomous robot, we developed a new Smart Tissue Autonomous Robot (STAR). STAR consists of a bedside lightweight robot arm extended with a U.S. Food and Drug Administration (FDA)–approved articulated laparoscopic suturing tool for a combined eight–degree of freedom (DOF) robot. An early prototype of the STAR system has been used previously for planar suturing tasks on phantom tissue models (16) with submillimeter planar accuracy (17). The new STAR is equipped with smart imaging technologies including plenoptic three-dimensional (3D) surface reconstruction with an average reconstruction accuracy of 1.14 mm on a checkerboard-patterned surface (18) and a 3D visual tracking system using custom near-infrared fluorescent (NIRF) imaging (19). Plenoptic imaging computes a 3D point for each pixel in an image through the use of a microlens array focused onto an image sensor and a software that triangulates the position of features seen in more than one microlens image. The combination of NIRF technology and 3D quantitative plenoptic imaging obviates the problems of occlusion and recognition of tissue targets by observing luminescent NIRF markers.

Here, we demonstrate supervised autonomous RAS in various soft tissue surgical tasks ranging in complexity but all with clinical relevance: ex vivo linear suturing of a longitudinal cut along a length of suspended intestine, ex vivo end-to-end anastomosis, and in vivo end-to-end anastomosis of porcine small intestine. We compared the ex vivo suturing metrics of STAR to those of experienced surgeons with at least 7 years of relevant training, using standard clinical surgical techniques. The in vivo experiment included a survival study to show preclinical feasibility and functional outcome.

RESULTS

Detecting and tracking soft tissue deformations

The STAR system integrates NIRF and 3D plenoptic vision, force sensing, submillimeter positioning, and actuated surgical tools (Fig. 1A). An enhanced view of the suturing tool and setup for the ex vivo linear suturing task are seen in Fig. 1B. STAR tracks custom NIRF markers (Fig. 1C), which were used as reference points to plan suture locations. Once placed by syringe, the NIRF markers remained stable in the surgical field, were robust to tissue motion and occlusion, and could be left in the surgical site or removed by peeling at the end of the procedure. STAR also constructs and updates suture plans (Fig. 1D), combining knots and running sutures with interpolated suture locations.

Fig. 1. Automating soft tissue surgery.

(A) The STAR system integrated NIRF and 3D plenoptic vision, force sensing, submillimeter positioning, and actuated surgical tools. (B) Surgical site detail during linear suturing task showing a longitudinally cut porcine intestine suspended by five stay sutures. (C) The tissue was first marked by syringe with biocompatible NIRF markers to track motions through blood and tissue occlusions. A 3D surface point cloud was then acquired with the plenoptic camera. The NIRF markers (now white) were projected onto the point cloud after offline registration of the 2D NIR image and the 3D point cloud data. The suture plan was computed as a sequence of coordinates and suture types (yellow knots and blue running sutures) to achieve leak-free anastomosis by overlapping stitch compression zones. (D) Point clouds of initial (blue) and deformed (red) tissue. STAR performance was then evaluated under random ex vivo deformations. Three deformations were induced on NIRF markers 2, 3, and 4 with an additional bowel rotation at the second deformation. m, marker. (E) Representative average marker deformation results from (D). Data are averages ± SD (n = 5 trials). (Additional combinations of induced deformations are in fig. S2.) (F) The 3D point cloud during end-to-end anastomosis before (blue) and after (red) deformations. The tissue was tracked by the combined NIRF and 3D plenoptic vision system. (G) The theory behind spacing of running suture is adapted from (21). The spacing was informed by the tissue thickness to ensure that the suture spacing S was smaller than bite size H. The zones of compression for each stitch, illustrated in pink, overlapped for leak-free anastomosis. A suture plan was initially created by interpolating between the tracked positions of NIRF markers and updated as the tissue deformed. On the right, the suture plan is overlaid on the bowel point cloud. Points outlined in green are knot locations. The plan for end-to-end anastomosis used two sutures. First, a running suture (red) was applied on the back wall (from right to left) after tying the first knot (green). At the left corner, the needle pierced the tissue from inside, which was later used to tie the final knot. Second, another running suture (blue) was applied on the back wall (left to right) after tying the second knot. Similarly, at the corner, the tissue was pierced from inside to outside. (H) Once the right corner was completed, the tissue was flipped manually, such that the edges of the front walls were close to each other. The suture plan calculated the running suture positions (blue) to continue from the right corner with the same suture.

We first assessed the range of possible soft tissue deformations during the linear suturing task in realistic surgical environments. The typical tension force of 2 N in intestinal anastomosis (20) and the calculated stiffness of the cadaveric porcine bowel tissue (Young’s modulus, 1.83 MPa) (fig. S1) were together used to induce tissue deformations via the stay sutures (Fig. 1B). Five different sequences of deformations were repeated five times (fig. S2). A range of motion between 0 and 6.5 mm was detected by STAR as typical of soft tissue deformations. We confirmed that the maximum induced motion in the ex vivo setup was within the typical range of soft tissue deformation in anastomosis of pig intestinal tissues (fig. S2). However, motions propagated through the tissue without a predictable pattern (Fig. 1, D and E, and fig. S2), making it difficult to model tissue deformation mathematically. Therefore, we designed our robotic system to detect arbitrary motions during a procedure and adjust the surgical plan autonomously. This was accomplished by observing and tracking NIRF markers to continuously interpolate the planned suture locations after each suture was placed (movie S1).

The current standards for ideal suturing require evenly spaced sutures with small gaps between consecutive sutures to prevent leaks; yet, gaps should be large enough to allow blood flow for healing. Therefore, we evaluated the consistency of suturing informed by the average suture spacing, the pressure at which the anastomosis leaked, the number of mistakes in the plan that required removing the needle from the tissue, completion time, and, in the case of the end-to-end anastomoses, lumen reduction. The linear suturing task consisted of closing a longitudinal cut along a piece of porcine small intestine ex vivo using hand-sewn suturing (OPEN), laparoscopy (LAP), RAS with da Vinci Surgical System (Intuitive Surgical), and our STAR platform. Deformations were artificially induced to mimic realistic surgical environments.

Leak-free suturing has been characterized by taking into account tissue thickness T, bite depth H, and spacing between consecutive sutures S (21). Leak-free suturing can be achieved when S < H (22) (Fig. 1G). For example, with at least 25% overlap (S < H/1.25), lumen diameter D, and lumen circumference C = πD, the geometrically optimal number of sutures N was calculated as N = C/S = ceil(1.25C/H) = ceil(1.25πD/3T). Considering intestinal tissue with thickness T = 1.3 mm and diameter D = 15 mm, the number of sutures was N = 16, with suture spacing S = C/N = 2.95 mm. The average (±SD) suture spacing across all modalities in our study was 3.01 mm (±2.42) (n = 684), which was in line with the theoretical optimal spacing of 2.95 mm. The average suture spacing for STAR was significantly less than LAP but similar to OPEN and RAS techniques (Fig. 2A). The variance for suture spacing was the smallest for STAR, indicating a greater consistency for semiautonomous suture placement compared to other techniques (table S1). Moreover, leak pressure reflects the functional quality of suturing. The linear closure from STAR was able to withstand a higher average leak pressure than all other techniques (Fig. 2B).

Fig. 2. Ex vivo linear suturing under deformations.

The experiment consisted of closing a longitudinal cut along pig intestine, whereas the tissue was deformed by pulling on stay sutures. Five samples were tested per technique (OPEN, LAP, RAS, and STAR). (A) Suture spacing. Central mark is the median; box edges are the 25th and 75th percentiles, error bars are the range excluding outliers, and red dots are outliers. The whiskers represent the range not including outliers. There is a different N number for each boxplot because each surgeon used a different number of sutures [OPEN (n = 174), LAP (n = 128), RAS (n = 176), and STAR (n = 206)]. These data are presented numerically in table S2, including the SDs. P values determined by ANOVA with post hoc Games-Howell. (B and C) Leak pressures and number of mistakes (repositioned stitches or robot reboot). Data are from individual tissue samples (n = 5) with averages marked by a horizontal line. P values determined by independent samples t test. (D) Completion times separated into knot-tying and suturing, and other time was spent restaging or changing sutures. Data are averages (n = 5). P values determined by independent samples t test.

Erroneous needle placements that require repositioning increase collateral tissue damage and potential anastomotic leak and should be avoided. Low numbers of repositioned needles in the STAR and OPEN approaches (<2) indicate confidence in the robotic hand-eye coordination and the suture plan (Fig. 2C). No significant differences in erroneous needle placement were noted among all surgical techniques, suggesting that STAR was as dexterous as expert surgeons in needle placement in deformable soft tissues (Fig. 2C). The efficiency of task execution was measured by total completion time for the closure. Faster execution times translate to potentially shorter operating and anesthesia times for patients. The completion time of the task, including all knots, sutures, and “other” (comprising time to restage and change sutures), was similar for all laparoscopic and robotic approaches requiring endoscopic vision (Fig. 2D). STAR was significantly slower than OPEN. STAR took slightly longer than RAS owing to preset initial slow robot speed and added restaging time for the purpose of completing the task autonomously.

Ex vivo end-to-end anastomosis feasibility

Next, we conducted ex vivo experiments with the more complicated task of leak-free end-to-end anastomosis in porcine bowel tissue. The experiment evaluated the autonomous creation, adjustment, and execution of a suture plan in a clinically relevant setting. A suturing algorithm was developed on the basis of a consensus of the best techniques of expert surgeons, target tissue geometry, and procedural awareness and consisted of a combination of knots and running sutures with more complex techniques, such as sliding the suturing tool into tight corners. Using four NIRF markers placed on the corners of the staged bowel as reference, two running sutures were placed starting from the center of the back wall outward, with the second suture coming around the front wall and tied to the first suture after completion of the front wall. An overview of the algorithm for end-to-end anastomosis is shown in Fig. 1 (G and H). The suture plan, created by the algorithm at the beginning of each procedure, incorporated a sequence of needle insertion coordinates, intersuture spacing, bite depths, pull tension forces, and suturing tool maneuvers before piercing. Using the NIRF markers as reference points, the plan interpolated intermediate suture placements on the bowel and adjusted placement of each suture, knot, and corner slide to accommodate deformations and induced scene rotations (Fig. 1F).

Comparing the consistency of suture placement and variations between consecutive stitches, STAR performed significantly better than the current standards of care for OPEN, LAP, and RAS techniques (table S2). The average suture spacing by STAR was significantly less than LAP and RAS, but was similar to OPEN (Fig. 3A). The more consistent suture spacing with STAR contributed to withstanding higher leak pressures, nearly double the pressure compared to all other manual techniques (Fig. 3B). There was an inverse relationship between maximum suture spacing and leak pressure, with a root mean square error of 0.11 psi (Fig. 3C).

Fig. 3. End-to-end anastomosis ex vivo.

The experiment consisted of closing a transverse cut in pig intestine. Five samples were tested per technique (OPEN, LAP, RAS, and STAR). (A) Suture spacing. Central mark is the median; box edges are the 25th and 75th percentiles; and red dots are outliers. The whiskers represent the range not including outliers. There is a different N number for each boxplot because each surgeon used a different number of sutures [OPEN (n = 138), LAP (n = 98), RAS (n = 132), and STAR (n = 180)]. The average spacing between consecutive sutures was calculated and compared between STAR and other modalities. The variance of suture spacing is presented numerically in table S2, including the SD. P values determined by ANOVA with post hoc Games-Howell. (B) Ex vivo end-to-end anastomosis leak pressures. Data are individual tissue samples, with means displayed as horizontal lines (n = 4 to 5). One sample was sutured closed and thus could not be tested for leak pressure. P values determined by independent samples t test. (C) The leak pressure as a function of maximum suture spacing. Data are individual tissue samples that were fit to a rational function (y = 0.854/x) (n = 4 to 5). (D) Number of mistakes (repositioned stitches or robot reboot). Data are individual tissue samples with means displayed as horizontal lines (n = 5). P values determined by independent samples t test. (E) Ex vivo end-to-end anastomosis completion times. Average times for n = 5 tissue samples per procedure are divided into subtasks of knots and running sutures. “Other” time was spent restaging and changing sutures. P values determined by independent samples t test. (F) Percent reduction in luminal area. Data are individual tissue samples with means displayed as horizontal lines (n = 5). P values determined by independent samples t test. (G) Ex vivo interaction map shows the frequency of user adjustments over the total number of robot motions. Leak locations for the five STAR experiments are shown with arrows.

The STAR system also had less than one mistake per sample that required repositioning of the needle (Fig. 3D). The average completion time by STAR was significantly longer than OPEN and RAS, but comparable to LAP (Fig. 3E). Ex vivo studies simulating luminal strictures from anastomosis suggest reducing the lumen diameter more than 40% critically reduces the flow of saline through the intestine. For more viscous fluids, such as those found in a jejunal loop, this critical value is closer to 20% (23). The average luminal reduction of STAR was 13.85%, with no significant differences among modalities (Fig. 3F). Because the average luminal reduction of STAR was less than 20%, anastomosis would not result in clinically significant narrowing or impedance of luminal flow. RAS narrowed the lumen by ~26% because of one instance where a surgeon closed the lumen.

The supervisory autonomous mode of STAR included the ability to make positioning adjustments during plan execution. More surgeon adjustments were applied in the corners (Fig. 3G) where space is limited and leaks are known to occur more frequently (24). Twenty-seven adjustments of 43 suture placements (62.8%) were made in the corners, whereas only 16 of 59 (27.1%) sutures required adjustments in the remaining circumference (Fig. 3G), suggesting that different levels of autonomy could be used effectively for different regions. Overall, 57.8% of the procedure was done fully autonomously with no adjustments. Alternatively, in the current system, fully autonomous mode without any human interaction would require suture adjustments in 42.2% of sutures placed, mostly in the corners. The completion time for STAR also included supervisory actions by the surgeon, which accounted for 12.9% of the total time (7% for suture location adjustment, 3.3% for confirmation of suture location, and 2.6% for mistake correction).

In vivo end-to-end anastomosis

Finally, we performed in vivo supervised autonomous surgery in pig intestines accessed through a laparotomy using STAR (n = 4) and compared these animals with an OPEN control (n = 1) (fig. S3). STAR used the same suture algorithm from the ex vivo trials (Fig. 1, G and H). For the OPEN control, the surgeon used standard surgical hand tools to open the abdomen, exposed the intestine, and sutured together a transverse incision. The average (±SD) STAR procedure time was 50.0 ± 14.7 min, where 77.4% was anastomosis time and 22.6% was restaging time between back and front walls, which included 5.00 ± 2.16 min for marking the tissue (fig. S3, B and E, and Table 1). Although the OPEN time was only 8 min, the STAR time was comparable to the average for clinical laparoscopic anastomoses that range from 30 min for vesicourethral (25), to 50 min for aortic (26), to 90 min for intestinal reconstructions (27).

Table 1. In vivo end-to-end anastomosis.

Bowel anastomoses were carried out in pigs using STAR (n = 4) and OPEN (n = 1).

View this table:

No complications were observed in the postsurgery follow-up of 7 days. Of the four STAR anastomoses, the leak pressure for one animal was similar to OPEN control (~0.22 psi), whereas two had no observed leak up to 1.2 psi (Table 1). The fourth STAR anastomosis did not yield a measurable leak pressure because the anastomosis was accidentally disrupted during harvesting of the specimen. However, no evidence of a previous leak was observed. The average reduction in luminal diameter for STAR was 19.9% (range, 7.1 to 33.3%) compared to 6.1% for the control animal. Histology on day 7 (Fig. 4A) illustrates that there was no observable difference in wound healing and inflammatory changes, as measured by number of polymorphic nucleated cell (PMN) infiltrates (Fig. 4B), continuity of mucosal edges, and degree of tissue injury such as hematoma, between STAR and the control.

Fig. 4. Histology of healed anastomoses.

(A) Representative hematoxylin and eosin (H&E) examples from each anastomosis tissue operated on with STAR (n = 4) and by hand sewing in vivo. The area of healed anastomosis is identified with an arrow, and the region of interest (ROI) for magnification is outlined. A blue arrowhead in the lower first column panel identifies suture fibers. PMNs as surrogates of inflammation were counted at a magnification of ×400 at three random ROIs from each anastomosis site. (B) PMN cell counts for each sample. Cells were counted three times for each sample, averages are marked with a horizontal line. P values determined by two sample t test.

DISCUSSION

Despite unpredictable scene changes and soft tissue deformation, complex surgical tasks requiring human dexterity and cognition can be programmed and executed. We demonstrate with soft porcine tissues that supervised autonomy with STAR not only is feasible but also, by some metrics, surpasses the performance of accepted surgical techniques, including RAS, LAP, and manual surgery. These results promise that autonomous surgery can bring better efficacy, safety, and access to the best surgical techniques regardless of human factors, including surgeon experience.

The intent of this demonstration of feasibility in soft tissue surgery was not to replace surgeons but to expand human capacity and capability through enhanced vision, dexterity, and complementary machine intelligence for improved surgical outcomes, safety, and patient access. The current paradigm of teleoperated RASs depends entirely on the capability of each operating surgeon. Autonomous robotic surgery has been limited to applications with rigid anatomy, such as bone cutting, because they are more predictable. Enabled by a novel plenoptic 3D and NIRF imaging system and an autonomous suturing algorithm, we were able to track the target tissues in dynamic surgical environments to execute complex surgical tasks autonomously. Our work specifically focused on anastomosis, performed each year more than 1 million times in the United States for visceral (gastrointestinal, urologic, and gynecologic) indications (28, 29). This task represents a proof of concept for all potential soft tissue surgical tasks requiring repetition, precision, accuracy, and efficiency that can potentially benefit from autonomous or supervised autonomous functionality. Further miniaturization of tools and improved sensors will allow for wider use even at smaller scales, for example, in vascular anastomoses.

For the purpose of demonstrating the feasibility of autonomous surgery in soft tissue, we chose an open surgical approach in a large animal. A miniaturized laparoscopic 3D camera will enable laparoscopic minimally invasive surgery approaches in the future. In addition, although the current vision technology combining plenoptic and NIRF imaging permits autonomous anastomoses, it does require an additional step—albeit simple—of placing NIRF markers in the workflow. Although stapling instruments have become popular in many procedures (for example, gastric bypass), their usage for anastomosis is not straightforward. Studies have shown that laparoscopic stapling with a linear laparoscopic stapler, such as Covidien’s Endo GIA, can perform anastomosis more quickly than manual suturing (30). However, because of the linear design of the stapler, bulky physical footprint, and unnatural side-to-side stapling step for end-to-end anastomoses, an increased rate of stricture has been reported (31). The utility of circular staplers for end-to-end anastomosis such as EEA (Covidien) and ILS (Ethicon Endo-Surgery) circular staplers is limited to proximal foregut and distal hindgut (32, 33). The burst pressures for end-to-end anastomoses in pig small bowels using linear and circular staplers (Endo GIA and PCEEA, both from Covidien) are reported as 19.8 ± 7.4 and 28.3 ± 6.8 mmHg, respectively (34). The burst pressure after STAR end-to-end ex vivo anastomosis was 16.76 ± 4.08 mmHg, which is comparable to these other techniques. The higher burst pressures sustained by staples is likely due to the density of staples (often three rows of tightly overlapping metal staples).

Supervisory control (10) of surgical tasks is the first step to address more seamless integration of fully autonomous surgical systems into the operating room in the future. Future work will include the development of laparoscopic imaging capabilities and a clinical-grade system to translate the work into the operating room. Efficiency is a variable that could be improved in future iterations. Additional imaging modalities, such as multispectral imaging, incorporating subsurface anatomic structures and physiology, such as perfusion status, in the suture planning stage will further improve the current surgical vision and clinical outcomes (35). The combination of autonomous planning and robotic execution of soft tissue surgery minimizes technical variability and achieves superior functional outcomes compared to manual and RAS approaches. Such interactive adaptive decision-making between the surgeon and the robot demonstrates a new paradigm for future robotic surgeries.

MATERIALS AND METHODS

Study design

The goal of the study was to determine whether a complex surgical task, such as anastomosis, can be performed in a soft, deformable, mobile tissue autonomously. To test this hypothesis, we chose two models of ex vivo pig intestinal tissue suspended in a similar manner to clinical anastomosis and a preclinical porcine in vivo model. We compared the performance of our robot to those human-performed laparoscopic, OPEN, and robot-assisted surgical techniques.

During the ex vivo experiments, a sample size of n = 5 tissue samples was chosen to facilitate an appropriate statistical analysis without placing undue burden on participant surgeons. Five surgeons, per technique, were asked to perform a linear and/or end-to-end anastomosis of porcine small intestine using one or more of the three standard techniques (OPEN, LAP, and RAS) with which they were familiar to compare with STAR anastomosis. Each of the five samples per technique was operated on by a different surgeon, and some surgeons were in more than one technique group. The study endpoint was reached when all five samples were collected for each technique, with no exclusion criteria or data collection stops. The surgeon and robot were blinded to the amount of randomized deformation induced during the procedure to test their adaptation to the changing environment. The metrics of assessment were the consistency of suturing informed by the average suture spacing, the pressure at which the anastomosis leaked, the number of mistakes that required removing the needle from the tissue or restarting the robot, and the completion time.

For the in vivo experiments, animals were randomly assigned to either the control (n = 1) or the experimental (n = 4) group. The purpose was to demonstrate safety and efficacy in living tissue. In addition to the metrics described in the ex vivo study, we also compared weight gain after surgery and histology.

Soft tissue deformation

Soft tissues vary widely in their deformability. The Young’s modulus for nonhomogeneous intestinal tissue averages 1.83 MPa (36) in contrast to 14.8 GPa in cortical bone (37). Because soft tissue deforms unpredictably when touched by surgical tools, tissue tracking and measurements of applied forces are particularly important for these delicate surgical tasks. Surgeons typically apply a tension force of 2 N when pulling a suturing thread (20). This tension results in 2.6 mm of deformation for bowel tissue of 1.3-mm thickness, while under the assumptions detailed under fig. S1B, with the boundary condition illustrated in fig. S1A. Any soft tissue robot must be able to adapt to this amount of deformation at a minimum.

Positioning platform

The main positioning platform of the STAR system is a seven-DOF lightweight robot arm (LWR 4+, KUKA Robotics Corporation) with repeatable 0.05-mm positioning. The robotic arm is kinematically similar to a human arm, which makes it a good candidate for both patient-side and collaborative surgical robots. The end effector is extended with a one-DOF suturing tool (Endo360, EndoEvolution). The suturing tool is controlled through a CANbus with two motors, one for articulation and the other for needle drive. The combined positioning kinematic chain has eight DOF.

Custom control software was developed using the Open Robot Control Software (OROCOS; www.orocos.org) to calculate forward and inverse kinematics of STAR. Once a goal trajectory was calculated for a desired configuration, the joint values were communicated to both the arm and the suturing tool. The KUKA robot controller (KRC) computer was in charge of the closed-loop control to reach the target configuration for the first seven DOF. A program written in KUKA robot language enables communication between the KRC and the OROCOS components via Ethernet using the Fast Research Interface. The suturing tool articulation was controlled by the CANbus controller OROCOS component. The Robot Operating System (www.ros.org) provided a high-level command interface though messages and topics architecture.

Force sensor and control

Limiting suture tension between the suture thread and the tissue can prevent strictures and stenosis. We placed the force sensor (Gamma, ATI Industrial Automation) between the last link of the robot arm and the suturing tool. With known orientation and mass parameters of the arm and the tool, the forces applied to the tool tip were calculated and added to the measured forces at the ATI frame. It would be ideal to measure force at the distal end of the suturing tool where the needle and suture thread are located, but this was not possible because we used a clinical tool that is designed for manual use, not automation. In practice, a threshold of 1 N indicated that the maximum suture tension force had been reached. To ensure that the robot does not deform the tissue unnecessarily during approach, we limited the applied force between the suturing tool and the tissue to 0.92 N measured by the force sensor.

Application and 3D tracking of NIRF markers

Motion and deformation of soft tissue necessitate online adjustments to positioning targets and the suture plan during the procedure. NIRF markers were created by diluting indocyanine green (ICG) (IC-Green, Akorn Inc.) into acetone and mixing with cyanoacrylate (Permabond) such that the solution was viscous enough to mark tissue in an irrigated surgical field, yet fluid enough to permit easy application. ICG and cyanoacrylate are FDA-approved for clinical use in other medical applications. A disposable syringe with a 25-gauge needle was used as the delivery mechanism so that 10 μl of the solution is consistently applied to the surface of the tissue in a roughly circular shape for each NIRF marker. The markers were applied to the four corners of the suspended intestine after setup for anastomosis. The surgical assistant placing the markers verified that each marker was well visible on the NIR camera.

The 3D NIRF visual tracking system observes changes in the point cloud and detects the motion of NIRF markers to inform other software components, including the automation software component, and motion control component (see the Supplementary Materials). To quickly adapt to dynamic changes, we took advantage of the higher frame rate of the NIR camera (30 Hz versus 10 Hz for 3D plenoptic) and performed visual tracking on the 2D NIR image stream (38). The coordinates of the NIRF markers were initialized. The tracking module computed the center of gravity and higher-order moments of the tracked marker blob. If a marker was lost, a bounding box was searched around the previous position, and tracking resumed once the marker reentered the box. The NIR image was registered to the plenoptic image such that each pixel on the 2D NIR image corresponds to a 3D data point on the metric point cloud computed by the plenoptic camera (fig. S1).

Suture automation software

The suture automation software fit a polyline through the tracked 3D NIRF markers. On the basis of measurements of cut length and tissue thickness, which were obtained by either sensors or surgeon input through a graphical interface, the software generated a geometrically optimized suture plan, which could be tracked and updated in real time despite tissue motion and deformation (fig. S2). If the automatic suturing tool placement was not perfect, the surgeon then had the opportunity to make position adjustments in supervisory mode. More information is available in the Supplementary Materials.

Ex vivo linear suturing

The ex vivo linear suturing experiment was designed to evaluate STAR’s ability to automatically detect and adapt to typical soft tissue deformations. A 7-cm length of 15-mm-diameter ex vivo porcine bowel tissue was gently suspended by five stay sutures within a ring. The stay sutures were positioned evenly along the length of a 5-cm longitudinal cut in the tissue and tensioned to 1 N of force at the start of the procedure. The task was to close the longitudinal cut without clinical leaks with a 3-0 suture using a running stitch, with three ties per knot. The leak pressure was measured similar to reported literature (39).

The tension of each stay suture was randomly varied at three instances during the procedure to deform the tissue between 2 and 6.5 mm. The induced tissue deformation occurred first at marker 2 and then at marker 3 with a random rotation of the tissue in the range of −7.5° to 17.5°, which resulted in motion of all markers, and finally at marker 4. These motions occurred without informing the robot or the surgeons. For STAR, the biocompatible NIRF markers were placed on the tissue at the start of the procedure (fig. S1). STAR generated a suture plan, detected deformations during the procedure, automatically adjusted the suture plan to correct for the unstructured motions, and executed the updated suture plan.

Ex vivo end-to-end anastomosis

The impact of STAR in the clinical workflow was evaluated by the ex vivo end-to-end bowel anastomosis with running sutures and three tie knots. A porcine bowel was transversally cut, and the mesenteric side was approximated. Two stay sutures were loosely placed at mesenteric and antimesenteric corners to suspend the two luminal openings. Two additional stay sutures were placed in the middle to open the lumens similar to a standard clinical technique. All stay sutures were then suspended from a staging ring and tensioned to 1 N to gently stabilize the tissue. To replicate the typical tissue motions during bowel anastomosis, the tissue was randomly rotated between −10° and 10° at three different instances. After completion of the suturing of the inside wall, the bowel was restaged manually to complete the anastomosis on the front wall (fig. S3D).

In vivo end-to-end anastomosis

To demonstrate safety and efficacy in living tissue, we used STAR to complete an end-to-end bowel anastomosis in a 1-week survival study (n = 4). Yorkshire piglets weighing between 20 and 25 kg (Archer Farms Inc.) were selected because of size, texture, color, and intestinal mechanical properties that are similar to pediatric patients. All animals received humane care following the National Institutes of Health Guide for the Care and Use of Laboratory Animals and the Institutional Animal Care and Use Committee (#00030403). A control group (n = 1) was performed using hand-sewn OPEN technique. Cefazolin was administered as a preoperative antibiotic. Each piglet was sedated with the use of ketamine (20 mg/kg) and xylazine (2 mg/kg). A saline drip through an IV placed in the ear was administered continuously. Each piglet was then intubated and connected to a ventilator. Isoflurane was used to induce anesthesia, and the surgical site was prepared using sterile technique.

A laparotomy with an incision length of about 7 cm was performed to expose a loop of small bowel in each piglet. Once exposed, the tissue was transversely cut and gently suspended by four stay sutures similar to the ex vivo setup. The STAR system was used to reconstruct the tissue using 3-0 polyester braided/coated suture with running stitch. Completion time for the anastomoses and number of suture mistakes were recorded for each animal. Upon completion, the anastomosed bowel was visually inspected for leaks and then placed back inside the abdominal cavity. The abdominal wall was closed using 2-0 suture, and a bandage was placed over the incision site. At the time of sacrifice, the weight of the animal was recorded, and the small bowel was excised after euthanasia and inspected for leak. Leak pressure and luminal reduction tests were performed on the excised small bowel containing the anastomosed region, using the same techniques as the ex vivo experiments. To assess healing at the cellular level, the sample was cut along the mesenteric edge, sectioned, and prepared with H&E staining.

Statistical analysis

IBM SPSS Statistics 24 software was used to conduct analysis of suturing metrics. One-way analysis of variance (ANOVA) is used to test for significance across modalities when examining the spacing results of all ex vivo trials. The results of the STAR experiments were compared using ANOVA with post hoc Games-Howell. Games-Howell test was chosen because it is designed for unequal variances and takes into account unequal group sizes. Levene’s robust test statistic was used to compare the equality of variances in suture spacing across modalities. An independent samples t test was used for orthogonal comparisons between STAR and each of the manual suturing techniques when examining the leak pressure, number of mistakes, procedure time, and lumen reduction results. T test was chosen because it is more robust to errors from small data sets, and our initial hypothesis only considers the relationships between STAR and each of the manual techniques. Outliers in suture spacing were required to be more than ±2.7σ from the mean. P values account for the effects of each comparison and are reported against the STAR system. P < 0.05 was considered statistically significant for all tests.

SUPPLEMENTARY MATERIALS

www.sciencetranslationalmedicine.org/cgi/content/full/8/337/337ra64/DC1

Materials and Methods

Fig. S1. Young’s modulus calculations for pig bowel and human bowel, with geometry and applied forces kept constant.

Fig. S2. Deformation patterns for markers illustrate the uncertain nature of motion propagation in soft tissue.

Fig. S3. Robotic construction of end-to-end anastomosis.

Table S1. Quantitative geometric quality of ex vivo linear suturing.

Table S2. Quantitative geometric quality of ex vivo end-to-end anastomosis.

Movie S1. Supervised autonomous end-to-end intestinal anastomosis.

REFERENCES AND NOTES

Acknowledgments: We acknowledge C. Cochenour for technical assistance and R. McCarter for help in statistical analysis. Funding: This work was supported by the Sheikh Zayed Institute for Pediatric Surgical Innovation and Joseph E. Robert Jr. Endowment Awards. Author contributions: A.S., A.K., and P.C.W.K. created the study design. A.S., R.S.D., and S.L. developed the software. A.S., R.S.D., A.K., and J.D.O. carried out deformability experiments and analyzed the data. P.C.W.K. performed the in vivo OPEN procedure. A.S., R.S.D., J.D.O., A.K., S.L., and P.C.W.K. performed the STAR in vivo procedures. A.K. and J.D.O. provided animal care oversight. J.D.O. performed the statistical analysis. P.C.W.K. and J.D.O. performed histology analysis. A.S., R.S.D., J.D.O., S.L., A.K., and P.C.W.K. reviewed some or all of the primary data. A.S., P.C.W.K., A.K., R.S.D., and J.D.O. wrote the manuscript. All authors reviewed the manuscript. Competing interests: P.C.W.K., A.K., A.S., S.L., J.D.O., and R.S.D. are on the following related patents: 61/705,875; 13/863954; US-2014-0005684-A1; 14/038,192; 14/172,502; 61/909,604; 62088545; and 14625425. P.C.W.K. is a founder of Omniboros Inc., which develops smart automated and soft robots. Data and materials availability: There are no material transfer agreements or restrictive patents.
View Abstract

Navigate This Article