A hybrid deformable model for real-time surgical simulation

https://doi.org/10.1016/j.compmedimag.2012.03.001Get rights and content

Abstract

Modeling organ deformation in real remains a challenge in virtual minimally invasive (MIS) surgery simulation. In this paper, we propose a new hybrid deformable model to simulate deformable organs in the real-time surgical training system. Our hybrid model uses boundary element method (BEM) to compute global deformation based on a coarse surface mesh and uses a mass-spring model to simulate the dynamic behaviors of soft tissue interacting with surgical instruments. The simulation result is coupled with a high-resolution rendering mesh through a particle surface interpolation algorithm. Accurate visual and haptic feedbacks are provided in real time and temporal behaviors of biological soft tissues including viscosity and creeping are modeled as well. We prove our model to be suitable to work in complex virtual surgical environment by integrating it into a MIS training system. The hybrid model is evaluated with respect to efficiency, accuracy and robustness by a series of experiments.

Introduction

Interactive surgery simulation technique plays an important role in today's medical education system. Virtual reality (VR) and surgical simulators can provide the opportunity to expose each surgical resident to a wider surgical experience and to make the training more uniform. Over the past decades, training systems based on VR technology for different types of surgeries have been developed, including facial plastic [19], cataract [7], neurosurgery [38], knee [5], laparoscope [40], hysteroscopy [41], and bone dissection [21]. Among these, training systems for minimally invasive surgery (MIS) have drawn a lot of attention. MIS is advantageous over other open surgeries for the smaller wound and less pain after operation. But it is much more difficult for a surgeon to operate based on the images provided by a laparoscopic camera in MIS than in a common open surgical situs. So a pre-operative training for a surgeon to adapt the anatomic knowledge to the laparoscopic environment, to practice manipulating surgical instruments with constraint degrees of freedom, and to get a sense of depth and direction in the new perspective is very necessary.

A variety of computational physics methods have been developed for realistically modeling the virtual surgery environment [4], [29], [31], [32] and the user interactions, including the collision detection and response between soft tissues and instruments [40], deformation of organs in contact with surgical instruments [45], surgical operations including needle insertion [6], cutting and sewing [42]. A fast and robust deformable model continuously providing visual and haptic feedback to users is essential to the surgery training systems. There are four basic requirements for a deformable model used in real-time MIS simulation: accuracy, efficiency, robustness, and easiness to integrate into the system. First, in contrast to deformable models used in video games and animations, the purpose of such models in medical simulation is to model the behaviors of realistic biological tissues. The deformation should be controlled by real material parameters (Poisson Ratio and Young's Modulus) taken from biomechanics experiments instead of intuitively adjusted parameters. Some specific deformation effects of biological soft tissues, such as viscous response and free vibration, need to be simulated too. Second, the model needs to be fast enough to provide results within 1/30 to 1/50 s. This eliminates most of the standard methods developed in computational physics, which are too slow to be used in real-time applications. Third, the model needs to be robust to provide results under large deformations and in large timesteps. Fourth, the model should be easy to integrate into a complex MIS simulator. In a typical virtual surgical environment, objects with different material properties, phases, geometry shapes and data representations are often combined together, and different types of interactions and responses between them need to be handled together.

Simulation of deformable objects becomes a hot topic in biomedical engineering, computational physics, and computer graphics. A lot of methods have been proposed over the past decades, and we refer the reader to the survey [27] for an overview.

Deformable models can be classified into two categories: physics-based and non physics-based. Physics-based methods are based on continuum mechanics, and could get accurate simulation results by directly solving the partial differential equations (PDEs) using numerical methods. Some of the prevailing methods include the finite element method (FEM) [28], boundary element method (BEM) [12], point-based method [24], and reduced model [11]. Non physical models use intuitive methods instead of solving PDE. For example, the mass-spring model [35] uses point masses connected by a network of springs to represent continuous material, and meshless shape matching model [23] computes deformations based on geometry shapes.

There are some advances in real-time modeling of deformable objects in medical simulation and computer graphics. Müller et al. [22] use corotational FEM to simulate large deformations with implicit integration in real time, and similar methods are used in simulating biological soft tissues in [28]. In [26], cube meshes are used instead of tetrahedron meshes to simplify the matrix computation in real-time FEM simulation. In [12] and [14], boundary element method, which focuses on the surface, is proposed by James and Pai to model deformable objects in interactive applications. Reduced models [13] divide the deformation into different modes and model the global deformations with very low computation cost. Meshless methods use discrete points instead of tetrahedron mesh to represent continuum, and solve the PDE by employing interpolation methods such as moving least squares (MLS) [24] or smoothed particle hydrodynamics (SPH) [20]. Non-physical methods can also be pushed to accurate modeling by carefully adjusting the parameters and adding physical constraints. In [35], additional volume conservation constraints are combined with mass-spring model to simulate incompressible material. In [8], mass-spring model with parameters measured in experiments is used to simulate the local deformation of anatomical organs in surgical environment.

BEM is a physics accurate method restricting the computation domain on boundaries. It computes the deformation by numerically solving the boundary integration equation on a surface mesh. Because the simulation is only on boundary, no computation resource is wasted in updating the volume and therefore the efficiency is very high. James and Pai achieved to simulate deformable objects using BEM in interactive application ArtDefo [12]. One problem of BEM is its scalability, because the global matrix of BEM is dense and asymmetric and needs to update in each timestep. To improve this, James and Pai [14] use multi-resolution Green Function to accelerate the computation of BEM matrix. BEM are also used in virtual surgery to model the deformation of organs. Kim et al. [16] use local surface mesh subdivision and interpolation to enhance the details of contact areas in surgical simulation. Zhu et al. [43] use a finite state machine to estimate the stressing state of soft body in surgical simulation. Wang et al. [39] integrate BEM with cutting algorithm, which extends the applications of BEM in surgical simulation.

Reduced deformable models [1], [11], [13] also draw a lot of attentions in real-time deformable modeling. In reduced models, modes of global deformation are computed using modal analyzing [11], dynamics response textures [13] or non-linear techniques [1]. The deformation is then embedded into a standard rigid body dynamics simulator and both systems evolve over time. In our coupling method, we use BEM computation and mass-spring model to couple the precise global deformation information with a time evolving model.

We propose a new hybrid deformable model for real-time surgical simulation. Our model is able to provide visually accurate and robust results for the surgical trainees and is able to work efficiently in a real-time MIS simulator. The basic idea is to couple the boundary element method into a mass-spring constraint model. Instead of just solving the boundary value problem of linear elasticity as in standard BEM, BEM is used to compute the global deformation which is regarded as target positions for mass particles in a mass-spring model. And the global deformation shape of mass-spring model is controlled by the target positions. By tuning the stiffness and damping parameters of spring, we are also able to model the temporal and detailed deformation behaviors of the organ.

Our hybrid model adds the ability of modeling temporal deformable behavior to the standard BEM. In essence, simulating deformable objects using BEM is solving a boundary value problem (BVP) which is unrelated to the time variable. BEM simulates the deformation behavior of perfect linear elastic material based on the current boundary displacement and traction conditions of the object. It cannot model the non-linear deformation behavior related to time including viscous or damping effect. We solve this problem by integrating BEM into a dynamic deformable model driven by an extra mass-spring model. This turns the pure boundary value problem into a initial value problem which is easy to solve.

Section snippets

Surface modeling from medical data

We use three different types of data structures to represent the surface of a deformable object: a high-resolution polygon mesh for rendering, a low-resolution polygon mesh for physics computation, and a particle sampled surface to synchronize the two meshes (as in Fig. 1). The two surface meshes are generated from raw medical data using standard surface reconstruction algorithm, and the particle surface is generated by sampling the coarse triangle mesh. Different surfaces are synchronized

Model overview

As in Fig. 2, our hybrid method contains three main parts: a static BEM-based elasticity solver, a dynamic mass-spring model, and a particle surface interpolation algorithm. In the preprocessing stage, the global deformation matrix of BEM is initialized using the coarse geometrical mesh and the measured linear elastic material parameters (Poisson Ratio and Young's Modulus). Other experimented parameters including the spring parameters are also loaded in this stage. In runtime computing, the

Experiments and evaluation

In this section we evaluate our hybrid deformable model by a series of experiments with respect to accuracy, efficiency and robustness. A MIS training system integrated with our hybrid model is presented in the last. Our model is implemented in C++ and rendered using OpenGL. All the experiments were carried out on a 2.26 GHz Pentium M notebook PC with GeForce 9650M graphics card and 2 GB of memory.

Conclusion

In this paper we propose a new hybrid deformable model for real-time surgical simulation. Our model uses a BEM model to compute the global deformation and uses a mass-spring model to interactively model the dynamic behaviors of organs. The hybrid model is suitable for interactive surgical training applications, and provides visually accurate results in simulating the deformation of biological soft tissues with experimental inputs. Since our model is represented in a hybrid way including both

Acknowledgments

We would thank all the group members of IGST for their valuable comments and suggestions. This paper was partially supported by the Chinese NSFC research fund (61190124) and the international research fund of STCSM (10440710600).

References (41)

  • N. Chentanez et al.

    Interactive simulation of surgical needle insertion and steering

  • F. Hartmann

    Introduction to boundary elements: theory and applications

    (1989)
  • Y.C. Fung

    Biomechanics: mechanical properties of living tissues

    (1993)
  • K.K. Hauser et al.

    Interactive deformation using modal analysis with constraints

    Graphics interface

    (2003)
  • D.L. James et al.

    ArtDefo: accurate real time deformable objects

  • D.L. James et al.

    DyRT: dynamic response textures for real time deformation simulation with graphics hardware

    (2002)
  • D.L. James et al.

    Multiresolution green's function methods for interactive simulation of large-scale elastostatic objects

    ACM Trans Graph

    (2003)
  • J. Kim et al.

    Virtual surgery simulation for medical training using multi-resolution organ models

    Int J Med Rob Comp Assist Surg

    (2007)
  • R. Keiser

    Meshless Lagrangian methods for physics-based animations of solids and fluids

    (2006)
  • R.M. Kenedi et al.

    Tissues mechanics

    Phys. Med. Biol.

    (1975)
  • Cited by (39)

    • Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning

      2023, Computerized Medical Imaging and Graphics
      Citation Excerpt :

      Virtual reality simulators thus enable trainees to practice on a variety of educational scenarios as well as to enable the definition of new training metrics and applications (e.g. remote training). To provide realistic touch sensations through haptic feedback devices, virtual reality simulators require a real-time simulation of tissue deformation (Zhu and Gu, 2012). The behavior of the tissue is critical during neurosurgery and its simulation is challenging due to its complex nonlinear viscoelastic nature.

    • Maximum likelihood-based extended Kalman filter for soft tissue modelling

      2023, Journal of the Mechanical Behavior of Biomedical Materials
      Citation Excerpt :

      The boundary element method (BEM) greatly simplifies the computational complexity of FEM by expressing the weak form of the virtual work as a surface integral form to condense the calculations to boundary surface elements only. However, one problem of BEM is its scalability, because the global matrix of BEM is dense and asymmetric and needs to be updated at each time step, which increases the computational cost (Zhu and Gu, 2012). The meshless method computes the deformation without involving the mesh topology of a discrete tissue model.

    • An improved method for soft tissue modeling

      2021, Biomedical Signal Processing and Control
    • Fractal analysis tools for early assessment of liver inflammation induced by chronic consumption of linseed, palm and sunflower oils

      2020, Biomedical Signal Processing and Control
      Citation Excerpt :

      The advancements in the computational capabilities, as well as information and communication technologies (ICT), over the past two decades, have contributed significantly to the improvements in the availability and quality of health care services. The attained benefits include improved accuracy and reliability of tools for the virtual intervention planning [1–3], development of very sophisticated clinical decision support systems (CDSS) through the extensive data mining [4–8], and computer-aided diagnostic (CAD) tools for early disease detection [9–12]. On the other hand, there has been significant recent progress in the development of non-invasive microscopies, such as the three-photon microscopy, contrast-enhanced MR microscopy, and similar modalities [13–15].

    View all citing articles on Scopus
    View full text