VoxeLine: a software program for 3D real-time visualization of biomedical images

Abstract

The architecture and implementation of VoxeLine, a new interactive environment for display and analysis of 2D and 3D images in real-time, is discussed. This modular software project comprises two main parts: a user part (without programming expertise) and a programming part which permits its adaptation to specific problems. VoxeLine has the ability to deal with almost all sorts of data types encountered in the biomedical field (e.g. images, vectors). Another important feature is its ability to show datasets in all directions without duplicating data into the main memory. This feature allows VoxeLine to be used on machines with limited memory capacities and power. Real-time 3D manipulations (10 Hz for a 256×256×124 MRI dataset) are possible on a classic monoprocessor architecture such as a personal computer.

Introduction

Medical imaging has experienced crucial changes since the 1980s. Previously, only X-ray radiographs were available which suffered from poor contrast and gave no volumetric information. New tomographic imaging modalities such as computed tomography (CT), positron emission tomography (PET) or magnetic resonance imaging (MRI) have been developed, due largely to the advent of computer technologies. Data types which mainly consist of images acquired from these biomedical devices can be represented in various modalities and dimensions. These modalities give researchers a 3D vision, closer to reality, of an object represented as a sequence of parallel cross-sections. A more complicated step was to render a set of cross-sections as a 3D object onto a computer screen 1, 2. A number of methods have been proposed to display 3D data acquired from medical imaging devices 3, 4. This volume visualization aspect was a real challenge during the past decade and various medical software programs now provide 3D views using computer graphics methods [5]. In parallel, 3D visualization applications in the biomedical field are emerging 6, 7. They essentially concern surgery [8] and neurosurgery 9, 10, endoscopic imaging [11], localization of macromolecules 12, 13, molecular graphic presentations [13] and research or educational applications [14].

3D visualization strategies were designed first to extract the surface of the object and then to display a meshed geometrical object composed of connected triangles. Thereafter, the main strategy was to form a 3D shape using each sample of the raw data set and to display it using surface shading algorithms [15]. This approach, usually referred to “surface rendering”, has the disadvantage of requiring a preliminary step in the 3D object shape construction. Nonetheless, the quality of the rendered pictures is now considerably improved and 3D merging of different modalities is possible 16, 17. However, a major limitation in these methods remains the computation time required to internally construct the 3D picture and display it. Another rendering technique, referred to as “volumic rendering”, consists of managing the 3D data set as it is, without the triangulation step. We used this technique in the VoxeLine software program both to be able to freely model 3D objects as desired and for better time performance.

3D rendering involves intensive computer work and as such the overall processing speed still remains a major problem. With the growing power of computer hardware, several software programs offer the ability to compute a 3D view within a few seconds. This feature has the global advantage of providing high-quality illustrations but forbids a 3D interactivity, which has to be finally performed in 2D projection views. The most recent implementations of 3D rendering algorithms do not provide 3D real-time manipulations 5, 18, 19 and only some research products are able to quickly display large volumes using several processors in parallel and on dedicated machines [20]. Parallel to these visualization products, other software programs were written to display and analyse 4D data sets in time and space, such as functional MRI [19], to apply specialized image processing routines [21], to implement image vision algorithms 22, 23 or analyse functional anatomy protocols [5]. Commercial products are also available and offer new perspectives in data visualization and educational purposes [16]. For instance, Curry™ (Neuro Scan, Herdon, VA, USA) specializes in combining electrical activity data of the brain (magnetoencephalography (MEG) or electroencephalography (EEG)) with magnetic resonance imaging (MRI). It essentially focuses on multimodal imaging and source location. MEDx (Sensor Systems, Sterling, VA, USA) is another example of a functional image visualization and analysis program capable of processing and combining multidimensional data. This package, based on research products like AIR [24] for image registration or SPM [25] for statistical mapping, remains a complete uniform image-processing platform but without true 3D possibilities 26, 27. More often than not, these available products focus on part of the problem only: a nice display without image processing, image manipulation capabilities without true interactivity or 3D views without 2D interactions. These limitations can be explained by historical reasons and the lack of computer power and memory to process medical images. Another example of projects dealing with 3D colour animation is given by Narayan et al. [28], but unfortunately, animation frames were precomputed and stored on video disk units. There are many other software packages that are able to generate 3D renderings of medical images. But as pointed out by Chen et al. [29] (discover program), they are of moderate interest only because of their globally poor time performances. Computer terminals that allow a display rate slower than five frames per second are not really reliable for use in routine circumstances: specialized Transputer architectures (24 T800 CPUs for discover) gave about six frames per second several years ago. To our knowledge, a very small number of programs are capable of interactively displaying a rendered 256×256×256 voxel-based object on a non-dedicated monoprocessor machine. However, several programs use resources provided by specialized 3D processors. Vitrea™ and VoxelView® (Vital Images, Minneapolis, MN, USA) are examples of such programs which are especially designed to operate on several different Silicon Graphics platforms. This, unfortunately, does not allow the same display performances on small graphics stations or personal computers. Even if a true real-time is rarely achieved (25 frames per second), one should consider that a real-time visualization program is a software program which must allow a mouse-driven motion of a 3D volume without saccadic artefactual movements due to the slow speed of the computation. This definition also implies a fully interactive 3D dissection capability of 3D discrete volumes. Thus, we attempted to build such a software program, avoiding, as far as possible, all machine or architecture dependencies.

This paper contains four main parts: (1) a design description of the VoxeLine project; (2) a global listing of the main features available for the non-computer-scientist user; (3) a brief description of the most important structures and libraries used internally in this software environment package; and (4) a discussion about time performances and software development issues.