Introduction

Proteins are three-dimensional (3D) objects [1] and, for the last century, spatially-resolved structural models of proteins and other biologically relevant molecules have been provided by various experimental techniques. X-ray crystallography was particularly instrumental in this revolution [2] with the very first structure of a protein resolved by this method in the 1950s [3]. Since then, X-ray crystallography has led to the building of detailed protein models and was instrumental for a number of important advances, with Watson and Crick’s accurate 3D model of the DNA structure as a prominent example [4]. The 3D characteristics of protein molecules are important in aiding our comprehension of many biological processes. Furthermore, beyond the classical “structure implies function” approach, it is now also becoming increasingly clear that protein dynamics is key to understanding protein function [5]. One of the ways we could potentially access this information is by interacting with, manipulating and visualising static and dynamic models of such proteins in 3D. These might be constructed as real objects or exist in a virtual reality (VR) environment. A large part of scientific and medicinal research on drugs, such as, e.g., understanding antimicrobial resistance (AMR), revolves around clarifying the ways drugs bind to proteins and vice versa. In the, e.g., AMR context, viewing protein structures and their dynamics and understanding how mutations can lead to conformational changes and, thus, changes in binding regions that are relevant to AMR, is essential.

Since computers became ubiquitous, the development of computer models for proteins, as opposed to physical ones, in turn became progressively easier and cheaper. With that came the possibility to show proteins in stereoscopic 3D view. Many such projects were developed, as TAMS, for example, which used polarized slide projectors to display stereoscopic 3D images [6]. Many tools exist to make viewing protein structures possible, such as PyMol [7], VMD [8], Rasmol [9], Chimera [10] and Isolde [11]. In recent years, there was a further push to develop systems based on web browsers, such as iView [12] and Jmol [13]. To some degree, every one of those tools can produce stereoscopic 3D images, be it through passive 3D (using chromatic distortion glasses), active 3D (with shutter glasses synchronised to the image displaying device) and autostereoscopic 3D (no headgear required) [14]. Though their method changes significantly, they all aim for the same near-3D effect for the end user. Besides technical drawbacks that either limit resolution or require expensive equipment, these attempts at providing 3D perception when analysing protein structures lack immersion into a different environment and do not lead to a feeling of true 3D presence.

VR allows us to address this lack of immersion, and introduce a level of interaction between user and visualisation tool that was not possible before. While the usage of VR in research is not new, the current levels of performance with affordable cost definitely are. Implementing a VR can be achieved through many different techniques. On one side of the complexity scale, we could point towards whole-room arrangements like cave automatic virtual environments [15], while extremely simple solutions like Google Cardboard [16] would lie at the other end of that scale. Somewhere in between we find modern head-mounted devices (HMDs), like the Oculus Rift [17] and the HTC Vive [18]. These create a stereoscopic 3D effect through the usage of LCD displays and lenses that allow the system to display different images for each eye, with slightly different points of view that mimic the position of the eyes of a virtual observer. These HMDs are relatively easy to use and to program, with excellent display quality and affordable prices. The prominence of such HMDs amongst the gaming community is particularly useful for researchers. Since VR gaming has gained popularity, the tools for programming software to use HMD capabilities have become better, more streamlined and easier to use for people even without a background in graphics programming. Initiatives such as SteamVR [19] and VR addons for the Unity3D game engine [20] decrease the amount of work necessary to build a VR application considerably. Thus, VR emerges as an ideal solution for visualising protein dynamics. It allows us to clearly communicate the large-scale motions taking place on the protein structure, while making interaction possible for looking closer at specific areas of interest, such as binding sites.

VR is now beginning to emerge as an add-on option to molecular visualizers. Projects such as RealityConvert [21] and the web-based tool Autodesk Molecule Viewer [22] already provide “blackbox” VR viewers, while the Nano Simbox [23] pioneers ambitious real-time molecular dynamics simulation visualized in VR. This paper reports on a protocol for introducing protein structures into VR programs, using a combination of widely and freely available software and custom-built scripts and programs. The protocol is easily adapted to other molecules rather than just proteins, such as, e.g., DNA, RNA and whole viruses. Besides being a useful tool for researchers to visualise conformational changes in proteins, VR also provides a great outreach opportunity to motivate the general public to understand proteins and their relationship to biochemical research and its applications in drug discovery better. We aim with this paper to give our readers the necessary tools to start their own VR applications. Our freely available VR setups for HTC Vive and Oculus Rift can be downloaded from Ref. [24].

Materials and methods

We choose the HTC Vive [18] as a convenient HMD for VR. It uses two infrared emitters (“lighthouses”) in the corners of the play area, which generate infrared beams in sweeping patterns. The headset possess multiple infrared photo diodes, which will detect these beams, and the position of the headset in the room can be reconstructed from the time differences between signals received at each diode [25]. The user can move around the room freely, with the only inconvenience being the cables attached to the headset. Furthermore, the HTC Vive can also track controllers to grab and manipulate objects in virtual environments, and can be precisely tracked and displayed inside the virtual environment as well. This allows for an immersive, interactive experience, where the limits are only the HMD cable extension and the corners of the delimited play area defined by the position of the lighthouses.

Our software pipeline uses Unity3D, a programming and execution environment [20]. This is a game engine, which allows us to implement ideas quickly and easily. Since it provides standard VR features such as graphics, physics (gravity and rigid-body modelling) and lighting, people without programming experience in these areas can also develop their ideas with minimal training involved. Whenever the standard features are not enough, Unity3D also allows to write scripts (in C#, in our case) to treat special cases or implement features that are not readily available. One of these examples is presented as Fig. 1. Here, we have written some code to implement protein animation by replacing the structure being presented multiple times per second. For interfacing with the VR equipment, we use the SteamVR abstraction layer. By using such a tool, it is possible to create a single project that works seamlessly on the two most popular HMDs (Oculus Rift and HTC Vive), without any code duplication. However, using such a tool for creating our applications brings downsides as well. Not every file format can be easily imported into Unity3D; specifically, importing 3D models with correct colours can be tricky. Since protein structures will be represented as 3D models, this requires a specific set up to allow us to visualise these structures inside Unity3D. Furthermore, the SteamVR environment is sometimes not easily accessible from inside centrally managed networks due to restrictions on gaming.

Fig. 1
figure 1

C# code for Unity3D, a part of the script written to animate proteins. There is a father object with multiple children objects containing the 3D models of consecutive frames of animation, and this script replaces the object being displayed multiple times per second

Full size image
Fig. 2
figure 2

Workflow diagram. Conformational changes for a protein PDB file are simulated, the snapshots are turned into 3D models by VMD and then imported into Unity3D, which builds the final application called ProteinViewer

Full size image
Fig. 3
figure 3

Standard secondary structure visualization using the “cartoon” option in Pymol. Different colors highlight either different domains (a, c) or chains (b, d) of the proteins, while the “stick” representation has been used to indicate ligands and nearby interacting side-chains for: a monomeric MurC with bound substrate UDP-N-acetylmuramoyl-l-alanine, non-hydrolyzable gamma-imino-ATP and two manganese atoms (PDB code 1P3D), b dimeric MraY in complex with tunicamycin (PDB code 5JNQ), c monomeric PBP1B in complex with TG domain inhibitor, moenomycin, and TP inhibitor, ampicillin (unpublished data) and d trimeric OmpF in complex with ampicillin (PDB code 4GCP)

Full size image

We present our current workflow for displaying protein structures in VR in Fig. 2. The initial inputs to our application are Protein Data Bank (PDB) [26] files. These are text files that describe the geometric structure of molecules. It allows for description of atomic coordinates, rotamers, secondary structures and connections between atoms. In our case, we are mainly concerned with the 3D position of each atom in the protein at each simulation time step; these positions need to be translated to an animated 3D protein structure.

Representing individual atoms in a protein does not indicate clearly 3D features such as twist and fold. Therefore, we will use the ribbon diagram representation to visualise secondary structures. These are representations where the polypeptide backbone is interpolated by a smooth curve, generating helices, sheets and loops. The result is a simple, yet informative diagram where 3D information about the protein can be easily displayed. Other representations such as the full “sphere” atom representation can of course be chosen as well if desired. We use the Visual Molecular Dynamics (VMD) program [8] for creating representations. The next step is generating 3D models for the ribbon representations. VMD allows us to create (Wavefront [27]) OBJ files; this is an open file format for 3D geometry, storing information about individual vertices, vertex normals, faces of polygons and textures associated with them. This way, we can export the 3D ribbon representations (and any other 3D structure, for that matter) from VMD while keeping any visual information such as colouring associated with it. Let us emphasize that upon exporting the structures from VMD, they become pure geometrical objects, and a user will not be able to query them for names of amino acids, ligands, hydrogen bonds, clashes, distances between atoms, etc. The user must therefore take care to label areas of interest in advance. We expect that future improvements in interfacing game engines and molecular visualizers will make interactive queries possible. Finally, it is necessary to import the generated OBJ files into Unity3D. Fortunately, Unity3D imports OBJ files natively, and integrates textures without any extra effort being necessary. Therefore, the VMD-generated structures can immediately become physical objects in the VR environment. Unity3D allows us to attach collider objects and rigid-body physics to the protein objects, i.e. giving them a virtual mass and spatial extent by integrating them seamlessly into the physics framework of the application. We find that this deepens a sense of physical reality for the user, which also translates in an enhanced, almost playful, engagement with the protein structures.

In addition, it is also possible to introduce dynamic structures into VR. We have used data generated by our coarse-graining molecular simulation method [28] in the form of multiple PDB files acting as snapshots of the protein conformation over time. Next, we load each individual PDB file into our pipeline, obtaining 3D models for each frame. We then use a script in Unity3D to flip through the models over time (cp. Fig. 2), creating an animation that shows the conformational changes calculated by our simulations.

Results and discussion

A “template” room was constructed for visualising proteins structures in Unity3D. This saves time when a quick visualisation is necessary by importing the relevant OBJ files into an existing “template” project where lighting and physics of the virtual room have previously been defined, and into which VR structures have already been hooked. Use of a “template” project reduces the tasks to simply importing OBJ files to create “father” objects for the 3D models and to define rigid body physics and collider boxes. If necessary, a script can be attached to that object for animation. Depending on the number of snapshots in PDB files, the process from simulation outputs to ready-to-use VR application can be as short as 5 min. A dataset containing all source code used in this project is available in a public repository for download [24].

The structural models used in this study as examples for the implementation of 3D visualization in VR are those of four bacterial proteins from different cellular compartments: cytoplasm, inner membrane, periplasm and outer membrane. MurC is a cytoplasmic ligase involved in the construction of the pentapeptide stem, which is a central component of the peptidoglycan (Fig. 3a). MraY is an integral cytoplasmic membrane enzyme that catalyzes the formation of lipid II by transferring the pentapeptide to the lipid carrier, which is another essential step in peptidoglycan biosynthesis (Fig. 3b). Penicillin-binding protein 1b (PBP1b) is a bifunctional enzyme containing both a transglycosilation (TG) and a transpeptidation (TP) domain, which is able to polymerize lipid II to form the peptidoglycan mesh (Fig. 3c). OmpF is an integral outer membrane channel exploited by most antimicrobial drugs to enter the organism on their way to the target (Fig. 3d). All structures are in complex with either the substrate or antimicrobial drugs that inhibit bacterial cell wall (peptidoglycan) synthesis. As such, they are currently studied targets for understanding AMR resistance mechanisms [29].

Intrinsic motions of these proteins were simulated by analysing protein flexibility, defining movement modes through elastic network modeling and generating conformers based on that information and steric interactions [28]. These simulations produced a series of PDB files detailing the state of protein structures at certain “snapshot” moments (e.g., every 100th conformer out of 5000 overall for each mode of motion). The output of these simulations are PDB files, each containing the protein atomic coordinations at a specific time step. The PDB format is not directly supported by the VR software Unity3D. Therefore, VMD was subsequently used to transform PDB files into OBJ files. Clearly, any other molecular dynamics package should also be able to result in the relevant OBJ files—perhaps after using a suitable “converter” such as VMD. Our specific choice of modelling [28] represents simply a generic test case scenario. We automated the process by writing TCL scripts [30] for VMD to examine output folders from the simulations and generate OBJ files of each conformer. An example of such script for a specific protein is presented as Fig. 4. Here, every PDB file for possible directions of protein movement gets loaded onto VMD, specific colouring and representation options for that protein are chosen and finally each frame is rendered and stored as an OBJ file [27]. Through this process, a series of OBJ files are obtained that act as 3D “frames” of animation. Upon importing these files into Unity3D, the only task left is to ensure that these frames are shown sequentially to impart upon the user the illusion of an animated protein. In this study we do so by updating the displayed object every 1/30 of a second. Though theoretically possible, it was decided against updating the collision structure of the object at each frame, as this would consume significant computational resources, even if it would allow for “correct” physics in VR at all times. Instead, we have chosen to define a bounding box spanning the whole range of movement for the protein as a constant collision structure.

Snapshots of the VR application being used and presented here are shown in Fig. 5a–d, where a user was interacting with the four proteins discussed above in a VR environment. Conveying a VR experience in pictures is a difficult task; readers with access to the HTC Vice or the Oculus Rift can download our preconfigured VR rooms from [24]. Figure 5a shows the template room containing the four protein structures. Some common physical objects such as cubes and balls have been added to emphasize the sense of “reality” and physicality around the protein models. We find that initial engagement with these familiar objects helps first-time users in their later interactions with the protein structures. This figure shows how is also possible to see both controllers being employed by the user in the VR room, with tool tips added for introducing new players to the controls. Figure 5b shows a snapshot of a protein structure being held by an user. In this case, the user is holding up dimeric MraY, with the same structure and coloring as in Fig. 3a. A close look to the drug and its interactions with protein residues can be taken as desired by “grabbing” the protein with the controller and moving it towards the viewer; tilting the controller allows to rotate the protein structure in an easy and intuitive way.

Fig. 4
figure 4

TCL code for VMD. We present the script written to generate multiple OBJ files in the correct order to be imported into Unity3D. We load all PDB files from specific folders, colour them accordingly for the specific protein structure at play and render them as OBJ files. The folders labelled with the “pos” and “neg” tags corresponds to possible closing and opening directions of protein movement

Full size image
Fig. 5
figure 5

a Template VR room containing the four protein structures discussed above. Both controllers are visible, with tool tips for teaching the controls. b MraY structure in VR. Details of specific residues and drug can be seen in both monomers. c Detail of PBP1b interactions with the drug. VR allows for close inspection of specific areas of a structure at any desired angle in an intuitive and straightforward way. d Close-up into MurC catalytic pocket containing the substrate, the cofactor and the two catalytic manganese ions. Areas deep inside a protein structure, i.e. away from its surfaces, are easily accessible through motion controls in VR

Full size image

Further detail on specific residues and drug-binding pockets can be seen by simply approaching the areas of interest, or through clipping the protein structure through the view point. Figure  5c shows a detail of ampicillin bound to the TP domain of PBP1b. This is the same region presented in light blue in Fig. 3d. The figure shows how much and how simple VR allows for in-depth analysis of specific areas of a protein structures. Finally, in Fig. 5d, we present a detail of an area of contact between MurC, the substrate and the cofactor, including the two catalytic manganese ions. In the VR environment, investigation of protein–ligand interactions is straightforward, since complete control of the 3D position and rotation of the protein model is mapped to movement of the controllers.

Conclusion

In this work, we present a way to visualise and interact with both static structure and dynamics of proteins by using VR. A software pipeline is outlined that enables non-expert researchers to straightforwardly embed protein structures into VR programs using a combination of widely available software, standard hardware and only few custom-built codes. The immersion and interactivity that VR brings can significantly change the level of details accessible to a researcher when analysing protein–ligand interactions or conformational changes. While it is difficult to convey this with static figures as in Fig. 5, it is interesting to report our personal experiences with immersive 3D VR. For example, despite having previously worked with these protein structures for a while and having observed the drug-bound catalytic pockets of these proteins for many times on 2D rendering softwares such as PyMol, it was still only after we observed the same in VR that we realized that the 3D configuration of the catalytic pocket was significantly different to the picture conveyed by the 2D projections and that we had built in our minds. These differences could translate into designing more effective modifications into the drug.

With the decreasing costs for customer VR hardware and an established workflow for importing structures into VR, immersive 3D visualization should become viable for an increasing number of research groups. It is a first step for developing VR-based ways of interacting with proteins and probing their properties. At the moment, there is still the limitation in both 2D and 3D protein visualization software packages that pre-computed conformers are necessary and the interaction between user and structure is only at a non-interactive visualisation level. In the future, physical interactions may become possible, where the user could bend the protein structure or add new functional groups to the ligand while background simulations calculate in real-time whether such changes are mechanically stable or energetically favorable. In such a scenario, immersive VR environments can aid much better than other 2D or 3D visualizations in guiding molecule manipulation.

As well as VR being a very useful tool for visualisation of both protein conformational changes and drug design, there is also the element of fantastic potential for outreach and engagement with the general public. Indeed, at our institution, the protein viewer VR project is now regularly showcased at such events.