# Journal of Robotics and Automation

ISSN: 2642-4312

Editor-in-chief

Dr. Xiao-Hua Yu
California Polytechnic State University,   USA

### Article Outline

Review Article | Volume 5 | Issue 1 | DOI: 10.36959/673/373 Open Access

# An Investigation of Haptic Modelling for Oral and Maxillofacial Surgical Training and Planning

Mutian Li, Susie Smart, Youhua Li, Ashraf Ayoub, Dimitris Drikakis and Xiu-Tian Yan

• Mutian Li 1*
• Susie Smart 1
• Youhua Li 1
• Ashraf Ayoub 1
• Dimitris Drikakis 1
• Xiu-Tian Yan 1*
• Space Mechatronic Systems Technology Laboratory (SMeSTech), Department of Design, Manufacture and Engineering Management, University of Strathclyde, Glasgow, UK

Li M, Smart S, Li Y, et al.(2021) An Investigation of Haptic Modelling for Oral and Maxillofacial Surgical Training and Planning. J Robotics Autom 5(1):227-235.

Accepted: May 04, 2021 | Published Online: May 06, 2021

# An Investigation of Haptic Modelling for Oral and Maxillofacial Surgical Training and Planning

## Abstract

This research investigates a haptic modelling approach where high resolutions are required for sensibility of force feedback in a target application - dental surgical operations. In particular the research focus is on maxillofacial deformity operations. The main aim of the research is to increase the realism of a computer model based simulation system that allows dental students and surgeons to feel like as if they were carrying out a real dental surgery procedure. A haptic framework has been designed and implemented to demonstrate its suitability to achieve the above aim. A generic set of jaw bone models have been developed and validated by collaborating surgeon from Glasgow University Dental Hospital & School. The model can be customized and obtained for each specific patient. A chosen model can be used to control the force feedback generated by a haptic device in order to give a realistic force feedback representation and experience for a user. This meets the requirements of the targeted dental operations. The simulation model is constructed based on force calculation model, which include the consideration of mechanical properties of bones, cutting tools used and region of cut. This haptical jaw bone model has been generalised based on a contact mechanic model Hertz's equation and is used as the driving haptical model for the study. Using a haptic device as a controller, a user can perform relevant operations, such as cutting procedure and manipulating bone segments from the virtual jaw bone model. The paper describes a frame work for a haptic assisted surgical plan (HASP) for surgical planning of dentofacial deformities. The haptic system generates the corresponding force feedback to a user as in a real world operation. It is planed that validation and feedback experiments will be conducted with dental students and surgeons to assess the effectiveness of the system in providing assistance for oral and maxillofacial surgical planning and training.

## Keywords

Haptic, Maxillofacial surgery, Dental modelling, Virtual reality simulator

## Introduction

Surgical procedures are well known to be complex and these vary from one patient to another. Training of student doctors to master these procedures is essential for high quality patient care and it can be even more challenging. A key factor which causes problems is the ability for a surgeon to estimate the correct amount force required to make a cut, especially for hard tissues, and then to implement with hand dexterity.

Haptic feedback can provide a force feedback sensation for a surgeon and hence it can play an important role in surgical planning operations and in training as well. There are a range of computer simulation based system developed for different medical surgical operations. These include pure computer based surgical simulation systems developed, but they can't provide true haptic feedback for a given surgical case [1]. A new development using haptic enabled virtual reality system has also been reported by Zaragoza-Siqueiros, et al. [2] and this work proposed an integrated haptic system for conventional orthognathic surgery planning as a feasibility study. Escobar-Castillejos, et al. [3] undertook a comprehensive review of recent work on medical simulators reported and a comparison is made of selected systems. This review was performed from training perspective and concluded that there are limitations in that few works are implementing guidance, and provide any aid during a session. There is only one paper reviewed covers maxilla and mandible and the work is for maxilla implantology only. It is however work on the intraoral welding technique using computer‐assisted surgery to obtain a precise passive fit of a prosthesis [4]. It is concluded there is a gap in literature on modelling for maxillofacial deformity surgical planning and training.

This paper presents a new haptic modelling in addressing the challenge faced in dealing with the hard tissues, in particular the maxillofacial deformity surgical procedures. A novel haptic model is proposed and validated by an expert in such operations. Next section briefly summaries a literature review of relevant work in haptic modelling.

Research investigations have been reported on the haptic feedback modelling or touch sensation in various medical applications.

A 3D planning system based on computed tomographic (CT) data was presented by Troulis, et al. [5]. In this work, a three-dimensional CT scan of the craniofacial skeleton forms the database. Software developed in the Harvard Surgical Planning Laboratory was modified for the craniofacial skeleton. A 'cutting tool' is used to segment the mandible and segments are moved to their predicted regions. A 'collision tool', alerts the operator of skeletal interferences. This gave the idea of mandible modelling. However, the concept of a haptic system was not yet introduced and there was no haptic force feedback in the system.

A method of building a cutting simulation using triangle mesh [6] was added along with a haptic device into the system. The tooth model was built by multilayer points scan using a laser scanner, and then reconstructed to form a triangle meshed model. A contact force model was proposed to describe the cutting process, while three different type of tools, the Physical Tool, the Virtual Tool, and the Cutting Tool, were introduced to perform the force computation and material removal.

In a force analysis in haptic simulation of bone machining Arbabtafti, et al. [7] introduced a force model for a bone milling tool with the calculated tangential cutting force. A voxelized force model was built for haptic simulation, including the consideration of size, spindle speed and rendering algorithm. Compared to a frictionless point based model that implements a spring force acting perpendicular to the surface of the bone, and the multipoint approach in which the spherical tool is represented by a number of sample points over the tool surface, more force analyses were considered. As a result, this approach makes their model computationally more expensive. Considered to be the low rate haptic rendering now, their stability doesn't meet the requirement for real-time force feedback and still need to be improved.

In 2012, another physics based model of bone machining was created [8]. This model considered impact mechanics to model the bone cutting process. Previous models had been created which used similar theories to model the cutting, including Agus, et al. [9] model. Wang, et al. [8] proposed that the speed of the drilling and sawing tools would be an important factor to consider when modelling the forces felt during the bone machining process. Haptic forces and torques generated on the motor driven tools were computed using impulse based dynamics. Using these values mathematical models for bone sawing, cutting and burring were generated. This use of impulse based dynamics for calculating the cutting forces as the tools rotated at high speeds is in contrast to Arbabtafti, et al. [7] model which considered spindle speeds but simply averaged the forces rather than applying the accurate forces felt for every tooth of the tool during its angular rotation.

A novel haptic model [10] was built that a user can use a virtual cutting tool to cut a thin slice of maxillofacial bone off. When the tool attaches the slice that need to be cut, the slice with the same thickness of the cutting tool can be dragged down until it no longer attaches the main part. This simulated the cutting process as a removing process of a targeted jaw bone operations. Although the realism of this model and the haptic feedback is less ideal than that from a triangle mesh and voxel model, it's far more computational efficient and has less time lag. As a result, the user experience has be improved. This model is however based on an experimental friction model and it is required further scientific improvement on its accuracy. Thus, this simulation method of cutting procedure is used for further investigation in this research.

One area where this model may lose some realism is the way in which the material properties of the bone have been modelled. Although mechanical properties have been considered, for example the cutting coefficient is calculated using Young's modulus and Poisson's Ratio, the values used for these properties are assumed and considered to be constant throughout a cutting section. In reality, these properties would vary throughout the various tissues within a bone and also in addition, they vary from one patient to another. Schwartz, et al. argued that, "The direction of maximum stiffness, cortical thickness, cortical density, and elastic properties for the dentate human mandible demonstrate unique regional variation" [11]. As this model considers the mandible to be one component with consistent Young's Modulus and Poisons Ratio, it is fundamentally unrealistic and inherently in accurate. This hence creates a less realistic force feedback for any simulation.

## Modelling of Bone Tissues and Their Contact with Cutting Tools

Schwartz-Dabney and Dechow [11] found that "the direction of maximum stiffness, cortical thickness, cortical density, and elastic properties for the dentate human mandible demonstrate unique regional variations. This study analyzed 10 adult human mandibles taken from cadavers. The material properties of the mandible were mapped into 31 regions as shown in Figure 1.

Tests were done to analyse material properties at each sample area shown in Figure 1. As it will be discussed later, the important material properties for this application are the Young's Modulus and Poisson's Ratio. Tables of these properties were provided in the original research paper Schwartz-Dabney and Dechow [11]. Some of relevant data from Schwartz-Dabney and Dechow's work are used in this study and listed in Table 1. It is assumed that within each region in the figure the mechanical properties of that region are identical as those regions are small enough to be different from its adjacent areas. This modelling of material properties throughout the mandible will be used further to attach material properties to a region within the mandible to calculate the forces felt when cutting through specific points within the bone.

## Building Force Model

### Impact model

There are two main types of physics based models of the cutting process. The first type of model has been developed for metal cutting and chip formation models and the other type that considers contact mechanics. The models which consider material removal in the form of chips, focuses more on the feed rate and cutting parameters rather than the work piece material properties. Cutting bone into segments is a different cutting process to metal cutting as bone is brittle and has different type of properties to metal which has large elastic deformation. Bone cutting is therefore considered to be a process of making correct contact and impact. The solution to this type of problems therefore utilizes impact mechanics to build a model of the cutting process which will take into account the variation in properties throughout the mandible. The contact mechanics model by Arbabtafti, et al. [7] include a consideration of the high speed cutting tools. For this research however the high speed aspect of the tools will be moderated to a slow speed to develop a first realistic model based on contact mechanics and also not to over complicate the model. This will allow a real-time computational performance in working out the haptic feedback to a user. This should not detract from the realism of the forces felt by the user as stated by Agus, et al. [9], "This is a minor limitation of the model because human tactile sensing is limited, except for very fine feature recognition tasks, to 400 Hz bandwidth".

To conduct the physics analysis during cutting procedure, there is a method called Hertz's equation that mainly studies the contact mechanics. Hertz's equation studies a contact relationship between any elastic bodies that can be used to find contact areas and indentation depths for simple geometries. Due to this features, Hertz's equation is therefore used to calculate the force in the impact model during the bone cutting processing. The contact between two spheres shown in Figure 2 based on Hertz equation, can be defined by equation 1 and 2 [12]:

The radius α of the circular area of the contact between the two circles is

where F is the impact force to be identified for the cutting process. Constant Kα is called cutting force constant and is defined as

Where E1, E2 are the elastic moduli and ${\upsilon }_{1}$, ${\upsilon }_{2}$ are the Poisson's ratios of each body. d1, d2 are diameters of spheres1 and 2.

This equation can further be applied to an additional case shown in Figure 3 whereby the circle representing a cutting tool is in contact with a planar surface representing mandible. This is a closer representation of the cutting process of a jaw bone.

In this case d2 = ∞, equation 2 is simplified to:

A diagram in the study [11] (Table 1) shows the elastic moduli and Poisson's ratios properties of each area on the mandible. Based on these values, elastic moduli E2 are the Poisson's ratios ${\upsilon }_{2}$ of the mandible can be derived from Table 1 to describe the selected cutting path. If a different cutting path has been selected, a new set of data table values should be derived and used instead (Figure 4).

### Cutting tool

In order to define exactly the behaviour of contact between the cutting tool and a hard tissue, the cutting tool must be defined to allow material properties of ${\upsilon }_{1}$ and E1 to be determined and also to assign a value of d1 for the cutting tool and calculate and estimate a value of α.

A rotating cutting tool manufactured from steel was selected for this study based on a tool used in surgery. Further work could be undertaken to look at incorporating other surgical instruments, for example chisel or drills, into the model. The rotating tools ranged in diameter and thickness. For the purposes of this model, a saw tool was selected which had a diameter of 30 mm and a thickness of 0.25 mm with a speed of 5000 to 10000 rpm from an available tool specification (Shofu Dental Corporation).

### Depth of a cut

Using the Schwartz-Dabney model, it is clear that the mechanical properties vary from the cutting selection plane as well as the depth of a cut. It is therefore important to define the depth of cut so that exact regions could be identified during the cutting process and correct material mechanical properties can be assigned to the cutting area in order to work out an accurate cutting force to drive the haptic device. For this initial feasibility study, it was decided that for this test model the average dimensions of thickness and depth could be used as the first attempt. However, the final haptic model should use real patient scanned data to allow patient specific dimensions to be added to ensure accurate force modelling. The average height of the body of the human mandible was found to be 3.2 cm [13]. A depth of cut of 3.2 cm was therefore selected for this test model.

## Force Calculation

Rearranging equation (1), the cutting force is expressed as in equation (4) and this is to be used to calculate the force experienced by a user when operating on a patient:

Based on the selected cutting region shown in Table 2, the cutting operation parameters of each region have different values of Yong's Modulus and Poisson's Ratio. Each region also has different depth of cutting. Applying these values in Table 2 for each region into equation (3) to work out the cutting force constant Kα, which is then used in equation (4) to work out a force value. The cutting force for each region along the cutting plane can be worked out in the same way and a result for one region 24 can be seen in Figure 5.

α value in equation (1) is determined by the number of tooth engaged during cutting and estimated small contact between the cutting tool tip with 1% of the cutter pitch size contact. Similarly, the cutting force for region 21, 13 and 6 can be calculated and results are shown in Table 3, Table 4 and Table 5 respectively. These results form a flexible haptic model for different region of cutting and the corresponding forces are applied to provide realistic force feedback for either planning or training.

## Mandible Modelling and Patient's Model Creation

With the above haptic model developed, it is possible to apply this generic model onto a patient mandible so that a haptic model can be generated for one to use to drive a visual three-dimensional model for either surgical planning or training. The following describes the process of creating a three-dimensional mandible geometric model, which can be used as the basis for applying the haptic model established in Section 5.

### Creation of the mandible model

To improve the responsiveness of the haptic system, it is proposed that a slice of bone is created from a mandible at the selected cutting plane and a friction model is created between this slice of bone tissue with its two contact surfaces of adjacent bone segments. In this way, a cutting depth dependent force model based on the force calculation results shown in Table 6 for the selected cutting plane is developed. By creating three bone segments, it is possible to create a clearer visual representation of the cutting process as the removed slice can still be seen by the operator during the cutting process. CT scanned mandible models can be obtained as a raw data for establishing the geometric representation of a patient's mandible. The first step to prepare for such a haptic interface is to import a CT scanned mandible model represented in Dicom format. This model can be obtained by scanning the skull of a specific patient, creating a medical Dicom format file.

The second step is that the Dicom file is translated to a STL file using AutoCAD software system. The geometry represented in SLT file format can be opened by most CAD software systems. In this study, a raw Dicom file was obtained from the Glasgow Dental School, and 3D Studio Max was used to translate the Dicom original model into a SLT file shown in Figure 6.

The STL file was extremely large and was made up of a very fine mesh. This mesh had to be reduced or it would have required too much computational power in the haptic system. Increase computational power requirement will in turn cause too much delays in a response to a demand of movement through a haptic interface. The Autodesk software 3ds Max was used to reduce the mesh as far as possible without losing the virtual appearance of the skull.

Thirdly, in order to reduce the requirement for a high computational power in a real-time haptic cutting and manipulation of the patient model, the mandible was then isolated by removing the rest of the skull as shown in Figure 7. This was done using 3D Studio Max.

The final step in establishing a haptic model for mandible cutting is to cut the slice off the mandible using cut tool in 3D Studio Max. This helps to create three bone segments prepared for haptic cutting and manipulation. These three parts are imported into a haptic system, which is Omni haptic system from Sensible. To study the different thickness of cutting tools, three thicknesses of the thin vertical strips with a size of 0.25 mm, 0.5 mm and 1 mm were created. These newly created parts can be saved as .obj files and they are readable to the haptic system and its working environment Open Haptics.

## Haptic Assisted Surgical Planner (HASP)

To support the haptic modelling and manipulation of mandible cutting and manipulation, a Haptic Assisted Surgical Planner (HASP) has been developed to facilitate the definition of a mandible cutting and manipulation model, run a simulated cutting process and provide a planning environment for users to plan and practice the mandible cutting procedure by using this haptic environment.

### Adding parts into haptic modelling space

An haptic interface was developed to allow objects to be added and dragged around in space and to build the desired simulation. In order to constrain the possible movement and rotation of these three mandible segment parts in three dimensional space, these mandible parts were added to the haptic modelling environment and were constrained to allow movement in they direction only. That is they can only move in the direction of the cutting plane to reflect the real possible motion. During the cutting process, it is important for the haptic model to be aware of the exact to location of the cutter. These constrains can be defined to suit difference cutting directions. The mandible segment parts could all be felt by a user haptically and could be slided up and down along the constrained direction.

In the C++ program these mandible segment parts are swapped for the three ".obj files". These objects are then translated and rotated to form the whole mandible.

The mandible is then visible on the screen and can be felt through haptic feedback. The parts are also constrained to only move up and down. The final step of the implemented procedure is to allow for only the middle section to be grasped by the cursor and moved (Figure 8).

### Building the force equation into the simulation

Based on the above geometric models, a haptic force model is now incorporated into the haptic interface. A haptic interface example is available that generates a point mass and added three different effects in the form of spring stiffness, dampening and friction.

This force model is therefore incorporated into the jaw program and the point mass is limited so that it is only felt when the middle section of the jaw is selected by the user. At this stage when the middle section is selected, friction forces between the middle slice plate and its adjacent mandible are produced and can be felt by the user.

The first task is to get the program to assess the cutting region of the user cursor and use this to calculate the depth of cut. The depth D can be obtained by getting current region of the cursor, which is the cutting tool, from a function called in Open haptics. And after calculating the force F, the corresponding value of force on the y direction can be output to the haptic device to make a user to feel the force feedback.

An experiment has been planned to undertake with dental students to assess the haptic feedback provided by the HASP system. Due to the space limit, this will be reported in a separate paper in detail. The preliminary experiments show that dental students found that HASP system does provide a better training tool for them.

## Discussion

There are a variety of applications where active haptic interfaces developed specifically for dental surgery have been seen to be of particular benefit to dental students, and oral & maxillofacial surgeons. Simulations can be used to practise procedures over and over again to allow dexterity skills to be developed and to allow a feel for an operation beforehand. The simulations can also be used to plan specific procedures to help surgeons assess the best way to carry out an operation for the specific patient's needs. The more realistic these simulations are the more useful they can be to the user as they become closer to being able to simulate the exact operational conditions of real life situations.

One main challenge in developing a realistic haptic system is that of the computational power required and can be relatively adequately allocated. Haptic simulation must have both the visual display and force feedback in perfect synchronization without any lag or the simulation will lose its realism. Research has been undertaken over the years to develop material removal systems that can incorporate enough detail in the model whilst reducing computational power. Arbabtafti, et al. [7] developed a Voxel based model to explore the accuracy of a detailed quantitative force model the same accuracy in the heterogeneous characteristics could not be achieved through surface based techniques. However, this detailed voxel model still caused a large computational power requirement as well as Wang, et al.'s [6] FEM model; This is easier in tooth machining as specific teeth can be modelled in detail whilst the rest of the jaw and teeth can be of low quality detail. This is more difficult with jaw machining and novel methods like Yan, et al. [10]' model should be adopted.

The main limitation of the study is that the mathematical model had to be scaled down to fit within the maximum force available from the haptic device. This meant that the actual forces generated from the mathematical model were not tested. However, the variation in these forces has been tested, and this study assessed this variation that was based on cutting down through the different parts of the jaw bone and through different types of material with different properties, against a model that had no variation at all. Due to challenge in obtaining ethical approval for measuring and testing the precise cutting force on a patient actual jawbone, it was not possible to validate the theoretical force models and results on patients. The consultant co-author however confirmed that the force generated is realistic and adequate for training and planning purposes.

The next recommendation therefore is that a prototype for a haptic device that provides a force of at least 28 N is built for future testing. This would allow for not only the variation in forces but the overall scale of the simulation to be assessed in a more valid manner.

## Conclusion

In this paper, a framework has been created to demonstrate how a hatpic system has been created to create a haptic model for a targeted application in the surgical correction of maxillofacial deformities including both planning and training. This provides a useful method to simulate corrective surgery of jaw deformity using haptic technology is presented. The haptic model hence can be used to plan the surgical procedure by a surgeon, and equally it can be used to train next generation of junior trainees who currently don't have enough time to achieve a satisfactory training on a wide range of surgical procedures.

A digital composite skull model, which reproduces both the bony structures and the dentition with a high degree of accuracy, was created by translating Dicom slice images through the CT scans into an new model in STL file. In the next step a mathematical model has been developed and incorporated into a haptic simulation of dental deformity surgery so that a force feedback sensing can be created for one to experience a force during the interaction with the CAD model. Finally, research techniques are used to identify the required values for the mathematical model and to gain an understanding of the surgical procedure. The above work has proven to be effective in supporting the surgeons during the prediction planning phase and provides a platform for training.

Further work will be undertaken to assess the effectiveness of the haptic model and its simulation in training future oral and maxillofacial surgeons. Further improvement will be make in increasing accuracy of force representation and scale of the forces, which also need further validation.

## Abstract

This research investigates a haptic modelling approach where high resolutions are required for sensibility of force feedback in a target application - dental surgical operations. In particular the research focus is on maxillofacial deformity operations. The main aim of the research is to increase the realism of a computer model based simulation system that allows dental students and surgeons to feel like as if they were carrying out a real dental surgery procedure. A haptic framework has been designed and implemented to demonstrate its suitability to achieve the above aim. A generic set of jaw bone models have been developed and validated by collaborating surgeon from Glasgow University Dental Hospital & School. The model can be customized and obtained for each specific patient. A chosen model can be used to control the force feedback generated by a haptic device in order to give a realistic force feedback representation and experience for a user. This meets the requirements of the targeted dental operations. The simulation model is constructed based on force calculation model, which include the consideration of mechanical properties of bones, cutting tools used and region of cut. This haptical jaw bone model has been generalised based on a contact mechanic model Hertz's equation and is used as the driving haptical model for the study. Using a haptic device as a controller, a user can perform relevant operations, such as cutting procedure and manipulating bone segments from the virtual jaw bone model. The paper describes a frame work for a haptic assisted surgical plan (HASP) for surgical planning of dentofacial deformities. The haptic system generates the corresponding force feedback to a user as in a real world operation. It is planed that validation and feedback experiments will be conducted with dental students and surgeons to assess the effectiveness of the system in providing assistance for oral and maxillofacial surgical planning and training.