Computer Aided Surgery
¨ Class:
I 1271 Bio-Informatics I
¨ Professor:
Dr. D.B.Karron
¨ Team
Members:
§ Qun Chen
§
Guoliang
Qian
§
Chuan
Yu
§
Miki
Yamaguchi
§
Dongyun
Wu
Ì
Contents
Abstract
1
Introduction
2
Surgical planning
2.1 Goals
2.2
Project Description
3
Surgical Navigation
3.1
Surgical
Navigation System
3.2
Tracking
Signal Instrument
3.3
Endoscopic
Navigation
3.4
Training
System
4 Treatment
4.1 Monitoring Technology for Patients of Chronic
Disease
4.2 Some examples of the patient monitoring system for chronic
illness
4.2.1 Cardiac Monitoring System
4.2.2 Asthma Monitoring System
4.2.3 Diabetes
Monitoring System
4.3 Discussion
5
Virtual Reality in Surgery
5.1
VR for Surgery
5.2 Image-guided Surgery
5.3 Education and Training
5.4 Preoperative Planning
6 Human
Interfaces in Surgery
7
Conclusion
Bibliography
Illustrations
1
Figure
2.1
Medical Image Processing Laboratory
2
Figure
2.2 FRACAS: A
System for Computer-Aided Image-Guided Long Bone Fracture Surgery
3
Figure
3.1 Surgical
Navigation System in Operating Room
4
Figure
3.2 A surgeon is
practicing
5
Figure 3.3
Dissection of a vessel
6
Figure
4.1
VisiTran5.0 HeartStation screenshot (Image reproduction from
MedVision,Inc)
7
Figure
4.2 AirWatch(Image
reproduction from LifeChart)
8
Figure
4.3 One Touch II
Blood Glucose Meter(Image reproduction from LifeChart)
9
Figure
5.1 Open surgery
10
Figure
5.2 Elekta's
Gammaknife(left ) and the X-knife from Radionics (right)
11
Figure
5.3 Image-guided
surgery, implemented as Augmented Reality
12
Figure
5.4 Endoscopic
Surgery Trainer
13
Figure
5.5 Endoscopic
Surgical Simulator
14
Figure
5.6 The EVL eye
used by a group in the CAVE
15
Figure
5.7 JHU/KRDL
Skull-base Surgery Simulator
16
Figure
5.8 Stereotactic
frame-based
17
Figure
5.9 Combined
neurosurgery
18
Figure
5.10 Used in
pre-operative planning
Abstract
In these days, we can find a lot of achievement of new surgery
using technique such as robotics, optics or virtual reality. Each of them
are called separately, "medical/surgical robotics" or "VR
for medicine". All of these new techniques and Computer Aided Surgery
share the same motivation, and researchers need to exchange their
knowledge each other. CAS can be categorized as surgical planning,
surgical navigation, treatment, education and some other techniques that
are difficult to categorize. None of medical imagings is an all-round. For
adequate diagnoses and treatments, more than one imaging modality are
needed. Especially in surgery, modalities for intraoperative use are quite
limited: surgical microscope, endoscope, ultrasound imaging and surgeons'
eyes. These images should be compared and corresponded with preoperative
images.
Image integration is a quantitative and precise method of the
corresponding of multimodal images, mostly using computers.
Image integration is also called image registration. Endoscopic surgeries
have been widely spread in the world. However, something remains to be
solved. Surgeons' field of view in endoscopic view is quite limited and
they tend to wonder where they are actually seeing. Difficulty in
unexpected phenomena as bleeding is also pointed out. Visual simulation of
endoscopic view, position measurement of endoscope and endoscope guiding
robot have possibility to solve these problems. VR is one of the hottest
technological topics. CAS is among the large fields of VR application.
Many achievements in VR research as stereoscopic display, force-feedback
or human-machine interface are quite important for CAS research.
Introduction
CAS
is a term for the research field concerning "advanced surgical
techniques, and technology that creates them". Initially, CAS meant a
technology of surgical simulation using three-dimensional organ models
reconstructed medical imaging, by computer graphics technique.
The aims of Computer Aided
Surgery are to advance the utilization of computers in the
administration of treatment to patients, to evaluate the benefits and
risks associated with the integration of advanced digital technologies
into surgical practice; to provide a means to disseminate clinical and
basic research relevant to stereotactic surgery, minimal access surgery,
endoscopy, and surgical robotics; to encourage interdisciplinary
collaboration between engineers and physicians in developing new concepts
and applications; to educate clinicians about the principles and
techniques of computer assisted surgery and therapeutics; and to serve the
international scientific community as a medium for the transfer of new
information relating to theory, research, and practice in biomedical
imaging and the surgical specialties.
The scope of Computer
Aided Surgery encompasses all fields within surgery, as
well as biomedical imaging and instrumentation, and digital technology
employed as an adjunct to imaging in diagnosis, therapeutics, and surgery.
Topics featured include frameless as well as conventional stereotaxic
procedures, surgery guided by ultrasound, image guided focal irradiation,
robotic surgery, and other therapeutic interventions that are performed
with the use of digital imaging technology.
CAS can be categorized as surgical planning, surgical navigation,
treatment and education. Surgical planning may be divided into medical
image processing and visualization such as CT, MRI, ultrasound or
angiography, modeling - for example, CT and angiography, image
registration is included, and surgical simulation using medical image
information for the optimization of surgical procedure. Simulation of
intraoperative imaging (endoscope or US) is included. Surgical navigation
is a kind of Technology related to the precise guidance of surgical tools.
Measurement of the position of a surgical tool is a good example. Image
registration between pre- and intraoperative images, and organ deformation
analysis, are complementary but important. Intraoperative imaging devices
enable the acquisition of images of a patient in surgery. Endoscope,
ultrasonography, interventional CT/MRI and surgical stereoscopic display
are good examples. Treatment technology is a new method of treatment or
surgical tools. Surgical laser, microsurgery robotics etc. It relates to
safe and accurate guidance of a surgical tool toward a lesion.
Stereotactic surgery is its original form. Surgical robotics is included.
This type of technology shares many features with surgical navigation.
Training for surgical planning supports training of decision-making in
surgery and surgical skill. An example is a puncture simulation using
force feedback. In addition,
CAS contains some techniques that are difficult to categorize. Human
interface is an example. From the surgeon's view, an intraoperative
imaging device, guidance device or treatment device is a kind of interface
to access his/her patient. Their usefulness and safety from the point of
human-interface should be evaluated.
2 Surgical
Planning
Figure 2.1
Medical Image Processing Laboratory
2.1 Goals
The goal of the laboratory is to develop innovative computer-based
methods for assisting surgeons in the planning, execution, and evaluation
of surgical procedures based on medical images.
Recent worldwide clinical trends point towards precise, minimally invasive surgery as the method of choice in many surgeries. Coupled with new medical imaging and computer technology, they are beginning to show the potential for better clinical results, reduced morbidity, shorter recovery and hospital stay times, and lower costs. To fully benefit from this technology, new algorithms and computer-based systems must be developed.
They follow a synergistic methodology, which consists of developing
basic building blocks for computer-aided surgery while simultaneously
developing solutions for specific clinical applications. They conduct this
interdisciplinary research in collaboration with hospitals, academia, and
industry.
Their main focus is on computer-aided orthopaedic surgery and related applications. Specifically, They are developing techniques for preoperative planning and visualization, fluoroscopic X-ray image processing, anatomy-based registration, and image-guided navigation. They are integrating these techniques into FRACAS, a system that helps surgeons in performing femur fracture reduction surgeries. They have also started a new project, in collaboration with the Technion for image guided robot for precise minimally invasive surgery.
Figure2.2 FRACAS: A System for Computer-Aided Image-Guided Long Bone Fracture Surgery
2.2 Project
Description
They are currently developing a computer-integrated system, called
FRACAS, for assisting surgeons in closed reduction of long bone fractures.
Fluoroscopy-based orthopaedic procedures crucially depend on the ability
of the surgeon to mentally recreate the spatio-temporal intraoperative
situation from uncorrelated, two-dimensional fluoroscopic X-ray images.
Significant skill, time, and frequent use of the fluoroscope are required,
leading to positioning errors and complications in a non-negligible number
of cases, and to significant cumulative radiation exposure of the surgeon.
Recent research shows that computer-aided systems can significantly
improve the accuracy of orthopaedic procedures by replacing fluoroscopic
guidance with interactive display of 3D bone models created from
preoperative CT studies and tracked in real time. Examples include systems
for acetabular cup placement, total knee arthroplasty planning and total
knee replacement, and systems pedicle screw insertion.
FRACAS' goals are to reduce the surgeon's cumulative exposure to
radiation and improve the positioning and navigation accuracy, and to
improve preoperative planning. FRACAS replaces uncorrelated static
fluoroscopic images with a virtual reality display of three-dimensional
bone models created from preoperative Computerized Tomography CT data and
tracked intraoperatively in real-time. Fluoroscopic images are used for
registration -- establishing a common reference frame -- between the bone
models and the intraoperative situation, and to verify that the
registration is maintained.
They have implemented a complete prototype of the system and integrated it with a commercial tracking device. The prototype includes modules for modeling, preoperative planning, visualization, fluoroscopic image processing, registration, and calibration. The system builds 3D geometric models of the healthy and broken bones from a sequence of 2D images obtained before surgery by CT. Using the visualization module, the surgeon interactively examines the bone models, identifies the characteristics of the fracture, and determines the upper and lower bone fragments to be joined by the nail. The planning system allows the surgeon to determine the optimal length and diameter of the nail by interactively positioning a nail CAD model chosen from a catalog inside the healthy bone model. The bone fragment models are used to visualize their relative position during surgery and match the fluorosopic images. The fluoroscopic image processing module corrects the image distortion, computes the fluroscopic C-arm camera parameters, and extracts the bone contours that will be matched to the 3D bone frament models. The registration module establishes the correspondence between the model and patient reference frame.
Their current work focuses on registering the extracted bone
contours with surface bone models obtained from preoperative CT images and
(2D/3D anatomy based registration), accuracy experiments, and in-vitro
prototype testing. They are also considering closely related clinical
applications including intramedulary nailing of the tibia and humerus.
While the system is targeted to closed intramedulary nailing, many of its
components can be used for other orthopaedic procedures.
3 Surgical Navigation
Traditional open surgical techniques are being replaced by new
technology in which a small incision is made and a rigid or flexible
endoscope is inserted, enabling internal video imaging.
With endoscopic surgery,
surgeons can examine the interior of the body with more detail that in
turn leads to higher accuracy in diagnosis and surgical operation. As the
endoscope is inserted usually through a natural body opening or small
incision, the patients will experience less painful period and
post-surgical recovery is expected to be much shorter.
Various instruments are used in different situations among which
fiber-optic endoscope is a pliable, highly maneuverable instrument that
allows access to channels in the digestive tract that were previously
inaccessible. Composed of multiple hair like glass rods bundled together,
this instrument can be more easily bent and twisted, and the intense light
enables the endoscopist to see around corners as well as forward and
backward. Accessories can be added that make it possible to obtain cell
and tissue samples, excise polyps and small tumors, and remove foreign
objects. The surgical navigation process relies heavily on the analysis
and visualization of the internal structures that is accomplished through
an automatic system that reconstructs the 3D live image of the internal
anatomy.
Well-designed Surgical Navigation system has been used to serve in
many medical cases such as soft tissue surgery, orthopedic surgery,
radiation oncology, neurosurgical planning and so on.
3.1
Surgical Navigation System
Figure 3.1 Surgical Navigation System in Operating Room
A surgical navigation system has been built that is currently used
regularly for neurosurgical cases such as tumor resection at Brigham and
Women's Hospital. The system consists of a portable cart containing a Sun
UltraSPARC workstation and the hardware to drive the laser scanner and
Flashpoint tracking system (Image Guided Technologies, Boulder, CO). On
top of the cart is mounted an articulated extendible arm to which a bar is
attached to house the laser scanner and Flashpoint cameras. The three
linear Flashpoint cameras are inside the bar. The laser is attached to one
end of the bar, and a video camera to the other. The joint between the arm
and scanning bar has three degrees-of-freedom to allow easy placement of
the bar in desired configurations. The figure below shows the cart set up
in the operating room.
3.2 Tracking Signal Instrument
The medical instrument is tracked by attaching a stimulator. This
stimulator is used to determine the location of vital regions of the
brain, including motor and sensory corticies and language area. When the
stimulator is placed on motor cortex, a muscle response occurs, and when
placed on sensory cortex, sensation in different areas is reported.
Language suppression (including temporary loss of speech) occurs when the
stimulator touches the languages area. As the neurosurgeon stimulates
different areas of the brain and receives responses, it is common for him
to place numbered markers on the cortex highlighting regions to avoid.
When our probe is attached to the stimulator, we can obtain the position
of the tip during stimulations and immediately produce a color-coded
visualization highlighting these important areas.
3.3
Endoscopic Navigation
Virtual endoscopy is the navigation of a virtual camera through a
3D reconstruction of a patient's anatomy enabling the exploration of the
internal structures to assist in surgical planning. Augmented endoscopy
includes the registration of the patient and tracking of the endoscope to
produce a visualization illustrating the position of the endoscope in the
MR scan during a procedure.
An augmented endoscopy guided intubation was
performed on a 55-year-old male with a retropharyngeal tumor using our
system. The patient was registered to the MR scan and tracked. A flexible
endoscope and a miniature (1.2 mm diameter) electromagnetic sensor (Biosense
Inc. Seatauket, NY) were introduced into the endotracheal tube together.
During the intubation, a registered visualization of the endoscope's
position was displayed. The image to the left illustrates an example of
the display during the procedure. The rendered image in the upper left
corner represents the virtual view corresponding to the endoscope's
position and orientation. An exterior virtual view is shown in the upper
right. The three orthogonal slices of the MR scan are shown along the
bottom, with the cross hairs highlighting the position of the endoscope.
These views are updated at about 1-2 frames per second, demonstrating the
progress of the endotracheal tube.
3.4 Training System
Although it has many advantages over traditional surgery, as a new
technique endosurgery is more complicated than traditional approaches,
which makes efficient training an indispensable process. The “Karlsruhe
Endoscopic Surgery Trainer” is a 'Virtual Reality` based Training System
for Minimally Invasive Surgery currently used in German to support the
teaching and training of the operators with a computer-based simulation
system which imitates the operation area and provides a real time
“synthetic” endoscopic view.
The
system can imitate roughly the outward and structure of the organ that
being operated. With electromechanical instrument guidance and tracking
systems, the trainer allows the trainee surgeon to manipulate the
instruments in the usual way. Central
unit is a high-performance graphics workstation with the
simulation system 'KISMET' used as core
software. KISMET does all the necessary calculations and generates the
virtual endoscopic view in real time. For modeling and realistic
simulation, a
model - database is
required which defines the geometrical shapes and the
physical/mechanical properties of the tissues, organs and vessels as well
as the geometry and kinematics of the instruments. A knowledge base
specifies the interaction behavior and the handling of the model manipulation. Of great importance is the
realistic imitation of soft tissue with its physical behavior, which leads
to 'deformable objects'.
The relevant operation area in the training scenario is modeled by a
coupled system of deformable objects. Another critical subject is the
realistic simulation of the interaction between deformable objects and
instruments and the manipulation of the virtual tissues. So far several
typical surgical tasks have been
implemented such as grasping, cutting, coagulating and setting of clips.
The calculation and representation of realistic tissue deformation and
manipulation is done in real time.
4 Treatment
4.1
Monitoring Technology
for Patients of Chronic Disease
The health care sector is gradually entering the
field of information technologies (IT) as a trend of overall technologies
is getting smaller, cheaper, and easier to integrate. Imagine the
hospitals of the 21st century. The IT components used in such
hospitals will be far beyond the present level of scale and integration. The
integrated IT will be deemed
especially useful for
monitoring patients of chronic disease
This report overview currently available patient monitoring systems
built for chronic illnesses. The cases of asthma, diabetes, and
heart disease are introduced and the effectiveness of each system is
discussed. Furthermore, overviews the current and future technology
solutions to build patient monitoring systems will be discussed.
Over the past several years, developments in computer
technology have allowed the use of devices to monitor the patient’s
conditions outside the physician’s office. Medical conditions of
remotely located patients are monitored, diagnosed and treated using a
central date processing system (in clinician side) to communicate with and
receive data from patient monitoring systems.
Each patient monitoring system is capable of collecting patient
data relating to the patient’s health condition. A central data
processing system obtains patient data from each patient monitoring system
and analyzes the obtained patient data to identify medical conditions of
each patient.
4.2: Some examples of the patient monitoring system for chronic illness
4.2.1:Cardiac Monitoring System
Heart
disease cost the nation almost $260 billion in 1998, according to the
American Heart Association. Many expert believe that the use of the
cardiac monitoring system helps patients take better care of themselves
and can be less expensive than medical treatment. There are several
cardiac monitoring systems available for the patient.
a. HERTrec: the HEARTrec cardiac monitoring system from Cardiac
Telecom Corp. is a small 24-hour patient monitoring and allows physicians
to remotely access and analyze stored data.
b. VisiTran: the VisiTran cardiac monitoring system from MedVision, Inc. is Windows-based software for transmitting patient information and other imaged cardiac data.
Figure 4.2 VisiTran5.0 HeartStation screenshot (Image reproduction from MedVision,Inc)
4.2.2:Asthma
Monitoring System
About 14 million
Americans suffer from asthma, according to the American Academy of
Allergy, Asthma & Immunology. The AirWatch(Figure 4.2.2) digital
asthma monitor developed by LifeChart
measures lung constriction and transmits that information via a phone line
to the LifeChart network. The patient's data can be accessed by doctors or
pharmacists through an Internet browser. The monitoring system help the
chronic asthma patient better manage their condition. According to the
survey, using the company's technology, a group of 3,000 asthma sufferers
reduced their collective emergency room visits from 84 to just one because
doctors examine the Web status reports and let patients know when or if
they should increase their medications to stave off an attack.
Figure 4.2 AirWatch(Image reproduction from LifeChart)
4.2.3: Diabetes
Monitoring System
Diabetes
is the seventh leading cause of death by disease in the United States
according to the National Center for Health statistics. For many diabetes
patients, self-monitoring blood glucose levels on a regular basis plays a
critical role in maintaining a
health lifestyle.
However, only
one third
of the pain diabetics
patient test their
blood glucose regularly because of and inconvenient. The One Touch II
Blood Glucose Meter(Figure 4) developed by LifeChart, Inc. is a convenient
and easy way to monitor blood glucose levels. It transmits readings from
the patients to LifeChart network and converts patient glucose data into
graphs.
Figure 4.3 One Touch II Blood Glucose Meter(Image reproduction from LifeChart)
4.3 Discussion
Home monitoring systems have been providing un-measurably large amount of aids with the people in chronic conditions. Recent studies show that home monitoring systems can replace a large portion of office visits required for chronic patients. These people would now be able to maintain more productive lives by effectively managing their chronic situations. A rapid progress of the computer technology is one of the driving factors that accelerate the advent of various home monitoring systems. For instance, the above reviewed devices are designed for daily self-monitoring for people with chronic conditions.
These systems are simple enough to install, utilize, and maintain (even for children), and only a little supervisions are required in a regular basis. One drawback is that the companies or hospitals are providing distinctive home monitoring systems as their sales products. The disease conditions can be monitored for the patients who subscribed these services, and thus, only such data can be used for the state-of-the-arts research work. Supporting new treatment studies usually requires a large number of disease cases.
The advantage of using these monitoring systems should further be strengthened by 'systems globalization,' that is, collecting and sharing larger volume of patient information and disease data which may span multiple communities, multiple regions, and multiple countries. A more globalized home monitoring system will be expected to participate in our daily lives similar to the essential utilities (such as electricity, gas, phone line, cable TV, and mobile devices). The system envisioned will significantly help the medical expert to apply analysis and derive meaningful information from the ubiquitous disease cases collected from the patients.
It is
likely that the deployment of the globalized monitoring systems yields
more opportunities and potentials for the development of a new medical
treatment. For instance, a point of service medicine (POS medicine), as
illustrated in Figure 4.5, is one of the highly integrated medical monitoring systems.
The patient side of the POS medicine consists of a personalized
software system and medical devices that are combined with a local
database, capable of identifying the medical conditions of the patient
based on the various biometric data measured by the devices. A summary
data will be continuously transmitted to a central database system that is
maintained by a POS administrating site.
Abilities to monitor the medical cases for a large number patients
in real time will make it for the POS administrators and medical experts
possible to develop a new treatment methodology.
The developer team in the POS administration site will upgrade the
software systems. Accordingly,
the patient will automatically receive a new monitoring software system.
However, several system’s technical issues need to be addressed
as the number of the participants of the POS service is expected to
increase. In particular, the
POS system may need to support tens of millions of individuals, in which
case a massive amount of data transmissions and probably a tera byte level
data storage will result in a significant overheat to the system.
The technical challenges are (1) the way to establish a scaleable
system configuration such that the system should perform stably in online
real time even the number of transactions drastically increases, and (2)
the system needs to be fail safe in that the POS service should be fault
tolerant even communication
errors and software failures may frequently occur.
These are open issues, and today’s
progressive development of computer science field must be considered and
incorporated to solve these challenges.
Figure 4.4
4
Virtual Reality in Surgery
VR is
being applied to a wide range of medical areas, including remote and local
surgery, surgery planning, medical education and training, treatment of
phobias and other causes of psychological distress, skill training, and
pain reduction. It is also used for the visualization of large-scale
medical records, and in the architectural planning of medical facilities,
although these last two applications are not covered by this survey. The
survey focuses on three main application areas: surgery in general,
neurosurgery, and mental and physical health and rehabilitation.
5.1 VR for
Surgery
Surgery is mostly visual and manual. VR for surgery involves applications of interactive computer technologies to help perform, plan and simulate surgical procedures. In performance, the VR guides the surgeon, sometimes with a robot to execute the procedure under the surgeon's control (to remove hand tremor and scale down manipulations for key-hole surgery, for example). In other words, VR is used to give the surgeon 3D interactive views of areas within the patient. Planning is carried out preoperatively, to find the best approach to surgery, involving minimum damage. Simulation is mostly used in training, using patient data often registered with anatomical information from an atlas. It may be used for routine training, or to focus on particularly difficult cases and new surgical techniques.VR is being applied in all three major areas of surgery: open surgery, endoscopic surgery and radiosurgery. The surgery may be remote (through the use of robotics) or local.In open surgery, the surgeon opens the body and uses hands and instruments to operate. This is the most invasive form of surgery, with long recovery times. There is a strong movement away from open surgery and towards improved techniques of minimally-invasive surgery.
Figure5.1
open surgery
Endoscopy is minimally invasive surgery through
natural body openings or small artificial incisions ('keyhole surgery'):
laparoscopy, thoracoscopy, arthroscopy, and so on. A small endoscopic
camera is used in combination with several long, thin, rigid instruments.
The trend is to carry our as much surgery as is feasible by this means, to
minimise the risk to patients. Advantages
for the patient
include less
pain, and less strain
on he organism, and faster recovery.
There are also relatively small injuries, and an economic gain
arising through shorter illness time. However,
for the surgeon, there are several disadvantages, including restricted
vision and mobility, difficult handling of the instruments, difficult
hand-eye coordination and no tactile perception except force feedback.
Endoscopic surgery is becoming increasingly popular, because of its
significant advantages. It is also the most popular surgical application
of VR, partly because it expands on what is already an
"unnatural" view of the locus of operation. Another reason is
that endoscopic surgery is relatively easy to simulate because of the
limited access, restricted feedback (especially tactile) and limited
freedom of movement of instruments. Endoscopic simulators are being
produced by all the main medical VR companies, usually with a focus on
training. Another recent trend is towards so-called Virtual Endoscopy.
This is a technique whereby data from non-intrusive sources - such as
scans - are combined into a virtual data model that can be explored by the
surgeon as if an endoscope were inserted in the patient. VR is
increasingly being used to provide surgeons with a meaningful and
interactive 3D view of areas and structures they would otherwise be unable
or unwilling to deal with directly. In radiosurgery, X-ray beams from a
Linear Accelerator are finely collimated and accurately aimed at a lesion.
Popular products include Radionics X-knife, and Elekta`s Gammaknife.
Planning radiosurgery is suitable for VR, since it involves detailed
understanding of 3D structure.
Figure
5.2 Elekta's Gammaknife(left ) and the X-knife from Radionics (right)
VR
in surgery differs from most other VR in its focus on contact with
objects, which must often be deformable objects and interdependent. The
focus is on looking into objects rather than looking into space - there is
less room available. The data is essentially volumetric and finger and
hand interaction must be extremely precise. The above characteristics
bring with them certain technical requirements, such as real-time response
to user`s action - which implies fast graphics, low latency input devices.
The images must be of high resolution and faithful to the actual patient
data, since life-critical decisions are based on the presentation of
patient data. For simulators, the physical procedures must match those
used in the actual operation. Other requirements of VR for surgery include
registration of patient data with atlases and the ability to coregister
multimodal data. For use over extended periods, which is often needed in
surgery, the style of user interaction should be natural, comfortable, and
easy to use.
5.2
Image-guided Surgery
VR can in principle be applied to enhance reality for
image-guided surgery. When applied to image-guided surgery in this way,
the images obviously need to be available intra-operatively, and accurate
registration of the real patient with the data becomes a crucial issue.
Currently, VR is used much more for preoperative planning than to guide
actual surgery (due to the understandable conservatism of medical
practitioners). When VR is used intra-operatively, it tends to be
implemented as some form of Augmented Reality. Image-guided surgery is
also a prerequisite of remote telemedicine and collaboration.
Figure 5.3 Image-guided surgery, implemented as Augmented Reality
5.3 Education
and Training
VR provides a unique resource for education about anatomical structure. One of the main problems for medical education in general is to provide a realistic sense of the inter-relation of anatomical structures in 3D space. With VR, the learner can repeatedly explore the structures of interest, take them apart, put them together, view them from almost any perspective. This is obviously impossible with a live patient, and is economically infeasible with cadavers (which, in any case, have already lost many of the important characteristics of live tissue).Another advantage of VR for medical education is that demonstrations and exercises or explorations can easily be combined. For example, a "canned" tour of a particular structure, perhaps with voice annotations from an expert, can be used to provide an overview. The learner may then explore the structure freely and, perhaps later, be assigned the task of locating particular aspects of this structure. It is also possible to preserve particularly instructive cases, which would be impossible by other means. There is something of crisis in current surgical training. As the techniques become more complicated, and more surgeons require longer training, fewer opportunities for such training exist. Training in the operating theatre itself brings increased risk to the patient and longer operations. New surgical procedures require training by other doctors, who are usually busy with their own clinical work. It is difficult to train physicians in rural areas in new procedures. Training opportunities for surgeons are on a case-by-case basis. Animal experiments are expensive, and of course the anatomy is different. The solution to these problems is seen to be the development of VR training simulators. These allow the surgeon to practice difficult procedures under computer control. The usual analogy made is with flight simulators, where trainee pilots gain many hours of experience before moving on to practice in a real cockpit. The advantages of training simulators are obvious. Training can be done anytime and anywhere the equipment is available. They make possible the reduction of operative risks associated with the use of new techniques, reducing surgical morbidity and mortality.However, the big challenge is to simulate with sufficient fidelity for skills to be transferred from performing with the simulation to performing surgery on patients. Faithfulness is hard to achieve and much more evaluation of different approaches to training simulation are needed. Many experienced surgeons predict that in time, experience with training simulators will constitute a component of medical certification. But this will require new regulations and legislation. Hot topics in the area include the use of force feedback increased accuracy of modelling of soft tissue, and the role of auditory feedback. For simple operations like suturing and biopsy needle placement, VR is effective, but perhaps an overkill to train skills that can easily and cheaply be acquired in other ways. The most useful and tractable areas for the development of training simulators are the various techniques of endoscopic surgery in widespread use today. It is relatively easy to reproduce in VR the restricted field of view and limited tactile feedback of endoscopic surgery. It is much more problematic to reproduce open surgery techniques realistically.
For complex anatomical structures, this is definitely not yet possible.
Figure5.4 Endoscopic Surgery Trainer
The
pictures above illustrate both the value of simulators for training
procedures, but also their current weaknesses in terms of realism. To
realistically simulate an operation, the method of interaction should be
the same as in the real case (as with flight simulators). When this is not
the case, the VR can serve as an anatomy educational system rather
than a training simulation. One
way of increasing the reality of interaction
Figure 5.5 Endoscopic Surgical Simulator
is to combine VR with physical models, as illustrated in the Gatech simulators for endoscopy and eye surgery, and the Penn State University bronchoscopy simulator (see below). These systems focus on training the surgeon in the use of particular medical devices, rather than on training a better awareness of general or specific patient anatomy.
An example of an anatomy educational system is the EVL eye (shown
below) from the University of Illinois. Since the VR is immersive and
based around the CAVE, it cannot be said to duplicate the interaction
methods of real eye surgery (since surgeons cannot get physically inside
eyes) and so is not a training simulator, unlike the Georgia Tech system
above.
Figure 5.6 The EVL eye used by a group in the CAVE
More realistically in terms of interaction, the Responsive Workbench is another candidate for anatomy teaching (see below). As with CAVE-based applications, a shared VR enhances the potential for collaborative learning. The most technologically challenging area of simulator training is for highly specialized aspects of life-critical operations such as brain surgery. The Johns Hopkins/KRDL skull-base surgery simulator for training aneurysm clipping (see below) is one example. The interaction is entirely with the VR itself.
Figure 5.7 JHU/KRDL Skull-base Surgery Simulator
Researchers at University of California San Diego Applied Technology Lab have developed an interesting [http://cybermed.ucsd.edu/AT/AT-anat.html]. They point out that the main challenges they identified from talking to medical faculty and students included visualizing potential spaces; studying relatively inaccessible areas; tracing layers and linings; establishing external landmarks for deep structures; and cogently presenting embryological origins. Correlating gross anatomy with various diagnostic imaging modalities, and portraying complex physiological processes using virtual representation were also considered highly valuable goals.
5.4
Preoperative Planning
Simulators
blur into systems for pre-operative planning. Planning systems also
sometimes blend with augmented reality, since the planning is on a actual,
particular patient, so that physical reality (the patient) and the VR
naturally come together in planning. The aim in such planning is to study
patient data before surgery and so plan the best way to carry out that
surgery.
Simulators blur into systems for pre-operative
planning. Planning systems also sometimes blend with augmented reality,
since the planning is on a actual, particular patient, so that physical
reality (the patient) and the VR naturally come together in planning. The
aim in such planning is to study patient data before surgery and so plan
the best way to carry out that surgery.
The aim of Stereoplan is to allow surgeons to examine patient data
as fully as possible, and evaluate possible routes for intervention.
Further, the system then provides the coordinates for the
stereotactic frames that are standardly used to guide the route for
brain surgery. Similar to the Radionics' Stereoplan aims at helping
planning of stereotactic frame-based functional neurosurgery.
Figure5.8 stereotactic frame-based Figure 5.9 Combined neurosurgery
neurosurgery
planning
planning
and augmented reality
In pre-operative planning the interaction method need not be
realistic and generally is not. The main focus is on exploring the patient
data as fully as possible, and evaluating possible intervention procedures
against that data, not in reproducing the actual operation. The University
of Virginia "Props" interface illustrates this (below). A doll's
head is used in the interaction with the dataset, without any suggestion
that the surgeon will ever interact with a patient's head in quite this
way.
Figure 5.10 used in pre-operative planning
Of
course, the simulation must be accurate. Given this, techniques developed
for planning can sometimes be applied to the prediction of outcomes of
interventions, as in bone replacements or reconstructive plastic surgery.
Such simulations can also help in training, and in communications between
doctors and patients (and their families).An important aspect of such
systems for use by medical staff is the design of the tools and how this
affects usability.
6 Human Interfaces in Surgery
The clinical potential of computer assisted surgery ( CAS ) has been more and more widely acknowledged since CAS systems have been introduced into the operating room (OR) theater. Especially the improvements in safety and accuracy are remarkable and strengthen the ties between surgeons and engineers. Tumor stereotactic was introduced to neurological surgery in the early 1980s, and currently systems with and without robotic navigation are in use for specific medical indications.
Recently, solutions for computer assisted orthopedic surgery were developed and applied to various anatomical regions. However, with the establishment of CAS in vivo, a new complex of problems, which was not present in the laboratory setup, was introduced: the man-machine interface.
Currently, the complexity of available CAS
systems requires the presence of at least one system engineer (often
called the operator) in the OR. As a consequence, there is no possibility
for direct communication between the surgeon and the machine or software.
Most of the program steps involved in CAS and choices to be made
intraoperatively have to be transferred to the software by means of
communication of the surgeon with the operator. Particularly, the
establishment of a relation between the virtual object (i.e., a medical
image) and the surgical object (i.e., the patient), often denoted as
matching or skeletal registration, requires intensive interaction of the
surgeon with the computer.
A literature survey revealed that no CAS system in clinical use
exists without a system engineer or a comparable person, and our clinical
experience indicated that the matching process is a weak point in most
systems. Because it appears to be contradictory to cost-reduction efforts
in health care to have a highly paid specialist in the OR, this research
evaluates strategies to facilitate the man-machine interface with the
final goal of establishing a direct control of the system by the surgeon
or the medical personnel traditionally present at surgery. Options to be
investigated include 1) a CAS control panel (virtual keyboard) as an
integrated component of the existing navigation system and 2) introduction
of a commercial voice-recognition system. The implementation of these
strategies into the existing CAS setup at the Department of Orthopaedic
Surgery at the Inselspital (University of Bern) and clinical experience
gained are reported.
7 Conclusion
CAS never replaces surgeons' hands with robots. For example, robots
in motor factories should not be used in the operating rooms.The first
reason is safety. A robot for industrial use does not know how to
cooperate with human. In OR, safety for patient, surgeons and their
assistants should be always considered.The second reason lays in its
characteristic. Generally speaking, we find advantage of using robots in
impossible or monotonic tasks for human. In this sense, we have little
merit in using them for current surgeries, most of which are possible but
not monotonic for surgeons.
If a robot is in the OR, it is not at all "a robot", but
a new surgical tool where we apply robotics. It is thought to be used in a
complex, precise or dangerous tasks for human, or in complemental tasks
such as positioning of endoscope.
CAS never replaces surgeons' brain with computers.
"Decision" is the most important process in surgery, and should
be done by surgeon himself/herself. Invention in CAS is always to SUPPORT
surgeons but not to replace them.
Many surgical techniques are done only by a very few specialized
surgeons or in specially-equipped hospitals. CAS will make these special
techniques available for young surgeons, small clinics, and many
developing countries.
CAS will provide a lot of new styles of surgery that we have never
imagined. Endoscopic surgery is a good example. Surgery in today is to
remove or replace legions. However, rapid progress in laser surgery or
radiological treatment may make it unnecessary. "Surgery without
operation" is not a dream.
In addition, Active introduction of new technology in surgery will
widen the market of medical equipment. Non-medical industries such as
computer, robotics, virtual reality can play a roll in this market.
Reduction of social cost in medicine is not an irony. The goal of CAS is
not at all to operate by more cumbersome and expensive equipment. Less
invasive surgeries reduce the period of hospitalization and surgical
assistants.
You
don't imagine that surgery in CAS is same as today's surgery, because CAS
has
possibility to revolutionary change the style of surgery.
Bibliography
1.
Besl, P.J., and McKay, N.D., “ A Method for Registration of 3D
Shapes”, IEEE Trans. Pattern
Analysis and Machine Intelligence, 14(2), 1992.
2.
Brack, C., Burghart, R., Czpof, A., et al “Accurate X-ray
Navigation in Computer-Assisted Surgery”, Proc.
Of the 12th Int. Symp. On Computer Assisted Radiology and
Surgery, H. Lemke et al eds., Springer, 1998.
3.
Glossop, N., and Hu, R., “Clinical Use Accuracy in Image Guided
Surgery”, Proc. 11th
Int. Symp. On Computer Assisted Radiology and Surgery, Berlin,
Elsevier 1997.
4.
Aston, Robert and Joyce Schwarz, Eds. Multimedia: Gateway to the
Next Millennium. AP Professional, 1994. (Specifically, Chapter 7, Virtual Reality, by Frank Biocca and Kenneth Meyer.)
5.
On the Cutting Edge of Reality; SAMS Publishing, 1993.
(Specifically, the chapter Virtual
Reality by Linda Jacobson.)
6.
MMWR. Asthma mortality and
hospitalization. Center for Disease Control,1996.
7.
Helsel, Sandra K. and Judith Paris Roth, Eds. Virtual Reality:
Theory, Practice, and Promise. Meckler Publishing, 1991.
8.
Ahring K, Ahring J, Joyce C, Farid N. Telephone modem access
improves diabetes control in those with insulin-requiring diabetes.
Diabetes Care 1992;15(8):971-5.
9.
DG, Vandagriff JL, Kronz K, et al. Using
telecommunication technology to manage children with Marrero diabetes: the
Computer-Linked Outpatient Clinic (CLOC) Study. Diabetes Educator
1995;21(4):313-9.
10.
Ignacio-Garcia JM, Gonzalez-Santos P. Asthma
self-management education program by home monitoring of peak expiratory
flow. American Journal of Respiratory & Critical Care Medicine
1995;151(2 Pt 1):353-9.
11.
Stahlman J, Salmun LM: New
Developments in the Home Monitoring of Asthma. The Internet Journal of
Asthma, Allergy and Immunology 1997; Vol1 N1.