Search in this site
SURGICAL TELEROBOTICS AND VIRTUAL TELEPRESENCE
MONTEREY (USA) TO MILAN (ITALY)
Eng. Alberto Rovetta, Eng. Remo Sala, Eng. Mirco Bressanelli, Eng. Flavio
Arch. Rita Pegoraro, Arch. MariaRita Canina, Ms. Maria Elena Garavaldi
This paper deals with the connection which has been held on 8th July
1997 in collaboration with the JPL of the NASA, Pasadena, California,
between the Eighth International Conference on the Advanced Robotics (ICAR
97) in course at Monterey, California and the Telerobotics Laboratory
of Politecnico di Milano connected in a multipoint teleconference through
the MCU of Rome with the Aula Magna of the same Politecnico and
the Palace Business of the Giureconsulti of the Chamber of Commerce of
Milan. The demonstration has allowed to telecontrol a scara robot of the
Sankyo and an ABB robot, which have affected simulations of operations
of biopsy to the prostate, to the liver and to the breast, a mechanical
hand and a model of a car, disposed in a space destined to reproduce the
Martian ground, from Monterey to Milan by means of the INTERNET+ISDN connection
form. In fact the event has taken place four days after the landing on
Mars happily successful of the spatial probe Pathfinder from which it
has gone out the "Sojourner" robot, telecontrolled from the
JPL of the NASA, which has begun to take photos of the Martian ground
and also some of these images have been transmitted in the course of the
The paper means to accentuate the increasing importance of the cooperation
between the INTERNET network and the ISDN network in the parallel transmission
of data, information and signals by offering a performance of the degree
of feasibility of the experimentation at the service of the more advanced
technology and of the medical and bioengineering science.
For facing the reading of this article from a correct perspective, its
opportune we don't forget (see References) that it isnt about the
first experiment of telecontrol by ISDN or by INTERNET, because many experiments
have already happened at international level, frequently in collaboration
with the same Telerobotics Laboratory of Politecnico di Milano. In fact,
this paper is presenting new results from the first transatlantic teleoperation
on 1993 (Ref. /1/) and the first telerobotic surgery on a patient on 1995
The innovation consists in the utilisation of INTERNET+ISDN together
in the same demonstration, even if the two networks are different and
work on different planes. In fact, INTERNET network is at low cost, it
may be maintained in a continuous connection also with various active
stations and its addressed to transmit and to receive data in a
great quantity, both by vision and sound and to manage complex files which
may be transmitted also in long times. On the contrary, ISDN network presents
about the same costs, but allows the connection only between two stations
in the same time, with the low cost at this moment and manages signals
in a immediate form. It transmits images and data in real time, depending
on the light speed, except the time delay due to the passage of signals
in the electronic systems for transmission and acquisition. The Telerobotics
Laboratory of Politecnico di Milano, directed by Prof. Alberto Rovetta,
during the teleconference with the Eighth International Conference on
the Advanced Robotics (ICAR 97) on 8th July 1997, has succeeded
in the achievement of the principal objective of his more following operational
line from May 1997 to today. In fact, during the International Conference
"The Hospital of the Future", which has been held from 8th to
10th May 1997 in the Kursaal of Merano, Bolzano in collaboration with
the Telerobotics Laboratory of Politecnico di Milano, the INTERNET+ISDN
form has been applied to the telerobotics with the attainment of a very
positive result. This success has allowed to develop the form by preparing
programs that make possible the remote telecontrol by means of the INTERNET
and ISDN potentialities.
By considering the response times, the idea of a perfect interaction
and compensation of the two networks was born: the two solutions are fit
to conjugate telesurgery by an ISDN direct link, but also teledidactics
and teletraining by INTERNET.
On 8th July 1997 at 17.00 Italian time (8.00 a.m American time) it has
begun the connection between the Telerobotics Laboratory of Politecnico
di Milano and the Eighth International Conference on the Advanced Robotics
(ICAR 97) in course at Monterey, California, from where the Prof.
Alberto Rovetta has presented an experiment of telecontrol from Monterey
to Milan. At 16.15 Italian time the Telerobotics Laboratory of Politecnico
di Milano was already in multipoint connection through the MCU of Rome
with the Aula Magna (Great Hall) of the Politecnico and with the Palace
Business of the Giureconsulti of the Chamber of Commerce to allow the
Rector of the Politecnico, the President of the Chamber of Commerce and
Dr. A. Cecchi Paone, journalist, author-conductor of the "The Machine
of the Time" broadcast, selected to coordinate and present the event
in Milan, to do their discourse. From 17.00 Italian time (8.00 a.m. American
time) the connection with the USA has been seen simultaneously from all
the connected points. Prof. A. Rovetta from Monterey has introduced each
individual application with the telecontrol phase. It has followed the
comment of a series of images which have been taken by the "Sojourner"
robot in the course of his exploration of the Martian ground after the
landing on Mars of the Pathfinder probe on 4th July 1997. The telecontrol
by INTERNET has been realised in two phases: during the first one, the
telecontrol of each individual application, an ABB robot for simulation
of operations of biopsy to the liver and to the breast, a mechanical hand
with force feed-back, a model of a car and a scara robot of the Sankyo,
easily moved by changing the digital coordinates by INTERNET and for simulation
of operation of biopsy to the prostate by ISDN. During the second phase,
the four applications by INTERNET have simultaneously functioned thanks
to the development of a specific program called "Grove 2000"
that allow to realise the telecontrol by displaying the windows relative
to each structure in movement on the Server and Client interfaces of the
Scientific and technical aspects
The connection has been supported by Alcatel and Telecom Italy (fig.
From 16.00 Italian time (7.00 a.m American time) on 8th July 1997, it
has been activated the ISDN connection between a 100 PCS of the Alcatel
with PictureTel teleconference system, 1.6 version, present in the Telerobotics
Laboratory of Politecnico di Milano and the MCU of Rome provided by Telecom
Italy to realise the multipoint teleconference foreseen between the Telerobotics
Laboratory of Politecnico di Milano, the Aula Magna of Politecnico and
the Chamber of Commerce of Milan. The Aula Magna S01 has been connected
with the MCU through the Codec CLI Rembrandt II standard audio G.722 present
in the direction of the Room T04 from which it has received the audio/video
signals. The Chamber of Commerce instead has been connected directly from
its main room where they were present two AVC 8000 systems, one in exercise
and one of backup.
At 17.00 Italian time (8.00 a.m American time) it has been established
the point-to-point connection from Milano to Monterey, fundamental to
the result of the operation. The call by ISDN has departed from Milan
from a PCC Olivetti teleconference system with a VC8000 British Telecom
card installed on an Olivetti computer and it has reached an analogous
apparatus present in Monterey, where there was also a 100 PCS of the Alcatel
with PictureTel teleconference system. In the Telerobotics Laboratory
of Politecnico di Milano the PCC Olivetti destined to receive the audio
and the video from the USA has been connected directly with a 100 PCS
of the Alcatel already in multiconference through the appropriate cables
to allow the same audio/video reception simultaneously to all the points
connected in Monterey and in Milan. In particular an In/Out double cable
has allowed the reception and distribution of the audio, while concerning
the video it has been necessary to install a powerful apparatus of cameras.
Substantially the video Out from the PCC by passing across the 4 entry
of the Laboratory video entries selector has become video In in the 100
PCS, as also the video Out of the PCS by passing across a VGA/PAL converter
provided from the Alcatel and by entering in the PCC video entries selector
has become video In in the PCC. Two Alcatel PC, one in exercise and one
of backup, connected with two special dedicated Telecom Italy INTERNET
lines with direct access to Interbusiness have been prearranged for the
telecontrol by INTERNET. Concerning the telecontrol of the scara robot
of the Sankyo (fig. 2), a switch has been inserted by choosing the INTERNET
formality or the ISDN formality. The powerful apparatus of cameras can
be divided in two blocks: the first of the colours environmental cameras
at high definition to frame the Sankyo robot, the ABB robot, the mechanical
hand and the model of a car; the second of the black and white cameras
to frame the details of the structure of the scara robot in movement.
Figure 2: The scara robot of the Sankyo
|In the first phase of the demonstration it has been realised the telecontrol
by INTERNET thanks to the development of four specific Client/Server applications
with the TCP/IP protocol by using the last version of Visual Basic, 5.0
version. Additionally it has been realised a final summary program called
"Grove 2000" that allows the simultaneous telecontrol of the four
applications by displaying the four relative windows in the monitor. Normally
a Quickcam connected with the Server computer could send the visual feed-back
to the remote Client, but in this case the frame has been realised by means
of environmental cameras connected with the computer used for the teleconference.
The first application telecontrolled by INTERNET in the course of the demonstration
concerns an industrial robot of the ABB Robotics with six degrees of liberty
(fig. 3) to perform simulations of operations of biopsy to the liver and
to the breast also in a continuous sequence. The TCP/IP protocol foresees
the development of a program for the Server and of a twin one for the Client
(the Client computer used in the USA by Prof. A. Rovetta was a portable
computer of the AST). For the ABB robot the interface of the Server program
offers two opportunities to impart the movement to the robot. Not before
being insured that the blue button has become red by passing from "not
connected" to "connected" the movement will be sent to the
robot like a predefinite program in the inferior buttons of the menu or
directly like coordinates which have been selected in the "X-Y",
"Z", "Speed" and "Alpha, Beta, Gamma Angles"
windows by means of the mouse.Once the formality of movement, by program
or by coordinates, has been selected, it needs to operate the "Send"
button and, if all it evenly proceeds, from the red rectangle of the menu
will appear the "received command" message. The "Chat"
button allows to establish the written communication with a Client program
in the course of the connection. The "Text" window shows the values
of coordinates, angles and speed which have been sent to the robot command.
Figure 3: The ABB robot
|The second application and the third one telecontrolled by INTERNET have
been the mechanical hand with force feed-back (fig. 4) and a model of a
car (fig. 5).
The mechanical hand constituted by a mechanism able to move a finger
by means of a strain applied to an electric motor could receive the strain
like a remote signal by INTERNET or directly like result of a force applied
on a load cell of a biorobotic system (VREPAR: Virtual Reality Environments
for Psycho-neuro-physiological Assessment Rehabilitation) developed to
value neuromotor responses of patients affected with the Parkinsons
Figure 4: The mechanical hand
|The model of a car has been commanded by infrared signal telecommanded
by a strain signal received in a Server computer in which it has been installed
a card of data acquisition able to emit an exit signal to the control.
The fourth application, a scara SC 3000 industrial robot of the Sankyo
with four degrees of liberty (fig. 2), programmed to perform simulations
of operations of biopsy to the prostate, has been telecontrolled through
the same serial entry of the computer by INTERNET and by ISDN.
At the end from the Jet Propulsion Laboratory of the NASA, USA have been
sent and commented the images which have been taken by the "Sojourner"
robot in the course of his exploration of the Martian ground.
|The demonstration shows that two parallel networks for communication may
be integrated. Because of the unforeseen (up to now) times of the INTERNET
network, the system may transmit images which are on parking, also for a
long time and may move the support structures for the surgery, may prepare
the surgeon table, may analyze the results of the surgeon operation and
transmit them as static data in the medical network. INTERNET network may
permit slow operations, with a high quantity of information. On the contrary,
the ISDN network time delay, already studied in previous researches in the
Telerobotics Laboratory of Politecnico di Milano in cooperation with Dr.
A. Bejczy, JPL, Pasadena, may be foreseen with a certain precision. The
developed system uses the ISDN line and transmits the data of the patient,
of the surgical operation and takes part on the basis of the diagnostic
analysis of the physician.
The appointed objective has been fully achieved. A further level of certainty
in the feasibility of the telecontrol by INTERNET+ISDN has been achieved.
From the International Congress of Merano "The Hospital of the Future"
from 8th to 10th May 1997, in the course of which for the first time it
has been applied the INTERNET+ISDN form to the Eighth International Conference
on the Advanced Robotics (ICAR 97) many tests and experimentations
have accompanied the scientific and technical way of the Telerobotics
Laboratory of Politecnico di Milano in the direction of a notable progress
in acquaintance and evaluation of reliability in the application of the
INTERNET and ISDN networks not only in the field of the telecontrol of
medical robotics and bioengineering applications.
The Telerobotics Laboratory of Politecnico di Milano means to continue
along the directive of experimentation of the INTERNET+ISDN form, above
all in the field of the telerobotics and telehealth by exposing the results
in the most important international conferences that will take place in
the future in order to compare and to discuss the achieved results.
The demonstration has allowed to open and develop a new frontier: the
operative interconnection of Internet and ISDN. In the teleconnection
with the Congress of Merano "The Hospital of the Future" the
INTERNET+ISDN form has been developed in the field of the telehealth like
tool of prevention, for diagnosis, therapy, assistance at a long distance
and training; in the connection with ICAR 97 has been used to develop
programs of telecontrol of medical robotics and bioengineering applications.
Concluding, the INTERNET network is at low cost and allows the telecontrol
with unforeseen (up to now) time delay; it may collect and transmit a
great quantity of data at a long distance by allowing simultaneously several
Clients to intervene and to actuate the Server computer without competition.
The test on such share are on development. The ISDN network is at about
the same low cost and its able to have a high efficiency for quick
actions and for transmission and share of audio and visual signals in
a small quantity, but allows the connection only between two stations
in the same time.
The Authors thank Dr. A. Bejczy and Eng. P. Fiorini of the Jet Propulsion
Laboratory of the NASA, USA, Prof. F. Falcone and Mr. C. Coniglio for
the cooperation, the undergraduate students of the Telerobotics Laboratory
of Politecnico di Milano for the connections and the software development.
Special thanks to: Eng. G. Gislon and Dr. G. Melini of the Alcatel, Eng.
P. Leone and Eng. P. Conca of the ABB Robotics, Eng. F. Meacci of the
Telecom Italy for the technical and scientific support and for the teleconference
equipment; Prof. A. De Maio, Rector of Politecnico di Milano and Dr. P.
Bassetti, President of the Chamber of Commerce of Milan for the hospitality,
Dr. A. Cecchi Paone, journalist, author-conductor of the "The Machine
of the Time" broadcast, for the presentation of the connection.
/1/ A. Rovetta, R. Sala, F. Cosmi, X. Wen, D. Sabbadini, S. Milanesi,
A. Togno, L. Angelini, A. Bejczy, Telerobotic Surgery in a Transatlantic
Experiment: Application in Laparoscopy, Proceed. Telemanipulator Technology
and Space Telerobotics, Boston, September 1993
/2/ A. Rovetta, R. Sala, Robotics and telerobotics applied to a prostatic
biopsy on a human patient, Second Symposium on Medical Robotics and Computer
Assisted Surgery, Baltimore, November 1995
/3/ A. Rovetta, R. Sala, F. Cosmi, X. Wen, S. Milanesi, D. Sabbadini,
A. Togno, L. Angelini, A. Bejczy, A New Telerobotic Application: Remote
Laparoscopic Surgery Using Satellites and Optical Fiber Networks for Data
Exchange, The International Journal of Robotics Research, Vol. 15, N.
3, June 1996
/4/ A. Rovetta, A. Bejczy, R. Sala, Telerobotic Surgery: Applications
On Human Patients and training With Virtual Reality, Congress "Medicine
Meets Virtual Reality", San Diego, January 1997
/5/ A. Rovetta, F. Falcone, R. Sala, M. E. Garavaldi, Telehealth in Africa,
Congress "Medicine Meets Virtual Reality", San Diego, January
/6/ A. Rovetta, Internet plus ISDN: a parallel cooperative transmission
for telehealth in the hospital of the future, Conclusive Article after
the Congress of Merano, Bolzano, "The Hospital of the Future",