The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 3 Issue 36

Tuesday, 12 August 1986


o Another Medical Risk?
Lee Breisacher
o RISKy Business in Surgery
Mark Jackson
o Reliance on word-processors discussed in the Israeli Supreme
Ady Wiernik
o Expert Systems - The New Cop on the Beat
Laws via Fred Ostapik
o Chernobyl
Art Evans
Dick Karpinski
o Air Traffic Control computer failure
Dan Melson
o Possible failures of BMD software
Herb Lin
o A note about stories "from memory"
Henry Mensch
o Info on RISKS (comp.risks)

Another Medical Risk?

12 Aug 86 09:25:49 PDT (Tuesday)
From the August PSA Airline magazine, extracted from an article about

[There's a photo of Dr. Kwoh in surgical garb in an operating room leaning
over a dummy patient with some elaborate equipment surrounding its head.
The caption reads:]

Robotic surgery is a reality because of the obsessive work of Yik San
Kwoh, medical research and development director of Long Beach Memorial
Hospital.  His computer controlled "surgeon," capable of conducting
brain surgery within an accuracy of 1/2000 of an inch, was the result of
three years of incessant programming.

[From the text of the article:]

Yik San Kwoh, medical research and development director of Long Beach
Memorial Hospital, explains, "I've got two Apple computers at home and
three IBMs.  I spend so much time on those damn things that I get sick
of it.  Only then can I stop."

It took three years of programming and reprogramming for Kwoh to turn
and industrial robot into a surgical instrument capable of conducting
brain surgery.

[As usual, we must weigh the risks of using such equipment against the
risks of NOT using it.  On the other hand, the description makes it
sound like he programmed this thing the way I wrote my first couple
programs (FORTRAN in the early 70's) — dive in and start writing code
then keep debugging til it sorta works.]


RISKy Business in Surgery

12 Aug 86 07:56:26 EDT (Tuesday)
From /Programmers at Work (1st Series):  Interviews/, by Susan Lammers
(Microsoft Press, 1986):

"My most amazing experience, though, was a phone call I got right after
I started Iris, from a surgeon who was using Symphony for real-time data
analysis during open heart surgery.  It is sobering to think that
someone was lying on an operating table potentially relying upon my
program running properly.  It reminds one of the real responsibility to
the end users."

    — Ray Ozzie
       project leader for Symphony

Reliance on word-processors discussed in the Israeli Supreme Court

Ady Wiernik <ady%taurus.BITNET@WISCVM.ARPA>
Tue, 12 Aug 86 21:19:31 -0300
     Rules of Court in Israel fix a time limit for bringing an appeal to the
Supreme Court against a decision of an inferior Court.

     A lawyer applied to Supreme Court for an extension of the period to
appeal. He has missed the statutory period by two days.  His excuse was that
the word-processor in his office (that has been recently installed)

     The text of the appeal that was typed into the computer has been erased
because of that computer malfunction.  He called the maintenance personnel.
They promised that the malfunction would be shortly repaired, but actually,
it lasted longer, causing him not to be able to bring the appeal at the same

     The appellant claimed that the trouble with the computer was an "act
of god", Force Majeure, which is considered a special ground that entitles
him the desired extension.

     The court has rejected this argument.

     In his judgement, Registra Tzur of the Supreme Court said:  "Indeed,
the computer is very useful, but one must prepare for possible malfunctions
in its operation.  When there is no computer, the good old typewriter should
replace it."

     This decision is the first recorded judicial reference to the use of
word-processing devices in lawyer offices, and displays the dangerous
results of reliance on high-tech.

                                        Ady Wiernik

Expert Systems - The New Cop on the Beat

<Laws@SRI-STRIPE.ARPA [courtesy of Fred Ostapik]>
Mon 4 Aug 86 22:38:23-PDT
The FBI has developed Big Floyd, an expert system to assist in criminal
investigations.  Similar programs are being developed to catch drug
smugglers and target potential terrorists.  The EPA wants to identify
polluters; the Treasury Department is looking for money-laundering
banks; the Energy Department would like to find contractors who cut
corners; the Customs service is after drug smugglers; the IRS is
developing a system to spot tax cheaters; the Secret Service is working
on a classified system to point out potential presidential assassins;
and the FBI's National Center for the Analysis of Violent Crimes is
developing expert systems to identify potential serial killers,
arsonists, and rapists.  Systems to target counterfeiters and bombers
are also being built.  — Michael Schrage, The Washington Post National
Weekly Edition, Vol. 3, No. 40, August 4, 1986, p. 6.


"Art Evans" <Evans@TL-20B.ARPA>
Tue 12 Aug 86 11:34:21-EDT
In RISKS-3.35, Robert Stroud comments on "Official Report on Chernobyl
disaster".  Although the discussion of what actually triggered that
disaster is interesting, I choose to focus instead on how the Russian
explanation was interpreted by others (not by Mr Stroud).

Quoting from the post:
    But many believe the explanation [offered by the Russians] is
    inadequate and that it is being  promoted mainly to protect the
    country's nuclear construction programme.
No justification is given for this belief.  A Peter Potter is quoted as saying
    By maintaining that human error and turbine problems were really to
    blame, the Russians could say that their reactors have no serious
    design flaws. They could then avoid calls for closures of other
    reactors or for the implementation of drastic redesign work.
This claim may in fact be true, but we are given no evidence.

Note what is happening: The Russians offer a technical explanation for
the disaster.  A western nuclear expert says the explanation is
inaccurate and was offered for political reasons.  But, no reason other
than political is given for this skepticism.  The Russians may well be
lying, and if there is evidence I would like to see it.  Lacking such
evidence, though, the public would be better served by less misleading
pronouncements by "experts".

Art Evans


Dick Karpinski <>
Tue, 12 Aug 86 11:13:17 PDT
The only unadvertised design deficiency that I know of in the Chernobyl
reactor is that it has a positive coeficient of reactivity with respect
to temperature.  That is, when the temperature goes up, so does the rate
of nuclear fission.  Such a design would be ruled out here, claims my
source, a former reactor containment vessel engineer.  Surely, such a
design would make the sort of accident which occurred more likely.
Dick Karpinski    Manager of Unix Services, UCSF Computer Center
UUCP: ...!ucbvax!ucsfcgl!cca.ucsf!dick   (415) 666-4529 (12-7)
BITNET: dick@ucsfcca   Compuserve: 70215,1277  Telemail: RKarpinski
USPS: U-76 UCSF, San Francisco, CA 94143

Air Traffic Control failure

Dan Melson <crash!pnet01!dm@nosc.ARPA>
Mon, 11 Aug 86 23:47:21 PDT
Computer failures at Air Route Centers are not as uncommon as we'd like, but
they're not as nasty as they could be.  Despite the fact that the computers
currently used are more than fifteen years old, they seem to handle the load
well enough for the present.  When the primary computers (IBM 9020's) go down,
however, the DARC backup system does not furnish the controllers with nearly
as much data, and it is far more difficult to get automated tasks done.

There is currently a new computer system in the works, and when it is
operational, delays due to computer failure should dramatically decrease.  The
estimate for this is 'around 1990'.

At any rate, even the bachup systems are far more pleasant than doing all
of the work manually.  


Possible failures of BMD software

Tue, 12 Aug 1986 00:38 EDT
I'm working on a paper on potential software-induced difficulties and
problems that might accompany the deployment of a BMD system.  I'd
like to enlist the collective imagination of the list on examples
apropos to this paper.

Please constrain your imagination by the limits of the possible (e.g.,
it is impossible for an X-ray laser to shoot x-rays at ground targets,
but it is not impossible that the firing of an X-ray laser creates an
electromagnetic pulse that has unanticipated effects).  Please specify
the scenario in as much detail as you can.  I am not specifying a
system architecture, so please tell me the one(s) you have in mind in
your scenario(s); that is necessary because softare — by itself — is
harmless no matter how buggy it is.  Also remember that BMD has
significant capability against satellites.

Thanks.  Acknowledgements will be provided if you so desire.

Herb Lin

A note about stories "from memory"

Henry Mensch <henry@ATHENA.MIT.EDU>
Mon, 11 Aug 86 23:44:12 -0500
I hate to sound like a nit-picker but I've noticed a rash of stories
which begin with words like "If I remember correctly ..." or "It's 
pretty late, so expect errors."  Is this sort of thing a product of
having such powerful communications tools at our fingertips?

Once these things happen we seem to spend a lot of time saying "Well,
*I* thought it went this way. . . "  In discussing risks to the public,
we risk wasting our time doing these tasks, which could be avoided
with a bit of research.

Striving for better communications,

 - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Henry Mensch     |   Technical Writer  | MIT/Project Athena          ..!mit-eddie!mit-athena!henry

      [On the one hand, it is nice to be precise.  On the other hand,
       if the report is novel and interesting, perhaps RISKS provides 
       a medium for getting feedback from an expert on a matter that
       would otherwise go unreported.  But, I certainly appreciate it
       when contributors take a little time to track down the reference
       — and especially when they cite that reference.  PGN]

Please report problems with the web pages to the maintainer