Forum on Risks to the Public in Computers and Related Systems
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Volume 3: Issue 36
Tuesday, 12 August 1986
Contents
Another Medical Risk?- Lee Breisacher
- RISKy Business in Surgery
- Mark Jackson
- Reliance on word-processors discussed in the Israeli Supreme
- Ady Wiernik
- Expert Systems - The New Cop on the Beat
- Laws via Fred Ostapik
- Chernobyl
- Art Evans
Dick Karpinski - Air Traffic Control computer failure
- Dan Melson
- Possible failures of BMD software
- Herb Lin
- A note about stories "from memory"
- Henry Mensch
- Info on RISKS (comp.risks)
Another Medical Risk?
<Breisacher.OsbuSouth@Xerox.COM>
12 Aug 86 09:25:49 PDT (Tuesday)
From the August PSA Airline magazine, extracted from an article about inventors: [There's a photo of Dr. Kwoh in surgical garb in an operating room leaning over a dummy patient with some elaborate equipment surrounding its head. The caption reads:] Robotic surgery is a reality because of the obsessive work of Yik San Kwoh, medical research and development director of Long Beach Memorial Hospital. His computer controlled "surgeon," capable of conducting brain surgery within an accuracy of 1/2000 of an inch, was the result of three years of incessant programming. [From the text of the article:] Yik San Kwoh, medical research and development director of Long Beach Memorial Hospital, explains, "I've got two Apple computers at home and three IBMs. I spend so much time on those damn things that I get sick of it. Only then can I stop." It took three years of programming and reprogramming for Kwoh to turn and industrial robot into a surgical instrument capable of conducting brain surgery. [As usual, we must weigh the risks of using such equipment against the risks of NOT using it. On the other hand, the description makes it sound like he programmed this thing the way I wrote my first couple programs (FORTRAN in the early 70's) -- dive in and start writing code then keep debugging til it sorta works.] Lee
RISKy Business in Surgery
<MJackson.Wbst@Xerox.COM>
12 Aug 86 07:56:26 EDT (Tuesday)
From /Programmers at Work (1st Series): Interviews/, by Susan Lammers
(Microsoft Press, 1986):
"My most amazing experience, though, was a phone call I got right after
I started Iris, from a surgeon who was using Symphony for real-time data
analysis during open heart surgery. It is sobering to think that
someone was lying on an operating table potentially relying upon my
program running properly. It reminds one of the real responsibility to
the end users."
-- Ray Ozzie
project leader for Symphony
Reliance on word-processors discussed in the Israeli Supreme Court
Ady Wiernik <ady%taurus.BITNET@WISCVM.ARPA>
Tue, 12 Aug 86 21:19:31 -0300
Rules of Court in Israel fix a time limit for bringing an appeal to the
Supreme Court against a decision of an inferior Court.
A lawyer applied to Supreme Court for an extension of the period to
appeal. He has missed the statutory period by two days. His excuse was that
the word-processor in his office (that has been recently installed)
malfunctioned.
The text of the appeal that was typed into the computer has been erased
because of that computer malfunction. He called the maintenance personnel.
They promised that the malfunction would be shortly repaired, but actually,
it lasted longer, causing him not to be able to bring the appeal at the same
day.
The appellant claimed that the trouble with the computer was an "act
of god", Force Majeure, which is considered a special ground that entitles
him the desired extension.
The court has rejected this argument.
In his judgement, Registra Tzur of the Supreme Court said: "Indeed,
the computer is very useful, but one must prepare for possible malfunctions
in its operation. When there is no computer, the good old typewriter should
replace it."
This decision is the first recorded judicial reference to the use of
word-processing devices in lawyer offices, and displays the dangerous
results of reliance on high-tech.
Ady Wiernik
Expert Systems - The New Cop on the Beat
<Laws@SRI-STRIPE.ARPA [courtesy of Fred Ostapik]>
Mon 4 Aug 86 22:38:23-PDT
The FBI has developed Big Floyd, an expert system to assist in criminal investigations. Similar programs are being developed to catch drug smugglers and target potential terrorists. The EPA wants to identify polluters; the Treasury Department is looking for money-laundering banks; the Energy Department would like to find contractors who cut corners; the Customs service is after drug smugglers; the IRS is developing a system to spot tax cheaters; the Secret Service is working on a classified system to point out potential presidential assassins; and the FBI's National Center for the Analysis of Violent Crimes is developing expert systems to identify potential serial killers, arsonists, and rapists. Systems to target counterfeiters and bombers are also being built. -- Michael Schrage, The Washington Post National Weekly Edition, Vol. 3, No. 40, August 4, 1986, p. 6.
Chernobyl
"Art Evans" <Evans@TL-20B.ARPA>
Tue 12 Aug 86 11:34:21-EDT
In RISKS-3.35, Robert Stroud comments on "Official Report on Chernobyl disaster". Although the discussion of what actually triggered that disaster is interesting, I choose to focus instead on how the Russian explanation was interpreted by others (not by Mr Stroud). Quoting from the post: But many believe the explanation [offered by the Russians] is inadequate and that it is being promoted mainly to protect the country's nuclear construction programme. No justification is given for this belief. A Peter Potter is quoted as saying By maintaining that human error and turbine problems were really to blame, the Russians could say that their reactors have no serious design flaws. They could then avoid calls for closures of other reactors or for the implementation of drastic redesign work. This claim may in fact be true, but we are given no evidence. Note what is happening: The Russians offer a technical explanation for the disaster. A western nuclear expert says the explanation is inaccurate and was offered for political reasons. But, no reason other than political is given for this skepticism. The Russians may well be lying, and if there is evidence I would like to see it. Lacking such evidence, though, the public would be better served by less misleading pronouncements by "experts". Art Evans
Chernobyl
Dick Karpinski <dick@cca.ucsf.edu>
Tue, 12 Aug 86 11:13:17 PDT
The only unadvertised design deficiency that I know of in the Chernobyl reactor is that it has a positive coeficient of reactivity with respect to temperature. That is, when the temperature goes up, so does the rate of nuclear fission. Such a design would be ruled out here, claims my source, a former reactor containment vessel engineer. Surely, such a design would make the sort of accident which occurred more likely. Dick Dick Karpinski Manager of Unix Services, UCSF Computer Center UUCP: ...!ucbvax!ucsfcgl!cca.ucsf!dick (415) 666-4529 (12-7) BITNET: dick@ucsfcca Compuserve: 70215,1277 Telemail: RKarpinski USPS: U-76 UCSF, San Francisco, CA 94143
Air Traffic Control failure
Dan Melson <crash!pnet01!dm@nosc.ARPA>
Mon, 11 Aug 86 23:47:21 PDT
Computer failures at Air Route Centers are not as uncommon as we'd like, but
they're not as nasty as they could be. Despite the fact that the computers
currently used are more than fifteen years old, they seem to handle the load
well enough for the present. When the primary computers (IBM 9020's) go down,
however, the DARC backup system does not furnish the controllers with nearly
as much data, and it is far more difficult to get automated tasks done.
There is currently a new computer system in the works, and when it is
operational, delays due to computer failure should dramatically decrease. The
estimate for this is 'around 1990'.
At any rate, even the bachup systems are far more pleasant than doing all
of the work manually.
DM
Possible failures of BMD software
<LIN@XX.LCS.MIT.EDU>
Tue, 12 Aug 1986 00:38 EDT
I'm working on a paper on potential software-induced difficulties and problems that might accompany the deployment of a BMD system. I'd like to enlist the collective imagination of the list on examples apropos to this paper. Please constrain your imagination by the limits of the possible (e.g., it is impossible for an X-ray laser to shoot x-rays at ground targets, but it is not impossible that the firing of an X-ray laser creates an electromagnetic pulse that has unanticipated effects). Please specify the scenario in as much detail as you can. I am not specifying a system architecture, so please tell me the one(s) you have in mind in your scenario(s); that is necessary because softare -- by itself -- is harmless no matter how buggy it is. Also remember that BMD has significant capability against satellites. Thanks. Acknowledgements will be provided if you so desire. Herb Lin
A note about stories "from memory"
Henry Mensch <henry@ATHENA.MIT.EDU>
Mon, 11 Aug 86 23:44:12 -0500
I hate to sound like a nit-picker but I've noticed a rash of stories
which begin with words like "If I remember correctly ..." or "It's
pretty late, so expect errors." Is this sort of thing a product of
having such powerful communications tools at our fingertips?
Once these things happen we seem to spend a lot of time saying "Well,
*I* thought it went this way. . . " In discussing risks to the public,
we risk wasting our time doing these tasks, which could be avoided
with a bit of research.
Striving for better communications,
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Henry Mensch | Technical Writer | MIT/Project Athena
henry@athena.mit.edu ..!mit-eddie!mit-athena!henry
[On the one hand, it is nice to be precise. On the other hand,
if the report is novel and interesting, perhaps RISKS provides
a medium for getting feedback from an expert on a matter that
would otherwise go unreported. But, I certainly appreciate it
when contributors take a little time to track down the reference
-- and especially when they cite that reference. PGN]
Report problems with the web pages to the maintainer