Forum on Risks to the Public in Computers and Related Systems
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
Volume 2: Issue 8
Friday, 7 Feb 1986
Contents
Expert systems and shuttles- Michael Brown
Dave Platt - Plutonium
- Martin J. Moore
- Earthquake Monitoring Systems
- Mike Raugh via Matt Bishop
Hal Murray
Eugene Miya - Info on RISKS (comp.risks)
Expert systems and shuttles
<mlbrown@nswc-wo.ARPA>
Fri, 7 Feb 86 09:17:13 est
In Risks 2.7, J. Giles speculates: >Has expert system technology been thought of as a fix for this problem? >... a really fast computer ... could monitor all those inputs which aren't >under the direction of human flight controllers... >Are expert systems yet advanced enough to make this worthwhile? Unfortunately, expert systems developed to handle such an occurrence would have to be based on a foreknowledge of the relationship of the various anomalies that occurred in the shuttle disaster. I seriously doubt that most competent systems safety engineers could have predicted the occurrence even with a full knowledge of the anomalies that occurred. Development of such an expert system would likely have to be based on that type of knowledge. However, expert systems aside, I am amazed that the NASA systems safety people would permit a multiple section rocket motor to be manufactured at one location and assembled at another. Misfortune has shown us in the past that these composite structure solid rockets have some very unique and undesirable properties. It will be interesting to see exactly where the failure occurred in the shuttle's SRB if in fact it did fail. If the failure occurred at some location other than the suspect joint, chalk another one up to experience. Michael Brown
Expert systems to detect shuttle failure
Dave Platt <Dave-Platt%LADC@CISL-SERVICE-MULTICS.ARPA>
Fri, 07 Feb 86 10:11 PST
Well, it's certainly possible to set up some sort of expert system that
would monitor incoming telemetry and issue warnings in case of
possibly-dangerous combinations of unusual conditions. However,
I can see a couple of difficulties involved here:
1- There are some conflicts re the amount of data that you want to
feed into the expert-system tool. Certainly, the more data that
is available (from many different classes of sensors), the smaller
the chance that the tool won't have the information needed to
detect the problem. [For example, Challenger was equipped with
far fewer sensors on the SRB than was Columbia during its tryoug
flights].
But... as you increase the number of individual sensors, and the
amount of data (# of different classes of data, especially), you
necessarily increase the number of rules in the system, and the
amount of crunchpower necessary to step through the rules and
determine whether any conclusions need to be brought to the
attention of the controllers. It doesn't do you much good to
receive a warning saying "Engine failure is probable, based on
conditions xxx and yyy" if you don't get the warning in time to
do anything about it.
In my [very limited] experience, very few if any existing expert
systems are capable of handling large amounts of real-time data;
the ones that I've seen tend to be somewhat sluggish. I don't
doubt that it would be possible to build special-purpose hardware
that would support such a system... but I don't believe it has been
done yet.
2- As I understand them, expert systems are designed to reproduce (or
mimic) the sort of what-if and maybe-then decision sequences that
an expert would go through when analyzing a particular sort of
problem. They work by encoding (in explicit form) the steps and
conclusion that an expert would use. A large part of the work
involved in developing an expert system is sitting down with the
expert(s), and assisting them in encoding their (often implicit and
unspoken) rules into rigorous form.
All well and good... BUT... the expert system's "expertise" is
entirely limited by the completeness of the rules that are used to
construct it. One cannot assume that an expert system will be able
to detect or diagnose a situation that has never been encountered
before... it may do so, if the rules were complete enough and if the
situation is similar to one that has occurred before, but you don't
want to bet your life on it!
Only the simplest expert systems can ever be considered to be
"complete". When solving a complex, real-world problem (such as
"Is the shuttle's current behavior normal?"), the best that you
can expect is that some useful fraction of all possible situations
will be analyzed in a meaningful fashion. Expert systems tend to
grow and evolve as they are used... just as a human expert's
capabilities do... and both humans and expert systems will tend to
misdiagnose situations that fall outside of their current knowledge
base.
3- Even if an expert system reacts quickly and accurately enough to give
a meaningful warning ("SRBs leaking, ET overheating, explosion
imminent"), you're still faced with: [A] Human reaction time (controller
and pilot); [B] taking the necessary immediate action (split the
SRBs from the ET and/or split the orbiter away from the ET); and
[C] surviving (getting far enough away from the ET before it goes
*BLOOIE*, and then completing a very difficult dead-stick turnaround
and landing, or a tough water ditching). In the case of the Challenger
explosion, it looks as if all three of these factors were dead-set
against the crew... there was very little time to react, no way
to get away, and a water ditching would probably have killed many
of the crew.
I imagine that you could certainly build an expert system that would
be capable of reading the shuttle's telemetry, and warning of most
conditions THAT THE DESIGNERS OF THE SYSTEM HAD TAKEN INTO ACCOUNT!
The real problem lie, of course, in detecting conditions that no one
had expected would occur... if the system has no rules that would lead
to a conclusion such as "The SRB segment ring seals are leaking",
then the system will never report such a condition. At best, some other
warning will be reported ("Asymmetric thrust from SRBs exceeding
2%"); at worst, no warning will be received, or the system will issue
warnings unnecessarily ("Heavy engine vibration").
Plutonium
"MARTIN J. MOORE" <mooremj@eglin-vax>
0 0 00:00:00 CDT
I don't think the worries about plutonium should be dismissed out of hand. It is my understanding that the lethality of plutonium is due to its extreme toxicity, as opposed to its radioactivity. Comments from a knowledgeable chemist are eagerly solicited.
Re: Earthquake Monitoring Systems
Matt Bishop <mab@riacs.ARPA>
7 Feb 1986 1502-PST (Friday)
I took the liberty of forwarding Gary Leaven's question on earthquake
monitoring systems (ie, are they designed to function during a major
earthquake?) to Mike Raugh, the author of the CACM article which
prompted the question. Here's his reply:
---------------------------------
Matt,
The question you forwarded to me is a good one: Are the
seismic instruments used in Calnet and the Southern California Array
built to withstand the shaking of a major earthquake? The answer is
Yes and No, but it doesn't matter! Even if a local subset of
instruments (or the telemetry system serving that subset) is
knocked out by a major quake, more distant instruments will pick up
signals from the quake that will be adequate for locating, timing and
calculating the earthquake "mechanism", i.e. direction of first motion,
plane of rupture, magnitude. The purpose of the two arrays is to
monitor earthquake activity throughout California, so you can see that
the entire combined two arrays will almost certainly not be totally
incapacited by a major quake, hence they will continue to monitor
activity (even distant activity) successfully.
That being said, it should also be mentioned that seismologists
are very interested in the fine-grained signals that are obtainable
only at close range to a major earthquake (seismic waves that have
traveled teleseismic distance through the earth lose much of the higher
frequency energy). Such close-in data from large earthquakes can only
be obtained from special "strong motion" instruments: this type of
instrument furnished the data for Archuleta's study of the Imperial
Valley quake discussed in my paper. Strong motion instruments are much
more difficult to make, for all the obvious reasons, and are
expensive compared to the ones that comprise the two arrays mentioned
above.
The problem of designing sophisticated modern microcomputer
based instruments that have sufficient sensitivity and dynamic range
and are robust in the presence of violent shaking is a big one.
Especially when you consider that such instruments must have local
storage and power supplies to back up data collection in the event of
telemetry break-downs. I can think of two groups at the USGS in Menlo
Park that are working on systems of this kind. The first is lead by Roger
Borchardt (his GEOS project was mentioned in my article). Another is
being conducted by Larry Baker, Joe Fletcher, and Paul Spudich, who are
developing a down-hole three-dimensional mesh of instruments for
observation of the detailed progression of faulting expected to occur
in the officially USGS-predicted earthquake at Parkfield. In other
words, new designs for such instruments are on the frontier of research
and development at the USGS. Very likely other work of similar import
is taking place elsewhere.
I hope this answers your question.
Mike
Earthquake Monitoring Systems
<Murray.pa@Xerox.COM>
Fri, 7 Feb 86 03:13:43 PST
Neither of these stories involves mainline computer risks, but they might contribute some insight. I got this story from a friend doing earthquake research for the USGS. I think it was '71 when a bigish quake near LA collapsed a VA hospital and a half constructed bridge. That generated a lot of interest in the way buildings (and bridges) react to quakes. Nobody really knew how much stress is present on various structural parts of a building. In response, many strain recording gizmos were installed in many large buildings. Time passed, and everybody went back to their normal work. After several years, another bigish quake came along, and somebody remembered all those installed instruments. So they went out to collected them. Most of them had died. I don't remember any numbers, but I was left with the feeling that everybody was discouraged that they didn't get much interesting data. Another friend worked on LASA (Large Area Seismic Array?). It was one of the early seismic arrays with hundreds of sensors scattered over eastern Montana. I think it was primarily part of the bomb test detection program. With that many sensors and that much wire and electronics to collect all the data, a few sensors were always off the air. They discovered that they got better data if they didn't tell the fixit crew that a test was coming.
Re: RISKS-2.7: Earthquake monitoring systems
Eugene Miya <eugene@AMES-NAS.ARPA>
7 Feb 1986 0849-PST (Friday)
... I can tell you that earthquake instrumentation really need not survive a local earthquake. Local measurement is very unreliable because of environmental factors: soil type, underlying geologic structures, and so forth
Report problems with the web pages to the maintainer