The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 3 Issue 83

Tuesday, 21 October 1986


o Risks from Expert Articles
David Parnas
Herb Lin
Andy Freeman
o Loss of Nuclear Submarine Scorpion
Donald W. Coley
o Staffing Nuclear Submarines
Martin Minow
o An SDI Debate from the Past
Ken Dymond
o System effectiveness is non-linear
Dave Benson
o Stealth vs Air Traffic Control
Schuster via Herb Lin
o Missing engines & volcano alarms
Martin Ewing
o Info on RISKS (comp.risks)

Re: Risks from Expert Articles (RISKS-3.82)

Tue, 21 Oct 86 09:39:51 EDT
   Andy Freeman criticizes the following by Michael L.  Scott, "Computers have
no such abilities.  They can only deal with situations they were programmed
in advance to expect."  He writes, "Dr.  Scott obviously doesn't write very
interesting programs.  :-) Operating systems, compilers, editors, mailers,
etc. all receive input that their designers/authors didn't know about
exactly.  "

   Scott's statement is not refuted by Freeman's.  Scott said that the
computer had to have been programmed, in advance, to deal with a situation.
Freeman said that sometimes the programmer did not expect what happened.
Scott made a statement about the computer.  Freeman's statement was about
the programmer.  Except for the anthropomorphic terms in which it is
couched, Scott's statement is obviously correct.

   It appears to me that Freeman considers a program interesting only if we
don't know what the program is supposed to do or what it does.  My
engineering education taught me that the first job of an engineer is to find
out what problem he is supposed to solve.  Then he must design a system
whose limits are well understood.  In Freeman's terminology, it is the job
of the software engineer to rid the world of interesting programs.

   Reliable compilers, editors, etc., (of which there are few) are all
designed on the basis of a definition of the class of inputs that they are
to process.  We cannot identify the actual indvidual inputs, but we must be
able to define the class of possible inputs if we are to talk about
trustworthiness or reliability.  In fact, to talk about reliability we need
to know, not just the set of possible inputs, but the statistical
distribution of those inputs.

Dave Parnas

Risks from Expert Articles

Tue, 21 Oct 1986 09:16 EDT
    From: Andy Freeman <ANDY at Sushi.Stanford.EDU>

    Operating systems, compilers, editors, mailers, etc. all receive input
    that their designers/authors didn't know about exactly.  

When was the last time you used a mailer, operating system, compiler,
etc.. that you trusted to work *exactly* as documented on all kinds of
input?  (If you have, pls share it with the rest of us!)

    It can be argued that SDI isn't understood well enough for humans to make
    the correct decisions (assuming super-speed people), let alone for them to
    be programmed.  That's a different argument, and Dr. Scott is (presumably)
    unqualified to give an expert opinion.  His expertise does apply
    to the "can
    SDI decision be programmed correctly?"  question, which he spends just one
    paragraph on.

You are essentially assuming away the essence of the problem by
asserting that the specs for the programs involved are not part of the
programming problem.  You can certainly SAY that, but that's too
narrow a definition in my view.

Re: Risks from Expert Articles

Andy Freeman <ANDY@Sushi.Stanford.EDU>
Tue 21 Oct 86 14:40:48-PDT
Herb Lin writes:

    When was the last time you used a mailer, operating system, compiler,
    etc.. that you trusted to work *exactly* as documented on all kinds of
    input?  (If you have, pls share it with the rest of us!)

The programs I use profit me, that is, their benefits to me exceed
their costs.  The latter includes their failures (as well as mine).  A
similar metric applies to weapons in general, including SDI.  (Machine
guns jam too, but I'd rather have one than a sword in most battle
conditions.  The latter are, for the most obsolete, but there aren't
perfect defenses against them.)

Lin continued with:

    You are essentially assuming away the essence of the problem by
    asserting that the specs for the programs involved are not part of the
    programming problem.  You can certainly SAY that, but that's too
    narrow a definition in my view.

Sorry, I was unclear.  Specification and implementation are related,
but they aren't the same.  There are specs that can't be implemented
acceptably (as opposed to perfectly).  Some specs can't be implemented
acceptably in some technologies, but can in others.  (This can be
context dependent.)  Dr. Scott's expertise applies to the question of
whether a given spec can be programmed acceptably, not whether there
is an spec that can be implemented acceptably.  Much of the spec,
including the interesting parts of the definition of "acceptable", is
outside CS, and (presumably) Dr. Scott's expertise.

Another danger (apart from simplification to incorrectness) of expert
opinion articles is unwarranted claims of expertise.  Dr. Scott
(presumably) has no expertise in directed energy weapons yet he claims
that they can be used against cities and missiles in silos.  Both
proponents and opponents of SDI usually agree that it doesn't deal
with cruise missiles.  If you can kill missiles in silos and attack
cities, cruise missiles are easy.


Loss of Nuclear Submarine Scorpion

Donald W. Coley <coley@SCRC-VALLECITO.ARPA>
Tue, 21 Oct 86 12:38 EDT
This is in response to John Allred's comments about the loss of both the
Thresher and the Scorpion (RISKS-3.82).

    Date:     Mon, 20 Oct 86 13:31:40 EDT
    From:     John Allred <>
    Subject:  Loss of the USS Thresher

    Thresher, according to the information I received while serving
    on submarines, was lost due to a catastrophic failure of a main
    sea water valve and/or pipe, causing the flooding of a major
    compartment.  The cause of the sinking was reported by the mother
    ship during the boat's sea trials.

Just to confirm what John stated, fracture of a hull-penetration fitting, at
the weld between the flange and the pipe, quickly flooded the engineering
spaces.  The sinking had nothing to do with the reactor.

    Scorpion, on the other hand, had no observer present.  No reason
    of loss has been given to the public.

Scorpion was in very high speed transit, westbound in one of the submarine
transit lanes, when she struck a previously uncharted undersea mountain.
The speed of the collision was "in excess of forty miles per hour" (probably
closer to sixty).  It was the very high speed that had rendered her
(acoustically) blind; unable to see the obstacle in her path.  True, no
observer was present, but a lot of people did get to hear the result.  The
"days spent searching for the lost sub" were just to avoid revealing how
accurate our tracking capabilities were.  All the Navy brass knew within the
hour, exactly what had happened and exactly where.

Staffing Nuclear Submarines

Martin Minow, DECtalk Engineering ML3-1/U47 223-9922 <minow%regent.DEC@decwrl.DEC.COM>
21-Oct-1986 1457
Disclaimer: a few months ago, my knuckles were rapped when I incorrectly
cited a study on airline safety.  Please be warned that I know absolutely
nothing about nuclear submarines and am using the ongoing discussion about
automatic controls for nuclear reactors (on submarines) only as a starting
place for a wider discussion.

From the discussion on Risks it seems that, while automatic controls may do
a satisfactory job of running the reactor in normal circumstances, people
will still be needed to run the reactor when the automatic controls

Adding automatic controls adds weight (and probably noise), making the
ship less effective. 

Adding automatic controls to a nuclear submarine's reactor frees personnel
for other tasks.  But, there isn't much else for them to do (they can hardly
chip rust on the deck), so they'll get bored and lose their "combat

Relying on totally manual control keeps the crew alert and aware of the
action of the reactor.  It also keeps them busy. 

In other words -- and I think this is directly relevant to Risks -- there
are times when external factors make it unwise to automate a task, even
when it can easily be done. 


An SDI Debate from the Past

"DYMOND, KEN" <dymond@nbs-vms.ARPA>
21 Oct 86 11:03:00 EDT
While looking something up in Martin Shooman's book on software 
engineering yesterday, I came across the following footnote (p.495):

    Alan Kaplan, the editor of Modern Data magazine, posed the question,
    "Is the ABM system capable of being practically implemented or is
    it beyond our current state-of-the-art ?"  The replies to this
    question were printed in the January and April 1970 issues of the
    magazine.  John S. Foster, director of the Office of Defense
    Research and Engineering, led the proponents, and Daniel D.
    McCracken, chairman of Computer Professionals against ABM, led
    the opposition.

It's startling that the very question that so interests us today was
put 15 or so years ago; to make it the exact question, all you have
to do is change the 3 letters of the acronym.  And this was 3 (?)
generations ago in computer hardware terms (LSI, VLSI, VHSIC ?) and
some indeterminate time in terms of software engineering (I can't
think of anything so clear-cut as circuit size to mark progress in
software).  International politics, however, seems not to have
changed much at all.

I'll try to track down those articles (Modern Data no longer exists
having become Mini-Micro Systems in 1976), but in the meantime can anyone 
shed light on this debate from the dim past ?

(BTW, Shooman comments "Technical and political considerations were finally
separated, and diplomatic success caused an abrupt termination of the
project." p. 498)

System effectiveness is non-linear

Dave Benson <benson%wsu.csnet@CSNET-RELAY.ARPA>
Mon, 20 Oct 86 16:01:06 pdt
I agree with Anon that overall system effectiveness is non-linear:

  >If 1000 missiles strains the system to the point that it can only
  >stop 800, why would anyone think it could stop more when the number of
  >missiles and decoys is doubled, straining the system's ability to
  >identify, track, and destroy missiles at least twice as much?

The more reasonable (and conservative) assumption is that the SDI system
would stop ZERO missles when faced with, say, 2000 targets.  Case in
point is revision n of the US Navy Aegis system -- seems that being
designed to track a maximum of (17) targets,  when there are (18)
targets the computer software crashed.

Any engineered artifact has design limits.  When stressed beyond those
limits, it fails.  We understand this for civil engineering artifacts,
such as bridges.  Clearly this is not well understood for software
engineering artifacts.

<Schuster.Pasa at Xerox.COM>
Tuesday, 21 October 1986 10:39-EDT
After reading the recent ARMS-D on the Stealth subject, particularly the
interesting message from Bryan Fugate where he says that "stealth
fighters and bombers have already gone into production", and in light of
some of the recent aircraft collisions, I couldn't help but wonder if
anyone has adequately considered the air traffic control consequences of
not being able to get a radar fix on a large, rapidly moving aircraft in
a high density air traffic area?

For that matter, what about ground-radar-assisted-landing in poor
visibility at a military base?

Sometimes you want an aircraft to present a GOOD radar target. As I was
writing this I thought of the answer, I guess. The stealth aircraft
would have to have a strong beacon turned on in these circumstances. I
guess it's easy to recreate a good target this way. All I can say is
that the beacon had better be working in the circumstances I described.

Missing engines & volcano alarms

Martin Ewing <mse%Phobos.Caltech.Edu@DEImos.Caltech.Edu>
Tue, 21 Oct 86 13:41:58 PDT
We visited New Zealand a few years ago and went to the major skiing area
on the North Island (the name escapes me).  It is built on the slopes of
an active volcano.  There were prominent warnings for skiers of what to
do in case of an eruption alarm.  (Head for a nearby ridge.  Don't try
to outrun the likely mud/ash slide coming down the hill.) 

How do they get the alarm?  There is an instrument hut at the lip of the
crater connected to park headquarters by a cable.  The instruments
measure some parameter(s) or other.  (heat, acceleration, pressure, ?) 
When something crosses a threshold, the warning alarms on the ski slopes 
are set off automatically. 

In fact, someone admitted, what would probably happen is that the
explosion would destroy the hut and cut the cable.  Loss of signal is
probably as good a diagnostic as anything else.

I can imagine a display on the DC-10 instrument panel inscribed with the
outline of the aircraft.  Little red lights come on when you lose
continuity on a wire to an engine, aileron, etc. - like what happens
when you leave your door open on a Honda Civic.  What you do with this
data is another matter. 

Please report problems with the web pages to the maintainer