The RISKS Digest
Volume 4 Issue 71

Sunday, 5th April 1987

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

Contents

Re: A real eye-catching headline — nuclear safety
Jerry Saltzer
Peter G. Neumann
Henry Spencer
A non-fail-safe ATM failure
Don Chiasson
Fumes from computers and other electronic appliances
Richard Thomsen
Open University Fire
Lindsay F. Marshall
Info on RISKS (comp.risks)

Re: A real eye-catching headline [David Chase, RISKS-4.70]

Jerome H. Saltzer <Saltzer@ATHENA.MIT.EDU>
Fri, 3 Apr 87 17:49:56 EST
>       "Inherently safe nuclear reactors"                      
>                                            [Add to the oxymoron list.  PGN]

Before assuming nonsense, one might try reading the article under the
headline.  It explores a series of design approaches with the common theme
that safety mechanisms should be driven by simple, passive, inexorable laws
of physics rather than being complex gadgetry in themselves.  (For example,
place the entire reactor system under water so that faults that would
usually produce loss-of-coolant failures tend to instead produce
too-much-coolant failures.)

Whether or not the specific technical ideas are competent I can't judge, but
the notion of designing safety measures that are simple and inevitable seems
something that people concerned with computer RISKS should want to ponder
rather than laugh at.
                        Jerry


Re: A real eye-catching headline

<Peter G. Neumann <Neumann@CSL.SRI.COM> [Edited]>
Fri 3 Apr 87 16:55:55-PST
Yes, indeed, I certainly agree that one should understand something well
enough before making light of it.  In fact, the IEEE Spectrum article is
quite significant.  Use of inexorable laws of physics is a marvelous idea --
if those laws are in fact complete, correctly understood, immutable, and
nonbypassable...  The principle is excellent in the small.  The practice may
not be so easy to guarantee in the large.  [See also Henry Spencer's message
below.]

I would like to add something that addresses not the inexorability, but
rather the limitations of the environment in the case of large-scale nuclear
power (to which this technique has not yet been applied):

   1. People are not infallible, and are certainly not "inherently safe".
      Incompent or careless people might make an "inherently safe"
      nuclear reactor ACTUALLY UNSAFE.  PBS' All Things Considered on 3 Apr
      87 concluded that BAD MANAGEMENT was probably the biggest source of
      problems.  Bad management is quite capable of rendering a system
      inherently unsafe, e.g., as a result of unwise cost-saving measures.
      (Philadelphia's Peach Bottom plant was just closed by the NRC; a
      surprise visit found the operators sleeping on the night shift.
      The PBS program also noted a safety system installed backwards.)

   2. The inexorable laws (in the small) may be circumvented under actual
      environmental conditions, i.e., to the system in-the-large — via
      accidents, sabotage, earthquakes, carelessness, and improper maintenance,
      as well as bad management and other human behavior noted above.

[  3. Despite claims to the contrary, nuclear waste disposal appears to be
      at least LONG-TERM RISKY, and may prove to be INHERENTLY UNSAFE.  There 
      appear to be no really appealing solutions in the long run, but that
      argument is beyond the scope of RISKS.  I toss it in simply to
      illustrate the holistic nature of the problem and the nonholistic
      nature of the assumptions of infallibility.  ]

It does seem that assumptions are being made about the INFALLIBILITY of the
technology.  I quote from the Spectrum article:

  "If a major system fails, for example, the core is flooded automatically
  with coolant that flows under immutable laws of gravity and thermohydraulics,
  not under propulsion by mechanical pumps and electromagnetic actuators."

If applied to nuclear power, does this ignore all sorts of fallibilities?
Are there not still combinations of mechanisms and components that might
fail, e.g., if the coolant suddenly springs a major leak, or if during
maintenance reliance on the "INHERENTLY SAFE" physical principles must
temporarily be circumvented, or if people do not always behave reasonably,
as assumed?  The notion that it is possible to design something that is
"100% reliable" UNDER ALL POSSIBLE CIRCUMSTANCES is clearly unrealistic.
But, even "99.9999% reliable" is not very good if the .0001% case can be
provoked accidentally or intentionally by a specific combination of
plausible circumstances (whether anticipated or unanticipated).  Of course,
A VERY REAL RISK LIES IN BELIEVING IN THE INFALLIBILITY OF TECHNOLOGY.

Nevertheless, the cited April 87 IEEE Spectrum article is worth reading, and
Jerry's points are very well taken.  For those technologies in which risks
can be substantially reduced by using homeostatic processes, that should be
encouraged.  (Although nuclear power has not yet been so based, the article
makes an important point that it should be!) A good example of homeostasis
is the human body, which is basically self-regulating — except that when it
breaks down, all bets are off.

(To the reader:  I know that Jerry doesn't believe in the infallibility of
technology.  I am not trying to shoot a straw herring in the foot.  This
message is by way of further discussion.)
                                                        Peter


Re: A real eye-catching headline

<pyramid!utzoo!henry@hplabs.HP.COM>
Sun, 5 Apr 87 16:46:06 pst
> IEEE Spectrum, April 1987:
>       "Inherently safe nuclear reactors"
>                                             [Add to the oxymoron list.  PGN]

Not so, actually.  The things actually exist, and the term accurately
describes them.  You could take a sledgehammer to the controls and nothing
much would happen.  U of T has one.  Apparently if you're the last one to
use it Friday afternoon, you just lock the door behind you and leave it
unattended for the weekend.  Unlike power reactors, the design is inherently
stable:  an increase in temperature causes a decrease in reaction rate, so
nothing you can do will make it overheat.  Unfortunately, the design does
not scale up well and hence isn't useful for power plants.

Henry Spencer @ U of Toronto Zoology {allegra,ihnp4,decvax,pyramid}!utzoo!henry

[The Spectrum article suggests that it COULD be useful for power plants... PGN]


A non-fail-safe ATM failure [Still one more interesting ATM saga!]

Don Chiasson <CHIASSON@DREA-XX.ARPA>
Thu 2 Apr 87 08:38:46-AST
I'd like to pass along one which happened to me and indicates the risk of
interactions between computers and mechanical components in automated
systems.  A few months ago I used an ATM to pay a bill.  At the end of the
transaction, the machine said to:
              "REMOVE CARD TO QUIT OR PRESS OK TO CONTINUE".
I was done, so I pulled out my card and started to leave.  I took a few
moments getting my credit card back in my wallet, then had one of those
"What wasn't that??" feelings.  The door on the ATM hadn't gone down.  The
mechanical switch which shows whether my card is or is not in the machine
had stuck and the ATM was patiently waiting for my next transaction.  I
aborted it by pressing a "CANCEL" button.  Had I not done that, anyone
passing by (this machine was outdoors) could have pressed a few buttons and
paid their own bills from my account or pulled out my daily cash limit.
Lesson: verify that the machine is doing its thing.
                                                    Don

    [Despite a RISKS moratorium on routine ATM stories, this one is worth
    including as an example of an uncompleted supposedly atomic transaction
    with nasty side-effects.  Another example of inconsistency between the
    state of the software and the state of the hardware was the THERAC 25
    therapeutic radiation device: you recall that the software thought it
    had switched the device to X-ray mode (1,000 rads), but the device was
    still in electron-beam mode (up to 25,000 rads) at the moment. 
    (See RISKS-3.9.)  PGN]


Fumes from computers and other electronic appliances

Richard Thomsen <rgt@LANL.ARPA>
Thu, 2 Apr 87 07:33:33 mst
Just as radon gas comes from cement walls and formaldehyde comes out of new
housing walls, carpets, and some modular furniture, there are gases that
are emitted from electronic appliances.  I do not know what they are, but
suspect they come from the plastic cases, and probably the circuit boards
and capacitors.

I know someone who is highly allergic to chemicals, and can smell these
gases.  They are more prone to be emitted when the computer is on, since
it is warmer.
                [This subject needs some real expert contributions.  PGN]


Open University Fire

"Lindsay F. Marshall" <lindsay%kelpie.newcastle.ac.uk@Cs.Ucl.AC.UK>
Sun, 5 Apr 87 11:29:51 gmt
Recently there has been considerable publicity given to a fire that took
place in a computer room at the Open University HQ. The fire destroyed a VAX
and all its back-up tapes that were stored in the machine room. The
interesting thing about this event was the various reports of the problems
caused by the loss of the filestore back-up.  Initial (non-trade) press
reports talked about people losing 15-20 years of research, this was then
whittled down to two to three years (in the trade papers) and eventually
seems to have come down to a couple of months as most people had personal
back-ups of critical data!!

Lindsay F. Marshall, Computing Lab., U of Newcastle upon Tyne, Tyne & Wear, UK

Please report problems with the web pages to the maintainer

x
Top