The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 12 Issue 50

Tuesday 15 October 1991


o TRW misreports local taxes
Mark Seecof
o ATM Doesn't Catch Cash Cache Problem
Ed Miller
o Re: buggy software
David Parnas
o Risks of genetic engineering?
Michael Pilling
o Electronic thermostat failures
Ralph Palmer
Mary Shafer
Bob Wilson
Nancy Leveson
o Info on RISKS (comp.risks)

TRW misreports local taxes

Mark Seecof <>
Tue, 15 Oct 91 09:48:31 -0700
According to the Wall Street Journal Monday 10-15 page B1, TRW has decided to
purge local tax delinquency info from the files of consumers residing in four
New England states.  You will recall the Norwich, CT, local tax info errors; it
seems that similar errors have been made in the files of consumers living in
several other communities.  TRW blames the problem on a lax sub-contractor.
Many other people blame the problem, more broadly, on TRW's apparent
unwillingness to verify input or accept corrections...  Consumers say it's
difficult to reach anyone at TRW who even appears capable of influencing the
data in a file, and further report that TRW's personnel refuse to rectify
errors even when the TRW folks' attention is drawn to them.

I heard a radio report (just a headline, really) this morning that TRW will
provide "free copies" of credit reports to some (of their New England?)
consumers, in a PR move.

I'll seize this moment (do not attempt to adjust your display) to suggest that,
darn it, Congress should require ALL credit reporting agencies and the like to
notify each consumer they report on every time they issue such a report, except
under a search warrant with a secrecy injunction.  The agencies should be
permitted to batch such notifications (e.g., must notify within 3 weeks and may
include more than one notification in the same envelope).  The cost of the
notification can be absorbed into the cost of the triggering report (honestly,
the cost would not exceed 25 cents/notice what with bulk mail rates and reports
usually cost $5-$10 to the requester so it's not a big burden).  Every
notification should include the name, address, and telephone number of the
original report requester and a copy of the report as issued (not some opaquely
coded extract, such as the disclosures Equifax is famous for--the NSA could
hardly decode them; they are MUCH harder to understand than the ones given to
paying customers).  Agencies should be liable for statutory damages in the
amount of $2500 or proven economic damages if greater plus reasonable
attorney's fees in any event if they fail to correct any substantive error in a
credit report within 30 days of written notification (with corrected reports
sent to anyone who got an erroneous one).
                                                Mark Seecof <>

    [WSJ article also noted by Will Martin <wmartin@STL-06SIMA.ARMY.MIL>.  PGN]

ATM Doesn't Catch Cash Cache Problem

Ed Miller <>
Mon, 14 Oct 91 17:34:17 PDT
I went to extract cash from the ATM at a nearby branch of my bank.  Instead of
the twenty dollar bills normally issued by the machine, I was given one dollar
bills. My account transaction indicated that the ATM believed it had given me
twenties. (Had I been given twenties, the number of bills would have been
correct.) RISKS of garbage in, garbage out? RISKS of computers that can not
"read" their output?

Since I was actually at the bank branch and since they were open I went in to
have my account corrected. One other customer had the same problem at the same
ATM and was in line ahead of me. After the bank personnel had taken the ATM
off-line, I asked several questions. I learned that the money put into the
machine comms from U.S. Federal Banks in sealed containers. The local bank
employees can neither open or inspect the contents of the containers. Since the
bank had already paid the Fed for the cache, the bank appeared to be the loser
in the situation, unless they can convince the Fed that they owned the problem.
The bank employees did not seem to know of a process by which they could report
this problem to the Fed. RISKS of a security system that does not allow a human

Ed Miller  415/336.4278

Re: buggy software (RISKS-12.49)

David Parnas <parnas@qusunt.Eng.McMaster.CA>
Mon, 14 Oct 91 21:38:33 EDT
James B. Shearer ( writes:, "So far as I know no one is
required by law to buy the products of Mr. Mitchell's company.  If mature
adults wish to buy buggy software I do not see why this should be any concern
of Mr. Parnas."

As far as I know no one is required by law to buy an electrical appliance.
Nonetheless, every country that I know requires appliances to meet certain
minimal standards.  Nobody is required by law to buy a car, but we do require
cars to meet certain minimal safety standards.  Nobody is required by law to
fly in a commercial aircraft but we all expect that those vehicles and their
pilots will be produced and/or trained in a professional way.  Moreover, the
manufacturers of all of these products are all expected to take responsibility
for the things that they sell.  When a defect is discovered in my car, the
manufacturer is required to issue a recall notice and to repair that defect
without cost to me.  He cannot simply announce an upgrade and try to
sell it to me, not if the problem is a real defect.

We could use the "mature adult" excuse to get rid of all of these regulations,
but we would all be worse off for doing so.  Your apartment could be
destroyed because one of your "mature adult" neighbours bought an appliance
that was not properly designed.  Your child could be injured because one of
your "mature adult" neighbours bought a car with defective brakes.  Further,
every time you bought one of those products you would have to determine its
safety for yourself, whether you knew enough to do so or not.

Those who object to the suggestion that software products should be subject to
safety requirements and that software manufacturers should be held responsible
for the results of any negligence seem to believe that we are asking for
special treatment of software.  Au contraire!  We are asking that software be
treated like other products, produced by registered or licenced engineers, and
that software manufacturers be treated like other manufacturers.  Now, because
of the supposedly non-physical nature of software, programmer's products seem
to have special exemption.  If cars were as buggy as the software on the market
today, the automobile manufacturers would have long ago been sued into

Mr. Shearer goes on to write, "A real risk is that laws will be passed
requiring people to use certain crackpot programming methodologies which
purport to be better than existing practice but which for some strange reason
people refuse to adopt voluntarily."

I can assure Mr. Shearer that we are all against "crackpots".  The problem is
that it is difficult to tell the difference between crackpots and visionaries.
Years ago I read a biography of Steinmetz, a visionary who thought that
mathematics could be used to analyse the behaviour of electrical power lines
and was considered a nutty theoretician by some practical people.  Today, those
who do not understand and use the methods that he proposed are considered
incompetent.  I am sure that there were some real crackpots around in
Steinmetz' time, but I am certainly glad that his views prevailed.

David L. Parnas

Risks of genetic engineering?

Dr Chocberry) 15 Oct 91 04:01:07 GMT
I would like to know what some molecular biologists, notably gene splicing
specialists, have to say about this.

As I computer scientist, I know that even though we perfectly understand every
single line of our programs, we often make mistakes in even small programs, and
it is very difficult if not impossible to generate a bug free program of any
medium to large size.

In genes, we do not even understand most of the basic instructions, and yet we
are trying to make new programs using these instructions.  Since DNA has far
more single instructions in it that the average program, I wonder just how
error prone genetic engineering is and how if at all you can protect against
the effects of latent errors in the code?

Electronic thermostat failures (Bukys, RISKS-12.49)

Ralph Palmer <rpalmer@Think.COM>
Tue, 15 Oct 91 09:31:52 EDT
I have also had an electronic thermostat fail 'ON'.  I have a RobertShaw model
T1020.  I've had a problem with my transformer on my oil fired furnace, it only
supplied ~5 volts to the thermostat, not the ~10 that the thermostat needed.
Since the voltage is low, the thermostat draws down the 9 volt backup battery.
I have observed two failure modes.  If the battery dies when the furnace is
off, the thermostat fails off.  However if the battery fails when the heat is
on, the heat doesn't shut off!  I was fortunate to find this out on a saturday
afternoon and shut down the furnace when the house was only ~90F.

I feel that the best design would be a fancy digital set back thermostat as the
primary control unit, defaulting to a mercury thermostat in case of power loss
to the control unit .  Until I find such a unit I'll stick with my round
Honeywell mercury thermostat, turn down the heat at night myself, and wake up
to a cold house.
                Ralph Palmer

Thermostat failure mode (Bukys, RISKS-12.49)

Mary Shafer <>
Tue, 15 Oct 91 07:53:29 PDT
I had a standard, non-computerized perfectly simple Honeywell round thermostat
break about 20 years ago.  We came home to find the house at about 95 deg and
the heater still running.

I think you overestimate the reliability of "old-fashioned" systems.  I've
never had the trouble with any of the electronic thermostats that I had with
the gems with the mercury switches.

By the way, many of us learned long ago to either turn off the heat when on
vacation or to have someone check the house every day.  Frozen water lines,
stuck toilets, broken thermostats--no computer technology needed to mess things
up.  A co-worker came home from a week of houseboating on Lake Powell to
discover that her house had caught fire.  Fortunately a neighbor noticed and
the damage was limited to the kitchen.  The fire department says that it was
probably the toaster oven.  Apparently these are known for suddenly immolating
themselves and the surrounding kitchen.

Mary Shafer  DoD #0362  NASA Ames Dryden Flight Research Facility, Edwards, CA

Re: Thermostat failure mode (Bukys, RISKS-12.49)

Bob Wilson <>
Tue, 15 Oct 91 10:35:20 CDT
Both the failure mode and the fact that it failed at all are things to worry
about, but your furnace almost surely does have a thermal shutdown. You don't
say what fuel it uses, or whether it distributes the heat through forced air,
hot water, or steam. I assume from your use of the word "furnace" and the
phrase "to run and run" that it is not any common form of electric heating.
Every plenum chamber (for hot air heat) should have an overheat switch. They
have been required by code wherever I have lived, and even if your local code
does not require it I am sure that so many do that it is much cheaper to
include them in all cases. Typically the switch is a bistable disk type
thermostatic switch mounted to the plenum chamber.  I am not sure that boilers
(for hot water or steam) have switches actuated by temperature but they do have
overpressure switches which in most cases will accomplish the same thing.

Bob Wilson, University of Wisconsin, Department of Mathematics


Nancy Leveson <nancy@ICS.UCI.EDU>
Mon, 14 Oct 91 17:16:26 PDT
    [With the deadlines approaching for reduced-rate early registration and
    for assured hotel space, it seems appropriate to run this reminder.  PGN]

                        4-6 December 1991
                    Fairmont Hotel, New Orleans

Computer systems are increasingly affecting nearly every aspect of our lives.
They control aircraft, shut down nuclear power reactors in emergencies, keep
our telephone systems running, monitor hospital patients, and execute financial
transactions.  Although such critical systems offer considerable benefits, they
also pose serious risks in that we are increasingly vulnerable to flaws and
other deficiencies in the software, hardware failures, and effects of
accidental and intentional computer misuse.  SIGSOFT '91 focuses on the
problems in building and validating critical software.

   General Chair:  Mark Moriconi, SRI International
   Program Co-Chairs:  Peter Neumann, SRI International
                       Nancy Leveson, Univ. of California, Irvine
   Travel Arrangements:  Johnette Hassell, Tulane University
   Registration and Coordination:  Judith Burgess, SRI International
 phone: (415) 859-5924, FAX (415) 859-2844

   Program Committee:
       David Barstow       (Schlumberger)
       Dines Bj/orner      (Technical University of Denmark)
       Marie-Claude Gaudel (Universite de Paris - Sud)
       Jim Horning         (DEC Systems Research Center, Palo Alto)
       Bill Howden         (University of California, San Diego)
       Hermann Kopetz      (Technical University of Vienna)
       Carl Landwehr       (Naval Research Laboratory)
       Bev Littlewood      (City University, London)
       Leon Osterweil      (University of California, Irvine)
       David Parnas        (McMaster University, Canada)
       Fred Schneider      (Cornell University)
       Vicky Stavridou     (University of London)
       Martyn Thomas       (Praxis, Inc.)
       Walter Tichy        (University of Karlsruhe)
       Elaine Weyuker      (NYU Courant Institute)


Welcome and Introduction: 8:45am - 9:00
  Mark Moriconi, SIGSOFT '91 Chair (SRI International)
  Peter G. Neumann, Program Co-chair (SRI International)

Session 1: 9:00 - 10:15, Carl Landwehr, Chair

  Formal Verification of Algorithms for Critical Systems
     John Rushby (SRI International), Friedrich von Henke (University of Ulm)

  State-Based Model Checking of Event-Driven System Requirements
     Joanne M. Atlee and John Gannon (University of Maryland)

  Open Discussion

Session 2: 10:45 - 12:30, Dines Bj/orner, Chair

  Rigorous Development Using RAISE
     Bent Dandanell (CRI, Birker/od, Denmark)

  Specifying and Verifying Requirements of Real-Time Systems
     K.M. Hansen, A.P. Ravn, and Hans Rischel (Tech. University of Denmark)

  A Systematic Kernel Development
     J.F. S/ogaard-Andersen, C.O. Rump and H.H. Lovengreen (Tech. Univ. Denmark)

  Open Discussion

Session 3: 2:00 - 3:45, Elaine Weyuker, Chair

  The Infeasibility of Experimental Quantification of Life-Critical
  Software Reliability
     Ricky Butler and George Finelli (NASA Langley Research Center)

  PANEL: The Limits of Probabilistic Risk Assessment

     Bev Littlewood (City University, London)
     David Parnas (McMaster University)
     Martyn Thomas (Praxis, Ltd)
     Ricky Butler (NASA Langley Research Center)
     John Musa (AT&T Bell Labs, Whippany, NJ)

    The Butler/Finelli paper argues that ultra-high reliability cannot be
    validated directly from testing, nor can be it demonstrated by appeals
    to software fault-tolerance.  What progress might we reasonably expect
    to make toward numerical risk assessment of life-critical software?

Session 4: 4:15 - 5:30, Martyn Thomas, Chair

   PANEL: The Confused World of Standards for Critical Software

   Martyn Thomas (Praxis, Ltd)
   Peter Neumann (SRI International)
   Mike DeWalt (FAA)

   This session will explain and assess current government regulation such as
   British MoD DEFence STANdard 00-55/56 and various security criteria (e.g.,
   U.S. TCSEC, European ITSEC, Canadian CTCPEC).  What role should such
   standards play?  What should be mandated?


Session 5: 9:00am - 10:30, Fred Schneider, Chair

  Comparing Fault Detecting Ability of Testing Methods
     P.G. Frankl (Polytechnic University), E.J. Weyuker (NYU Courant Institute)

  An Exception Handling Model For Parallel Programming and its Verification
     Valerie Issarny (IRISA/INRIA)

  Open Discussion

Session 6: 11:00 - 12:30

   INVITED TALK:  Human Error in Design
       Henry Petroski (Duke University)
         Author of the widely-acclaimed books ``To Engineer is Human: The
         Role of Failure in Successful Design'' and ``Pencil''

Session 7: 2:00 - 3:30, Victoria Stavridou, Chair

  A Real-Time Transition Model for Analyzing Behavioral Compatibility of
  Telecommunications Services
     E.J. Cameron and Y-J Lin (Bellcore)

  Programming and Verifying Critical Systems by Means of the Synchronous
  Data-Flow Language LUSTRE
     C. Ratel (Merlin-Gerin), N. Halbwachs and P. Raymond (IMAG/LGI)

  Open Discussion

Session 8: 3:45 - 5:30, Mark Moriconi, Chair

Invited Presentations on Practical Experiences:

  Validation of Critical Flight Controls
     Jim McWha (Chief Engineer in charge of 777 Flight Controls, Boeing)

  Reliable Software for the 4 ESS Switch
     Michael Meyers (AT&T Bell Labs)

  A Case Study of the THERAC-25 Accidents
     Nancy Leveson (U.C. Irvine)

Session 9: 8:00pm - 9:30pm, Evening Poster Session


Session 10: 8:30am - 10:30, Hermann Kopetz, Chair

  Stepwise Design of Real-Time Systems
     Reino Kurki-Suonio (University of Technology, Tampere)

  On Satisfying Timing Constraints in Hard-Real-Time Systems
     Jia Xu (York University) and David Parnas (McMaster University)

  Automated Analysis of Bounded Response Time for Two NASA Expert Systems
     C-K Wang, R-H Wang, D-C Tsou, J.C. Browne, and A.K. Mok (University
     of Texas, Austin)

  Open Discussion

Session 11: 11:00 - 12:30

PANEL: Future Directions, Nancy Leveson, Chair

Adjournment at 12:30

 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

AIR TRANSPORTATION.  Delta Airlines is offering 40% off RT Coach fares within
the U.S., 35% Canada, 5% off already discounted fares.  Call 1-800-221-1212,
ask for Special Meeting Network, refer to file ref no. V18006.  Valid for
travel from Nov. 30 to Dec. 10.  7-day advance purchase required.

 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

     SIGSOFT '91 -- Software for Critical Systems
    Fairmont Hotel, New Orleans, Dec. 4 -- 6, 1991

Name _________________________________________________________
Affiliation __________________________________________________
Address ______________________________________________________
City, State and Zip __________________________________________
Phone (and FAX) ______________________________________________
Email address ________________________________________________
ACM or SIGSOFT Membership No. ________________________________

Registration Fees
                            Before        After
   Category                    Nov. 1        Nov. 1
   ACM or SIGSOFT Member         $280        $330
   Non-Member                    $330        $380
   Full-time Student             $180        $230

To pay by credit card, circle one:    AMEX        VISA       MC
Name on card __________________________________________________
Card number ___________________________Exp. date ______________
Signature _____________________________________________________

Make checks payable to SIGSOFT '91 in U.S. dollars.  Fees include 3 continental
breakfasts, 2 lunches, and the Proceedings.

Dietary requests:  Vegetarian ______  Kosher ________

Judith Burgess / EL266, SRI International, 333 Ravenswood Ave.,
Menlo Park, CA 94025, USA

For further information, contact Judith Burgess,
telephone: (415) 859-5924, FAX (415) 859-2844, EMail


 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

    SIGSOFT '91 -- Software for Critical Systems
          New Orleans, Dec. 4 -- 6, 1991

Name _________________________________________________________
Affiliation __________________________________________________
Address ______________________________________________________
City, State and Zip __________________________________________
Phone (and FAX) ______________________________________________
Date/Time of Arrival _________________________________________
Date/Time of Departure _______________________________________

Room Rates (subject to taxes):

Circle one:                Single $99         Double/Twin $119

RESERVATIONS: 1-800-527-4727 or 1-504-529-7111

To guarantee your reservation by credit card:

Circle one: AMEX     MC     Visa    Carte Blanche  Diners Club

Name on card _________________________________________________
Card number ___________________ Exp. date ____________________
Signature ____________________________________________________

These rates apply from Nov. 29 through Dec. 8, subject to availability.
Reservations should be received 30 days in advance to ensure availability, but
later reservations will be accepted as possible.  A deposit for the first night
must accompany your reservation to guarantee it for arrival after 6:00pm.
Cancellations must be made 24 hours in advance.

The Fairmont Hotel, University Place, New Orleans, LA 70140, USA
 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

Please report problems with the web pages to the maintainer