The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 10 Issue 57

Sunday 4 November 1990


o $6.3 million electric bill
Mark W. Schumann
o Drug RISKS to software ??
Simon Rosenthal
o More of what really goes wrong
John Rushby
o Automotive electronic engine control failure modes
Dave Davis
o Re: Laxness, not modernization, at fault in train wreck
Chuck Weinstock
o Train Wreck and Weight Estimates
o Re: Risks of Modernization
Gerald Stafleu
o Re: Access to gov't computer files
Brinton Cooper
o Call for papers — ACM SIGSOFT '91
Nancy Leveson
o Info on RISKS (comp.risks)

$6.3 million electric bill

Mark W. Schumann <>
Fri, 2 Nov 90 19:09:53 EST
                     Whoops!  CEI bills customer $6.3 million
                          by Sabrina Eaton, Staff Writer
                           The Plain Dealer [Cleveland]
                             Friday, 2 November 1990

  Even when here cupboard was bare during the Depression, 76-year-old Faye
  Starman always paid her electricity bills.  That was before Tuesday, when she
  opened here $6.3 million Cleveland Electric Illuminating Co. bill.  "It's a
  good thing I'm a calm person because I could have had a heart attack or
  stroke," said Starman of Newbury Township.  The bill, payable by Nov. 9, also
  gave her the option of making a $300,021 budget payment--far more than her
  home on Ravenna Rd. is worth.

  CEI spokesman Mike Lumpe said the company's records were corrected
  immediately after her complaint and blamed the slip on mistakes made during a
  switch to a new computer billing system.  "The employee had tried to enter
  $63 in our computer, made a mistake, and the computer filled in the extra
  zeroes automatically," Lumpe said.  "People are learning the new system, and
  there are bound to be one or two mistakes, especially when you send out
  750,000 bills every month."  Lumpe said the company also has heard from an
  Ashtabula County family that received a $2 milion bill, and said he hoped "we
  don't have any more of those floating around out there."

  Only a few of CEI's industrial customers have monthly bills in the $1 million
  range, but Lumpe said he did not believe there were any customers with bills
  as high as $6 million.

I am curious about two obvious problems here.  First of all, shouldn't there
have been a "sanity check" for a reasonable range of billing amounts on the
data entry application?  But secondly, what modern billing system would require
manual entry of the bill amounts in the first place?  Here we aren't dealing
with a risk of computerization; this is over-reliance on manual procedures
without adequate controls.

Mark W. Schumann  3111 Mapledale Avenue, Cleveland 44109 USA
Domain:  ...!mailrus!!ncoast!catfood

Drug RISKS to software ??

Tue, 30 Oct 90 14:51:51 EST
Two occurrences today left me wondering of the RISKs to correct software of
substance abuse by its authors. Today, I attended a presentation by my employer
(Concurrent Computer Corporation) of its "drug free workplace" policy.  In
common with other companies which do business with the government, Concurrent
is now mandated by Federal law to maintain a drug-free workplace - establishing
and publicizing a policy, offering help or disciplining offenders, and
reporting people convicted of drug offences to the government.

Also today, the Boston Globe(10/30/90) reported a Supreme Court decision
upholding an award to a programmer who was fired for refusing to provide a
urine sample. "

    "The court declined to review a lower court ruling that
     the programmer, Barbara Luck, was not in a safety-related
     job for the railroad company (Southern Pacific) and could
     not be required to take the drug test"

Southern Pacific's argument was that federal railway labor laws (requiring
mandatory drug testing) should apply to all employees.

The description of the programmer's job as "non safety related" led me to think
of all the reports of failure of mission critical software which are regularly
described in RISKS, and to speculate on whether the checks and balances usually
present in the software development process (debugging, code reviews, Software
Quality Assurance procedures) could ever be insufficient to catch software
errors caused by drug-altered states of consciousness.  Like many of us, I'm
certainly aware of the mood-altering and productivity-diminishing effects of
such drugs as alcohol (although in my experience equally negative effects can
be caused by such things as inadequate sleep, or by pulling an all nighter).

I tend to doubt whether software malfunctions could ever be attributed
solely to drug abuse on the part of its authors. But ... I'd be
interested to know if there have ever been any reports to the contrary.

Simon Rosenthal, Concurrent Computer Corporation, Westford, MA 01886

More of what really goes wrong

John Rushby <>
Mon 29 Oct 90 22:19:22-PST
>From Aviation Week, October 22, 1990

    ``Engine Shutdown, Computer Glitch Delay YF-22A Test'' page 115

``Officials had been hoping to raise the landing gear---for the first
time---on the second flight Oct. 11, but two computers disagreed
with one another and prevented gear retraction.  The gear is extended
by electrical commands without computer intervention.''

    ``Rockwell/MBB X-31 Makes Second Flight, Reaching 20,000-Ft.
           Altitude, Mach 0.6'' page 117

``The electronic flight controls had several internal disagreements on the
second flight that led to the use of reversionary modes, but the disagreements
cleared when the computers were reset.  Testing was continued and the aircraft
landed in normal mode.''

Automotive electronic engine control failure modes

dave davis <>
Tue, 30 Oct 90 08:18:41 EST
i was a product design engineer with Ford in the late 70s through early 80s and
I can relate a story of electronic engine control failure that was subtle and
embarrassing to us.

The first models to be equipped with our EEC-1 system were those with the big
V-8s, including fleet sales, such as taxis, rental agencies, and police.  We
had a system of reporting difficulties that dealer mechanics had with
diagnosing and repairing our cars, so that engineering and management could be
aware of the need to correct design or manufacture.  One of our regional dealer
reps. reported that a police car equipped with EEC-1, had experienced complete
loss of engine operation, that is, ignition, each time the officer keyed the
mike button on his radio!

After a lengthy investigation, our people at the dealer found that a single
ground strap from the hood to the firewall had been bent during manufacture or
shipping at wasn't doing its job.  This had apparently allowed enough RF to
leak into the engine compartment, to be picked up by the EEC-1 system's wiring,
and fed into the control module (a Motorola computer chip) where it played hell
with its operation.

After much discussion at the highest levels, we were forced to recall every
police vehicle sold to date with EEC-1, because of safety--real safety, not
theoretical.  (The engineer responsible for the sensors recommended early on
that the cars had to be recalled.  He complained to me 'They stopped inviting
me to the meetings.')  The last time I visited the office where the EEC
engineers were located, they had a prototype of a new control unit with through
the wall capacitors--about 36 of them.  This was required to filter the RF, for
police buyers and anyone else who carried a powerful radio transmitter.  All of
this in a business where one is a hero if you can shave a nickel out of
producing several thousand cars.

Dave Davis, MITRE Corporation, McLean, VA                me := disclaimer.all

Re: Laxness, not modernization, at fault in train wreck

Chuck Weinstock <weinstoc@SEI.CMU.EDU>
Mon, 29 Oct 90 16:24:48 EST
In Risks Digest 10.56 various people commented that blaming
modernization as a cause of the train wreck was not correct.  For
instance, Roy Smith said:

      "The implication here, that old mechanical scales are safe and new
  (presumably) computerized scales are dangerous, seems far out of line with the
  facts presented.  The crash occurred because the train was overloaded and
  because they only had half the braking capacity they thought they had, both
  bits of misinformation due to plain old poor operating practices, not fancy
  modern scales."

Actually, I have no complaint with using more modern, computerized
scales.  My comment was that if the railroads had not gone to the more
expensive computerized technology, they might still have had scales at
more locations including at Mojave.  Certainly an old technology
mechanical scale at the top of the hill would have helped more than a
modern technology scale at the bottom of the hill.  This contributes
to the cavalier attitude towards of risks of their actions exhibited
by many of the people involved.

Certainly there were other contributing factors in the crash such as
those pointed out in my original post and expanded upon (and added to)
by Martin Minow and others.

Chuck Weinstock

Train Wreck and Weight Estimates

Mon, 29 Oct 1990 xST
In a recent message, it was pointed out that it was only through the
intervention of a human that the correct number of locomotives
was chosen to pull the load in question, even though the load weight
had been badly underestimated.

However, in this case, I wonder if that fellow basing the load on his
own estimates, rather than the provided information, didn't actually
contribute to the accident (though with the best of intentions).  If
he had believed the underestimates presumably provided to him, he
would probably have assigned fewer locomotives to the train, and the
fact that the load was "too heavy" would have likely been noticed by
the crew almost immediately, or at least well before they reached the
grade in question.  By providing the "proper" number of locomotives
for the actual load, the train was able to proceed easily to the
grade where the lack of sufficient braking became critical.

Of course, his using "correct" estimates, rather than the "low" estimates, only
became a problem due to the complex of other interacting factors involved with
the incident.

Re: Risks of Modernization (Trona Train Troubles)

Gerard Stafleu <>
Tue, 30 Oct 90 08:55:04 EDT writes:
> Do these trains run with no normal air brakes on every car?

That seems to be a more relevant question here than electronic scales.
I thought it was Standard Operating Procedure that train cars should be
able to out-brake the locomotive(s).  If they cannot do so, and the
locomotive starts to out-brake the cars, the cars will all pile neatly
on top of the locomotive.  This will indeed stop the train, but not in a
manner according to specs.

In other words, the total weight being pulled by the locomotives does
not (or should not) have any influence on the braking capacity needed
for each locomotive.  Each locomotive should be able to stop itself, as
should each car.

This concept is not unfamiliar to computer people:  while global error
detection and correction is desirable, that does not mean that each
module should not also do its own checking and correcting.

Gerard Stafleu   (519) 661-2151 Ext. 6043   BITNET:   gerard@uwovax

Re: Access to gov't computer files (Sullivan, RISKS-10.56)

Brinton Cooper <abc@BRL.MIL>
Mon, 29 Oct 90 23:28:50 EST
John Sullivan reported on a Freedom of Information request for
statistical info on real property in NYC.  The city resisted by offering
it in a million sheets of hardcopy form for about $10,000 .  A court
ruled that NYC had to provide it on magtape at a cost under $100.

John calls attention to a " concern about possible problems involved in making
too much computer data available..."

For many years,  NYC has been making property transaction records
available, quarterly, on magtape to anyone who will pay the nominal
copying fee.  They are used on PC-based, Novell-networked type systems
(and others, no doubt) by people who do many title searches for
themselves or as a service-for-fee for others.

The manual searching of titles in file cabinets at City Hall would be
prohibitively expensive, considering the size of the database.  A manual
title search involves beginning from the most recent deed and tracing
the sequence of sales as far back as possible, going from one "liber and
folio" to another, to another, etc.

It's difficult to imagine how, in a city the size of New York, such a
necessary activity could be carried out without the widespread use of
automated searches.  It's equally difficult to imagine that the manual,
labor-intensive method would be less prone to make undetected errors
than would the computerized system.


call for papers — ACM SIGSOFT '91

Nancy Leveson <nancy@murphy.ICS.UCI.EDU>
Mon, 29 Oct 90 17:45:06 -0800
                            CALL FOR PAPERS

                            ACM SIGSOFT '91
                     Software for Critical Systems

                        New Orleans, Louisiana
                         December 4-6, 1991

Computer systems are beginning to affect nearly every aspect of our lives.
Examples include programs that control aircraft, shut down nuclear power
reactors in emergencies, monitor hospital patients, and execute banking
transactions.  Although such programs offer considerable benefits, they also
pose serious risks in that we are increasingly vulnerable to errors and
deficiencies in the software.

The SIGSOFT '91 conference seeks papers on all aspects of quality in critical
systems.  A critical system is a system that must exhibit, with very high
assurance, some specific qualities such as safety, reliability,
confidentiality, integrity, availability, trustworthiness, and correctness.
The conference will focus on such topics as architectures, design
methodologies, languages, analysis techniques, and processes that can increase
the likelihood that a system exhibits its required qualities.

Papers will be judged on relevance, significance, originality, correctness, and
clarity.  Papers will be read and evaluated by the program committee and must
not be under consideration (or published) elsewhere in the same or similar
form.  Papers are limited to 6,000 words, with full-page figures counting as
300 words.  A paper that significantly exceeds this limit is likely to be

Authors should submit 6 copies of the full paper to:

   Peter G. Neumann, Computer Science Laboratory, SRI International,
   Room EL-243, 333 Ravenswood Ave., Menlo Park, CA 94025

Persons submitting papers from countries in which access to copying machines is
difficult or impossible may submit a single copy.  Submissions should be
received by May 3, 1991 and should include a return mailing address.  Authors
will be notified of acceptance or rejection by July 12, 1991.  Full versions of
accepted papers must be received in camera-ready form by August 30, 1991.
Authors of accepted papers will be expected to sign a copyright release form.
Proceedings will be distributed at the conference and will subsequently be
available from ACM.

  Mark Moriconi             Nancy Leveson                 Peter Neumann
  SRI International         Univ. of California, Irvine   SRI International 

    David Barstow          Schlumberger
    Dines Bjorner          Technical University of Denmark
    Marie-Claude Gaudel    Universite de Paris - Sud
    Jim Horning            DEC Systems Research Center
    Bill Howden            University of California, San Diego
    Hermann Kopetz         Technical University of Vienna
    Carl Landwehr          Naval Research Laboratory
    Bev Littlewood         City University, London
    Leon Osterweil         University of California, Irvine
    David Parnas           Queen's University
    Fred Schneider         Cornell University
    Vicky Stavridou        University of London
    Martyn Thomas          Praxis, Inc.
    Walter Tichy           University of Karlsruhe
    Elaine Weyuker         NYU Courant Institute

Please report problems with the web pages to the maintainer