The RISKS Digest, Volume 5 Issue 29

Peter G. Neumann

The RISKS Digest
Volume 5 Issue 29

Saturday, 15th August 1987

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

RISKS submissions: PGN
Lack of user training = legal liability? — Computer SNAFU Ruled a Rights Violation: Rodney Hoffman
London Docklands Light Railway: Mark Brader
Software and system safety: Nancy Leveson
New safety MIL-STD: Nancy Leveson
Info on RISKS (comp.risks)

RISKS submissions

Peter G. Neumann <Neumann@csl.sri.com>

Sat 15 Aug 87 18:12:52-PDT

I submit that your submissions over the past weeks have driven me into
submission.  The quality of contributions and self-control exhibited by
contributors has dropped off radically.  Various complaints have been
received recently that I have suddenly become far too generous in including
too much irrelevant, unsound, incoherent, and otherwise marginal material.
Well, I currently have a HUGE backlog of messages for consideration — 30 in
the last four days — but all are mostly minor variants of previous
contributions, and many of them will not emerge.  (Perhaps the recent deluge
is merely a summertime phenomenon?)  Well, due to my own heavily overloaded
schedule, RISKS will slow down for the next two weeks, when I will be
limited to a few short shots at NETland.  But please keep on submitting the
GOOD stuff — I'll get to it eventually.  And please excuse the slowdown.

By the way, I erred in including the MITRE notice in RISKS-5.28.  Please
don't expect me to do it again.

The responses to Denning/Saltzer/Hardie are running about three-to-one for
nonsecrecy about security flaws, with all sorts of caveats, hedges, special
cases, etc.  Logic tells you that you'd better know when — and how — YOU
are vulnerable.

The certification issue is not generating much enthusiasm either way.
(Perhaps certification is also needed for computer system users who have
authorization to do act dangerously or fraudulently.)  The legal aspects of
screwing up may be a driving force toward certification to protect system
developers (or procurers?) if a big liability suit gets won by the injured
parties.  (The Rogan case that follows is small potatoes.  By the way, Donn
Parker notes that the Equity Fund case, with direct losses of $200 million,
is actually estimated at $2 billion total losses if indirect costs such as
stockholder losses are included.)

The standards issue for system safety is warming up again, but is also not
likely go generate too much enthusiasm.  (If you care, see the concluding
item in this issue.  It also implies all sorts of risks.)

Lack of user training = legal liability?

Rodney Hoffman <Hoffman.es@Xerox.COM>

14 Aug 87 11:07:01 PDT (Friday)

                                          [Also submitted by Frederick Bingham]

Excerpted from the Los Angeles Times, Aug. 13, 1987:

          COMPUTER SNAFU RULED A RIGHTS VIOLATION
    Wrong Man Repeatedly Detained on Murder, Robbery Charges
                   By Jack Jones

The Los Angeles Police Department violated the constitutional rights of
a Michigan man by continuing to list him in a nationwide crime
information computer system as wanted for murder and robbery, even after
it was clear the real suspect was using his name, U.S. District Judge
Robert Kelleher has concluded.

Terry Dean Rogan, 29, will now go to Los Angeles Federal Court to seek
monetary damages ....  Kelleher found that [the two] Los Angeles Police
Detectives... do not share liability with the city, even though on
several occasions they re-entered Rogan's name in the National Crime
Information Center system and failed to add new descriptive information
that would have established Rogan as the wrong man.

The city failed, Kelleher ruled, to properly train and supervise the
officers in the constitutional protection aspects of using the crime
center system and the necessity of adding more accurate information when
it becomes available.  The actual suspect, Alabama state prison escapee
Bernard McKandes,... began passing himself off as Rogan in 1981....

Rogan ... was first arrested in October, 1982.... He was held in jail
until a fingerprint check showed he was not the man being sought.
Nevertheless, ... Los Angeles detectives put his name back into the
computer, neglecting to add qualifying information, and he was arrested
four more times — mostly after routine traffic stops.  On two
occasions, he was taken into custody at gunpoint, handcuffed and jailed.
It was not until early 1984 that Rogan's name was removed from the
computer system — after a Saginaw News reporter told [one of the L.A.
detectives] that McKandes ... was back in prison in Alabama.
McKandes... subsequently was convicted of the California murder and
robbery charges.
                     [See ACM Software Engineering Notes 10 3
                     (July 1985) for some further background.  PGN]

London Docklands Light Railway; Northern Line's Dot-Matrix Indicators

Mark Brader <msb@sq.com>

Thu, 13 Aug 87 12:44:03 EDT

The June issue of the British magazine Modern Railways carried this item:

     'UNAUTHORISED TESTS' CAUSED DLR CRASH

     The Managing Director of Docklands Light Railway Limited, Cliff Bonnett,
     has said the accident which occurred at Island Gardens station on
     10 March (Modern Railways, May) was primarily caused by unauthorised
     tests, carried out before required modifications had been carried out
     at the southern terminus.  The train, which ended up overhanging from
     the elevated track after crashing through buffers, would have been
     'arrested' if the protection system 'in its full and modified form'
     had been installed.  The train was being driven manually.

Incidentally, on the same page of the magazine is this:

     DOT MATRIX PROBLEMS

     A marked improvement in the performance of the Northern Line's
     gremlin-infested dot-matrix train indicators is promised by the
     autumn, but modified software cannot be satisfactorily commissioned
     for a year, says London Underground, while a new central computer
     at the line's Coburg Street, Euston control centre is awaited.
     Meanwhile, some indicators have lost the minutes-to-train-arrival
     feature, displaying only the order of train arrival.

Mark Brader, Toronto, utzoo!sq!msb

Software and system safety

Nancy Leveson <nancy@commerce.UCI.EDU>

Thu, 13 Aug 87 10:49:41 -0700

Two weeks ago I taught a 3-day continuing education class on software safety
at UCLA.  The makeup of the class says some interesting things about the
awareness of software safety issues in the U.S.  I was pleasantly surprised
to have 40 people enrolled (the average class size of software engineering
classes there is about 25).  Half the class was from outside California
which means their management was willing to invest money in sending people
across the country to take the class (implying some awareness and commitment
to the problem — the class itself also was not cheap).  It was also
interesting to note that although the majority of people came from the
aerospace industry (including someone from Morton Thiokol), there were 3
medical device manufacturers (all said that their attendance was directly
related to the Therac 25 incident — these accidents that you read about in
risks do have an effect, especially when lawsuits and media publicity are
involved); 2 commercial aircraft manufacturers and a manufacturer of
aircraft engines; the Air Force, Army, and Navy; a couple of firms that do
safety analysis on a contract basis; and one in entertainment (Walt Disney).

Many in the class identified themselves as safety engineers [see the
following message], but there were also software engineers and a fair number
of people who identified themselves as "software safety engineers" or
"software system safety engineers."  I was especially curious about the
software safety engineers and asked a few questions.  All but one had
previously been system safety engineers and had acquired this title within
the past few months.  One had been a software engineer previously and had
become a "software safety engineer" very recently.
                                                       Nancy Leveson

[Now I know where all the mickey-mouse computer systems are coming from.  PGN]

New safety MIL-STD