The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 4 Issue 52

Thursday, 26 February 1987

Contents

o B-1 plagued by problems
PGN
o Computer loses bus
Mark Biggar
o Human errors
Brian Randell
o Possessed terminal?
pom
o Entertainment risks
Walt Thode
o Automatic Call Tracing for Emergency Services
James Roche
Charley Wingate
o "Active" car suspensions
Graeme Dixon
o Altitude-Detecting Radar
Matthew Machlis
o Re: Results of a recent security review
Andrew Klossner
o Re: Sherizen talk; auto-landing
Eugene Miya
o Air Traffic Control, Auto-Land
Scott E. Preece
o Risks of autopilots (and risks of solutions)
Bill Janssen
o Another difference between electronic control in cars and fighters
Brent Chapman
o Re: Hurricane Iwa
Scott Dorsey
o Info on RISKS (comp.risks)

B-1 plagued by problems

Peter G. Neumann <Neumann@CSL.SRI.COM>
Thu 26 Feb 87 21:12:09-PST
(From the Stanford Daily, 26 Feb 87, in the "Dateline" section, compiled
from the wires of the AP and the LA Times/Washington Post News Service)

WASHINGTON -- Government investigators said Wednesday that as many as half
of the new B-1 bombers at a Texas air base have been grounded in recent
weeks because of nagging technical problems and that the aircraft's
shortcomings may persist well into the next decade, contrary to public
statements by the Air Force.  During hearings before subcommittees of the
House Armed Services Committee, Chairman Les Aspin, D-Wis, said the bomber's
heart -- its defensive electronics system -- not only fails to jam enemy
radar signals but actually serves as a beacon illuminating the B-1 as a
target.  Government Accounting Office officials ... testified that the
problems with the $28.3 billion bomber program, especially the critical
defensive electronic countermeasures (ECM), are far more serious than Air
Force officials have acknowledged.  GAO officials also predicted that the
Air Force will have to ask Congress for substantially more money in coming
years to repair and upgrade the bomber.


Computer loses bus

Mark Biggar <markb%sdcrdcf.UUCP@JOVE.CAM.UNISYS.COM>
Thu, 26 Feb 87 10:58:17 pst
    The Los Angeles bus system (also known as the Rapid Transit
District (RTD)) uses a computer to keep track of its buses.  The computer
knows which bus is traveling which route at what starting time.  The
computer also has the complete time schedule information.  The computer
can be used to estimate the position of any bus using this information.

    On Feb. 25 the driver in trouble radio alarm was set off on bus
#181, the computer was asked where the bus was and the LAPD was notified.
The LAPD patrol unit that responded to the call could not find the bus, so
they called in more units.  They still could not find the bus and asked for
a helicopter to help search for it.

    After about a hour, the bus driver was located in the drivers'
lounge at the bus yard.  The bus was in the repair yard and the repair crew
had accidentally set of the alarm.  It turned out that the driver had
assumed that the repair yard had told the RTD computer that the bus was out
of service, and the repair yard thought that the driver had told it.

Mark Biggar       Unisys - System Development Group, Santa Monica
{allegra,burdvax,cbosgd,hplabs,ihnp4,akgua,sdcsvax}!sdcrdcf!markb
markb%sdcrdcf@CAM.UNISYS.COM


Human errors

Brian Randell <brian%kelpie.newcastle.ac.uk@Cs.Ucl.AC.UK>
Thu, 26 Feb 87 19:11:49 gmt
There was a very interesting documentary on BBC TV in their QED series here
last night, entitled "A Fall from Grace: Patterns of Human Error", which
contained quite a bit of material of relevance to RISKS.

     The programme (Yes, that is how even I spell it when it isn't intended
for a computer!) used as its principal illustrations the 1977 collision of
two Jumbo jets at Tenerife airport, and the task on making tea! Various
types of human error were described, and discussed with several experts,
including Professor Jim Reason, (Dept of Psychology, Univ. of Manchester),
Dr. Ivan Brown (Applied Psychology unit, Medical Research Council), and
David Embrie (sp?), an ergonomist from Aston University.

     The principal thing which I learnt, to my shame, from the programme was
that psychologists seem have done a lot of useful study of the many different
types of errors that even highly trained human beings make when exercising a 
sophisticated skill.

     Some comments I jotted down:

(1) One could learn much of relevance regarding the errors made in carrying
out highly skilled safety-critical tasks, such as piloting an airplane, or
in a nuclear control room, from studying the errors made in inconsequential
tasks (hence the tea making example, which when you think about it, does
involve considerable, albeit informal, training) - i.e., the underlying causes
seem to be similar, even if the consequences of errors are grossly
different.

(2) With a highly skilled activity, you make more mistakes if you do it
consciously. This particularly applied to "sequencing" errors, such as
missing or repeating a step. For example, if you are so following a
well-known sequence of actions, on mental auto-pilot, and then suddenly
become aware of your actions, there is a good chance of your resuming the
sequence at the wrong place.

(3) When you have learnt two similar sequences, you have, so to speak,
constructed two similar competing "action daemons" - one can acccidentally
switch to the wrong one. This was illustrated with an account of how one of
the pilots (who was very skilled, and spent much time training others) was
thought to have reverted to a pattern of actions which he was familiar with
from simulator training, which did not quite match reality in the way that
the pilot was supposed to communicate with the air traffic controller.

(4) One characteristic of error-proneness concerns the notion of "field
dependence" - some people have difficulty, and are slow, at picking out a
relevant object from a complex field of view - a sort of mental tunnel
vision, for which there are standard tests. Pilot training would probably
select such people out, but drivers might well suffer from this, and the
idea of using the standard tests to decide whether someone should have a
driving licence was unlikely to be acceptable.

     The programme also contained a well-illustrated, though to me rather more
expectable, account of the problems of designing interfaces to try to minimise
human error - mainly illustrated by control room design, with reference to
Three mile Island. 

     Today I telephoned Prof Reason, and had a very interesting chat with
him. We have arranged that he will come and give a talk to our Systems
Research Group, and I have been given the following interesting sounding
reference: New Technology and Human Error (ed. J. Rasmussen, K. Duncan, & J.
Leplat), Wiley 1983, to which he contributed several chapters. My hope is
that his ideas on error classification might be of relevance to the sorts of
problems that s/w (and h/w) engineers suffer from which result in residual
design errors in complex computer systems.

      My apologies to readers for whom all this is familiar - perhaps I should 
have taken Psychology 1, after all!

Brian Randell - Computing Laboratory, University of Newcastle upon Tyne

  ARPA  : brian%cheviot.newcastle.ac.uk@cs.ucl.ac.uk
  UUCP  : <UK>!ukc!cheviot!brian
  JANET : brian@uk.ac.newcastle.cheviot


Possessed terminal?

<pom%under.s1.gov@mordor.s1.gov>
Thu, 26 Feb 87 09:48:41 PST
Since WWN is usually quite authentic, I will entertain some speculation on
the topic. While 'electric currents' cannot be ruled out (an incompetent
electrician could put full voltage into the 'ground' and many countries use
220V rather then US style 110V), the most likely explanation seems be the
good old 'VDT stress'. (VDT = Video Display Terminal).

There is a big volume of writing on the topic and even some solid
information. Radiation (soft x-rays from CRT) was often blamed but informed
consensus (which agrees well with my own observations) is that stress is
psychological. Introduction of any 'computerised system' could be an
enormous trauma to people who were never exposed to the computers (even when
all you do is replace IBM Selectrics with the word processors <=:: I have
seen secretaries crying and thinking of quitting or even retiring from the
workforce for good).

The proper procedure for converting to computer system is as follow:

 1) Introduce terminals to the workplace, while doing the 'real work' with
     the old, manual system.
 2) Put some games on the machine and let people play with VDTs (perhaps
    after hours or during lunch breaks).
 3) Introduce e-mail, first just as alternative to phone call or memo,
    so that it is not NEEDED to get the job done.
 4) When everybody (as measured by volume of use) is comfortable with
    the system, put some work-functions on the new system.
 5) After a month or two, convert the rest. (You may find out that some
    people will quit or ask for a transfer, even with slow transition;
    those requests for transfer should be honored from the start.)

I wonder how may 'mysterious accidents' that occur after new 'sophisticated 
safety systems' (e.g. in nuclear power plants) are introduced are caused by
ignoring these simple common sense rules.
                                                       pom


Entertainment risks

<thode@nprdc.arpa>
26 February 1987 0736-PST (Thursday)
I generally favor the broad interpretation of what gets into this list.
In that spirit, I offer the following item from the San Diego Evening
Tribune of Feb. 25.  It may or may not be "computer risk" related:

     "Los Angeles (AP) - Dialing a telephone is sometimes a gamble, as
callers found out when they got "Dial-Porn" instead of state lottery
information because of a switched line.
     "Pacific Bell fixed the problem yesterday, but before that callers
heard a suggestive recorded message from a sultry-voiced woman when they
sought Saturday's winning lottery numbers.
     "Maria de Marco, who manages 976 prefix lines for Pacific Bell, said
it wasn't known whether the switch was a prank or an accident..."

     [Since most telephone systems are now extensively computer controlled,
     this certainly falls into the class of human misuse of computers.  PGN]

In the same paper there was another item, also datelined Los Angeles,
that described the confusion of some Lawrence Welk compact disk buyers
when their mislabeled and mispackaged CDs turned out to contain the
soundtrack from a movie about former Sex Pistols member Sid Vicious.

     [I decided not to delete this paragraph on technology-irrelevance
     grounds.  It could have been a computer-related problem!  PGN]

If a computer is involved in these instances, it would appear to be one 
with a sense of humor.
                                --Walt Thode (thode@NPRDC)

      [Even if one wasn't involved, it has a sense of humor!  PGN]


Re: Automatic Call Tracing for Emergency Services

James Roche <roche@rochester.arpa>
Wed, 25 Feb 87 10:29:45 est
[...]
As a firefighter in Monroe County (where Rochester is located) I can offer some
insight to the troubles of the 911 system here. The 911 dispatch center here
provides services for more than 80 county-wide emergency agencies (police,
fire, ambulance). That is reportedly more than any 911 center in the US.
Among the problems encountered are that fire district boundaries don't match
postal service boundries which don't match ambulance service boundries which
don't match town boundries, etc. Therefore when the ALI indicates a particular 
address is in Town X is is necessary for the dispatcher to turn to another 
screen and determine which police/fire/ambulance agencies are to be dispatched.

Other problems encountered with 911 include the fact that the entire county
is served by more that one phone company. Most of the county is served
by Rochester Telephone which has set up its computers to route all
Monroe County 911 calls to the 911 dispatch center. There are however
locations in the county which are served by New York Telephone. NYT has
set up its computers to route the 911 calls from Monroe County to the
Syracuse dispatch center (70 miles east). The dispatcher on the Syracuse
end must recognize the call is from Monroe County and route the call to
the Monroe 911 center. There are also areas of the county served by
Ogden Telephone. I don't know how they handle the 911 calls.

  >(Incidentally, the county Commissioner of Public Safety took this
  >occasion to complain about duplicate street names within the county ...

While it is not clear that eliminating duplicate street names would
have avoided the above problem, it would eliminate other problems.
Not all emergency calls received by the 911 dispatch center come in
via the 911 number.  Many calls are still received on the old 7 digit
number.  When a call comes in on that number the pertinent data for
the address is not displayed. The dispatcher must then determine
which one of the many duplicates the caller is referring to. I recall hearing
6 fire departments dispatched one day to a false alarm on East Avenue
because there are multiple East Avenues within the county. The call
was received on the 7 digit number and the caller gave incomplete
information to the dispatcher (intentionally I imagine). The county feels
that it must continue to provide service on the 7 digit number since
for many years phone stickers were distributed with that 7 digit number. Also
the residents the the areas served by New York Tel are encouraged to use
the 7 digit number to avoid delays by going through Syracuse.

Jim Roche                                         UUCP: rochester!roche 
University of Rochester Computer Science Department Rochester, NY 14627


Re: Automatic Call Tracing and Addresses

Charley Wingate <mangoe@mimsy.umd.edu>
Thu, 26 Feb 87 23:44:03 EST
Here in Howard Co. Md., the county government took a big step years ago and
renumbered all the addresses so that with in some quanta the street numbers
are not only unique, but they also give the physical location of the property. 
This has done wonders for getting the FD to the right place.  Unfortunately...

"Laurel" phone exchanges lie in four counties; Laurel zip codes in three.
This makes dialing 911 a bit of an adventure because you had better know
which county you are in.  Sometimes even this doesn't help.  One zip code
was believed by the counties to lie entirely in P.G. county, when in fact a
small piece of it lay in Montgomery County.  This meant that these people
got no county services-- no fire, no trash, nothing.  After years of bickering,
the Postal Service cut the gordian knot and created a new zip code just for
these people.  The moral: "Garbage in, Gospel out" doesn't just apply to
computers; they can "bless" information that never came near them!

C. G. Wingate    U of Maryland, Dept. of Computer Science, Coll. Pk., MD 20742


"Active" car suspensions

Graeme Dixon <graeme%cheviot.newcastle.ac.uk@Cs.Ucl.AC.UK>
Wed, 25 Feb 87 19:14:57 GMT
Since the discussion has once again come around to the use of computers in
cars the "... most important single automotive advance since the accelerator
pedal ..." may be of interest.

There have been a number of articles in British motoring magazines (Car Oct
86, Fast Lane Jan 87) over the last few months describing the Lotus "Active"
suspension. This consists of a replacement for the normal passive suspension
of dampers, springs, and anti-roll bar, by a sensing system, computer, and a
set of hydraulically controlled actuators. The sensors return the cars
relative movement and driver inputs, and the computer adjusts the actuators
to compensate.  The resulting handling characteristics are by all accounts
superb - no roll, no understeer, no oversteer, just perfectly balance
handling.  Various parameters used by the computer may be adjusted to
provide different levels of ride, prompting one of the writers to speculate
that it would "be possible to build a schizophrenic car with His and Hers
alternative handling at the flick of a dashboard switch."

One of the more contentious claims of the system is that "it is truly
fail-safe". By providing a "get-you-home stand-by suspension" computer
failure does not render the car unusable. One of the articles even describes
the cars behaviour when the system is "dumped" as the car is negotiating a
corner - the car switches suddenly from neutral handling to oversteer
prompting the driver to think one of the rear tyres had punctured. What they
didn't try was the effects of over compensation though!

It will be a few years before active suspensions appear in cars (Lotus are
intending to use it in their supercar the Etna which they are currently
developing), but given that Lotus have been recently bought by GM, and a
number of rivals (notably Mercedes-Benz) are developing similar systems, then
this should provide another fertile area for discussion when the time comes....

Graeme Dixon


Altitude-Detecting Radar

<mmachlis@ATHENA.MIT.EDU>
Wed, 25 Feb 87 16:10:34 EST
     It is true that Mode C capability costs a bit of money, but I think
the majority of people who own planes could afford the extra $1500 or so,
especially considering the added safety.

     As to 3-D radar, it would be very nice but I am under the impression
that it is quite impossible, realistically speaking, with the present
technology.  A professor here at MIT who flew for the Navy for 20 years
told me it is reasonable to make altitude-detecting RADAR, but that it
is only economically reasonable for tracking a single target at a time.
Aircraft such as the F-14 and F-16 can track several targets at once, but
those systems are very expensive and have MTBF averages of only several
hours of operation because of their complexity.


Re: Results of a recent security review

Andrew Klossner <andrew%hammer.tek.com@RELAY.CS.NET>
Wed, 25 Feb 87 12:59:02 PST
    "Fifth problem: A program can be created with "OWNDIR"
    privileges.  While it is running, it has all the privileges
    associated with the account on which it resides."

Interesting ... did they license the use of this invention from AT&T,
the patent holder?

  -=- Andrew Klossner   (decvax!tektronix!tekecs!andrew)       [UUCP]
                        (tekecs!andrew.tektronix@csnet-relay)  [ARPA]
                        Tektronix, Inc., Wilsonville, OR

     [... and will someone sue AT&T if, after a license is duly obtained, a
     devastating Trojan horse is perpetrated using this flaw/feature ?  PGN]


Re: Sherizen talk; auto-landing

Eugene Miya <eugene@ames-nas.arpa>
Thu, 26 Feb 87 16:23:03 PST
I think an apology is in order.  I sent my notes to the CPSR Sherizen talk
to Peter (not with the intention of posting to the net).  Locally, we are
trying to have discussions on security trying to forego problems of
discussing security both when it was tried in unix-wizards (and it
subsequent list) and info-vax (for the VMS side).  Although the Sherizen
meeting of CPSR was open, our other meetings are not (they are not
classified either).

Regarding auto-land: I don't know if I would trust such a system yet.  I
know few pilots who would not feel at least a little uncomfortable.
Actually, I think systems like this would be great Darwinian tests of
AI.  The posting implied we control everything.  This is not true.
The plane is not everything, there are other planes and obstacles out there.

Put the developer on the plane, let his or her system land the plane.
If the plane survives, the developer goes on to create their next system.
(Might not be enough, but a good first cut.)
Similar tests for things like MYCIN, etc. can be used (infect using a blood
disease, developer then must trust system for diagnosis ;-).  Sound a little
too real world?  We know less about the real world than many think.  Thinking
is not enough.

--eugene miya

    [In the past I have been extraordinarily careful about not including
     obviously personal messages without explicit permission.  In this
     case I clearly goofed.  The message somehow seemed to be of general 
     interest and addressed to a large list...  And it was getting late.
     Sorry, Eugene...   PGN]


Air Traffic Control, Auto-Land

Scott E. Preece <preece%mycroft@gswd-vms.ARPA>
Wed, 25 Feb 87 09:13:49 CST
Use of automated landing also would leave the crew more free to spend its
time looking for things out of the ordinary -- unreported traffic, patterns
of air movement, the effect of the wind on preceding traffic, the overall
condition of the aircraft -- that automated systems are not good at detecting.

scott preece, gould/csd - urbana, uucp: ihnp4!uiucdcs!ccvaxa!preece


Risks of autopilots (and risks of solutions)

Bill Janssen <janssen@MCC.COM>
Wed, 25 Feb 87 17:02:01 CST
In Risks Digest 4.51, Matthew Machlis questions whether there may be
risks of pilots losing their flying skills, due to flying for extended
periods on autopilot.

At a conference last year, I spoke to folks from a major commercial aircraft
manufacturer, who were concerned about the same thing.  (One of the
speculations about KAL 007 was that the pilots just `lost track' of what
they were doing.)  This firm had the thought of dividing the cockpit in two,
using one half for flying the real airplane, and the other half for a
training simulator.  The pilots would trade off acting as `system monitor'
and practicing `real' problem flying.  The problem with this solution was
loss of orientation, along the lines of "Oh, damn, I just put the plane in
an unrecoverable spin; well, restart... that's funny, nothing seems to
happen...  Ohmygod, I'm sitting on the *real* side".
                                                            Bill


Another difference between electronic control in cars and fighters

Brent Chapman <chapman%mica.Berkeley.EDU@BERKELEY.EDU>
Thu, 26 Feb 87 17:03:14 PST
Another key difference, which to me seems just as important as the
maintenance issues already mentioned, is that cars (generally!) aren't
fitted with ejection seats.  A driver can't punch out when things get weird.

Also, cars tend to be operated in much more crowded conditions.  Usually in
fighters (except possibly during takeoff and landing), you really don't have
to worry about what your plane will come crashing down on, because most
operations (both real and training) occur over very sparse areas.  In a
runaway car, on the other hand, you stand a significant chance of wreaking
considerable havoc among other vehicles travelling in your vicinity, as well
as bystanders and property near the roadway.
                                                       Brent


Re: Hurricane Iwa (RISKS DIGEST 4.51)

Scott Dorsey <kludge%gitpyr%gatech.csnet@RELAY.CS.NET>
Thu, 26 Feb 87 12:24:31 est
    Winds from Hurricane Iwa passed through a small mountain pass, gathered
pressure from the narrow slit, and knocked out power lines which carried
power to most of Central Oahu.  They also did serious damage to an army base
on the exiting winds side of the pass, opening warehouses filled with emergency
supplies like sardine cans, or ripping the prefabricated buildings away from 
their foundations while leaving the contents sitting.
    The base was without power for three weeks, and without water for about
two.  The Mayor of Honolulu asked the military for help, and they refused
(being much harder hit than the civilian community, mainly due to the damage
at this base).  There were several scathing editorials in the Advertiser,
but the military did not really release any information about the extent of
the damage.
    The island of Kauai was worst hit.  Although the generating system was not 
heavily damaged, there was no way to restart the generators without power, as
no one had foreseen that all the turbines would go down at once.  The Navy sent
a nuclear submarine from Pearl Harbor over to Kauai to provide power for the 
starters, but by the time it arrived, the engineers had restarted the system,
using almost a hundred automotive batteries.

>  In the afternoon, winds started rising, and the Weather Service issued a
>  Hurricane Watch, then quickly a Warning, but still didn't have a precise fix
>  on Iwa, nor accurate information on speed or direction.

    At about noon, state employees were sent home, schools were cancelled.
I was in downtown Honolulu at 3:00 or so.  All the shop windows were taped up,
and a cold, dry breeze blew through the streets, picking up bits of paper and
carring them around.  There was not another soul on the streets, and I was not
able to get back to the base, as all the buses had stopped.  I eventually got
someone to come down and pick me up, and we were the only car on the roads.
I don't know much about the damage to Honolulu, being stuck on base for a while
because I had no form of transportation (tree fell on car).

>  [This could be a separate story in itself, but suffice it to say that the
>  Civil Defense Emergency Broadcast system didn't work.  Besides all the TV
>  stations, all the radio stations---except one--- went off the air that
>  night.  The single radio station that had an operating emergency generator
>  was running "on automatic", playing religious music.]

  Nope.  Radio station KGU was on almost all the time, on their standby 
generator.  They were off for a few hours when their antenna was damaged, but
brought the transmitter (at the studio site) back up with a long wire dipole.
At first they were calling various authorities, but after the phone went out,
they just sat around and played music, complaining about the weather.

  I don't think that the extent of the damage to the military installations
was ever revealed, so you can probably say you saw it first here.  It doesn't
have much to do with risks from computer systems, but it does have a bit to
do with risks to computer systems, as well as anything else that uses 
electricity.  At least, I know my PDP-11 did go down at the time.

Scott Dorsey   Kaptain_Kludge             ICS Programming Lab,  Rich 110,
    Georgia Institute of Technology, Box 36681, Atlanta, Georgia 30332
    ...!{akgua,allegra,amd,hplabs,ihnp4,seismo,ut-ngp}!gatech!gitpyr!kludge

Please report problems with the web pages to the maintainer

Top