The Risks Digest

The RISKS Digest

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Volume 9 Issue 41

Saturday 11 November 1989

Contents

o Stuffing the electronic ballot box (again)
PGN
o BART and the Bartered-Computer Commuters
o Coral reef ruined by poor user interface design?
Jim Helman
o Re: Computer errors and computer risks
Jerome H Saltzer
o Computer used to find scoflaws in Boston
David desJardins
o Reference on the early history of Ada -- killing reliably
Eugene Miya
o Info on RISKS (comp.risks)

Stuffing the electronic ballot box (again)

"Peter G. Neumann" <neumann@csl.sri.com>
Sat, 11 Nov 1989 13:30:42 PST
Charles Schwab & Co spent more than $100,000 to conduct a nationwide poll on
program(med) trading, with two widely advertised free 800 phone numbers set up
to record the pro and con votes.  Apparently someone at Wells Fargo (which has
a high-profile program trading subsidiary) set up an autodialer to vote YES
continuously, which was detected by their programmed monitoring.  (` "First we
have program trading.  Now we have program dialing," quipped Senior Vice
President Hugo Quackenbush. ')  In 12 hours, there were 12,191 votes, 65.3%
for, 34.7% against.  [Source: San Francisco Chronicle, 11 Nov 1989, pp. B1-B2.]

   [By the way, I observe that one call every twenty seconds for 12 hours would
   account for the entire difference between the pros and cons.  One call every
   six seconds would have accounted for ALL of the pro calls.  We might suspect
   that an autodialer could easily have skewed the results -- although perhaps
   others on both sides were also using the same strategy.]


BART and the Bartered-Computer Commuters

"Peter G. Neumann" <neumann@csl.sri.com>
Sat, 11 Nov 1989 13:46:02 PST
The Bay Area Rapid Transit has for years been having troubles with its computer
systems -- both old and new.  The old one is supposed to handle 64 trains at
once, but is barely able to handle 39.  The new system -- almost ten years late
when it went on-line three weeks ago -- was supposed to be able to handle 108
trains, but becomes seriously overloaded at 10.  Consequently, all of the
long-planned extensions are on hold.  (The system supplier is Logica.  The old
system is still being used, at least as a backup when the new one is being
tested.)

"An outside consultant that designed three control systems for the Paris Metro
and was brought in by [BART General Manager Frank J.] Wilson has already
determined that even if the new system were able to perform as promised, it is
obsolete."

[A summary of "BART's New Computer Called `Complete Failure'", By Harre
W. Demoro, San Francisco Chronicle, 11 Nov 1989.]

An earlier article indicated that the BART system has been sorely pressed in
trying to accommodate the great increase in traffic due to the loss of the Bay
Bridge in the earthquake, and many train cars are out of service.


Coral reef ruined by poor user interface design?

Jim Helman <helman@isl.Stanford.EDU>
Fri, 10 Nov 89 19:09:37 -0800
The captain of a ship which ran aground on an environmentally
sensitive live coral reef off the US coast attributed the accident to
a confused officer and a bad user interface.  Apparently, an officer
incorrectly changed course because the steering mechanism on the ship
operated in the opposite fashion from most such controls.

Whether or not the control was computerized, this demonstrates quite
dramatically the potential dangers of inconsistent user interfaces.  I suppose
the company which built the control could be held liable for the poor design.

There may be parallels with GUI issues yet to come.  What if a company has to
use an inconsistent interface because of patent restrictions?  Could the
company still liable for the errors of a confused user?

Excerpts from the UPI article follow:

    ``I think he made an error. I think he was confused,'' Capt.
    Zdravko Beran told Coast Guard investigators on the first day
    of testimony before a board of inquiry. ``He told me that
    (navigational) light was dead ahead and then he wanted to turn
    a few degrees to the port (left).''

    Instead, the officer, Zvonko Baric, turned the ship to the
    right, or starboard, causing it to run aground on sensitive
    coral in the Fort Jefferson National Monument, Beran said.

    The steering mechanism must be turned in the opposite
    direction of the intended course -- the reverse of how most
    such devices operate, Beran said.

    Asked how future such accidents could be avoided, Beran said
    the federal officials could require ships to go around rather
    than through the national monument.

Jim Helman, Department of Applied Physics, Stanford Univ., Stanford, CA 94309


Re: Computer errors and computer risks (RISKS-9.40)

Jerome H Saltzer <saltzer@src.dec.com>
10 Nov 1989 1613-PST (Friday)
In RISKS DIGEST 9.40, Randy Davis says,

> Numerous stories have been reported on this list under the title
> "computer error" and "computer risk," that seem to me to have nothing
> essential to do with computers, and a great deal to do with very
> different issues.
> . . . I suggest the simple test above: Ask, can the identical
> problem can arise in the absence of computers?

I claim that it is not that simple.  In a traditional library, it was possible
to invade your privacy by making a list of all the books you have every checked
out.  All an investigator had to do was open every book in the library and look
to see if you had signed the card inside.  The information was publicly
available, but actually it was benignly protected by an enormous collection
cost, so noone every worried about it.

These days, if the library has a computerized circulation system, and it is not
designed with proper regard for privacy, it may be possible to get this
information in a few seconds with a single request.  Most computerized
circulation system protect this information and many installations
systematically delete it to avoid any possibility of such investigations.  Yet,
when the first circulation systems were designed, some people said that the
circulation information has "always been public" as an argument against
protecting the information in the computerized system.

The Registrar of Motor Vehicles in Massachusetts has long used the "it has
always been public" argument on car registration information as an excuse for
selling tapes to mass mailers.  But as I recall, I didn't receive that class of
junk mail back in the days when you had to personally visit the Registry and
copy the information out of their ledgers.

The point is that the speed and efficiency with which data can be processed by
computer can convert a neglible risk into one worth discussing.  Had the RISKS
forum been around then, I think the library debate would have been quite an
appropriate topic.  Ditto the Mass. Registry.

Finally, the availability of the computer to discover connections may tempt
people to create data bases that they wouldn't otherwise consider feasible.
When I saw the report in RISKS on the proposed child-abuse data base I assumed
that this was an example of a data base that probably wouldn't have been
proposed had computers not been available to implement it--especially the part
about identifying people who use multiple doctors.

Thus I am prepared to tolerate a certain amount of fuzziness in identifying the
edge of the computer-RISKS arena by our beleaguered moderator.
                                                                  Jerry Saltzer


Computer used to find scoflaws in Boston

David desJardins <desj%idacrd@Princeton.EDU>
Fri, 10 Nov 89 21:01:38 EST
<>When five out of six hits are human errors, imagine the complaints!
>
> It goes to show the importance of considering the total effect of a system
> change, not just the project at hand. It was a serious design error [...]

   I am very surprised at how much complaining there has been about
this error rate.  I think finding one stolen car in every six stopped
is a phenomenally *high* success rate.  It doesn't seem a major
problem to me to inconvenience five people for a relatively short
period of time in order to apprehend one serious criminal.  I would
think almost any other form of police work would involve a much higher
level of inconvenience to the public for the same level of success.
   For comparison of order of magnitude, look at neutron-activation bomb
detectors, to be installed in airports.  They are said to have something like a
3% false positive rate (and that is in controlled tests, so the real rate will
most likely be higher).  I don't have a number at hand for the actual rate of
bombs in checked luggage, but let's say that it is 1 in 30 million (I'm sure it
is less).  That corresponds to 1 million false positives for every true
positive.  *That* is a high rate of error.  And our society has chosen to spend
nearly a billion dollars on that system.
                                               -- David desJardins,     IDA/CRD


Reference on the early history of Ada -- killing reliably

Eugene Miya <eugene@eos.arc.nasa.gov>
Fri, 10 Nov 89 23:21:42 PST
I finally had a reason to dig thru the same box which had the following
reference in the history of Ada.  The quote comes from Pascal News #13,
December 1978, then published by the Pascal User Group.  This is a bound
journal (although basically Xerox(tm) copied sheets); it was unrefereed.  By
way on context, it should be recalled that the initial proposals for the new
DOD language (then refered as DOD-1, various "colors," and the progression:
Strawman, Woodenman, Tinman, Ironman, and then Steelman) had 17 proposals based
on Pascal (the last was based on PL/1, you can guess who proposed that 8).  So,
the Pascal community was thrust into the limelight.  (It was still a somewhat
obscure language at the time.)  The quote is authored by Andy Mickel, U MN,
then the editor:

  Latest News About DOD-1 (ADA or DOD0)     --Andy Mickel

  As we've told you in previous issues of Pascal News, the U.S. Department of
  Defense (DOD) has endeavored to procure a common programming language
  based on Pascal for all "embedded" computer applications -- computer systems
  attached to weaponry.  Reliable software should kill people reliably! .....
  [above description removed] ...colors: BLUE-Softech, GREEN-Honeywell Bull;
  RED-Intermetrics; and YELLOW-SRI International.

The article goes on for two more paragraphs, but Andy's comment about killing
was not taken lightly.  Pages 9-10.  So much for the history of programming
languages.

--eugene miya, NASA Ames Res. Ctr.
  formerly joint ANSI X3J9/IEEE P770 Pascal Language Standards Committee

Please report problems with the web pages to the maintainer

Top