The RISKS Digest
Volume 3 Issue 51

Sunday, 7th September 1986

Forum on Risks to the Public in Computers and Related Systems

ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator

Please try the URL privacy information feature enabled by clicking the flashlight icon above. This will reveal two icons after each link the body of the digest. The shield takes you to a breakdown of Terms of Service for the site - however only a small number of sites are covered at the moment. The flashlight take you to an analysis of the various trackers etc. that the linked site delivers. Please let the website maintainer know if you find this useful or not. As a RISKS reader, you will probably not be surprised by what is revealed…

Contents

Computer almost created swing vote
Bjorn Freeman-Benson
Computer Sabotage of Encyclopedia Brittania
Rosanna Lee
F-16 software
Wayne Throop
Arbiter failures and design failures
Martin Harriman
Systems errors (hardware AND humans)
Bill Janssen
Re: Terminal (!) lockup
Roy Smith
Info on RISKS (comp.risks)

Computer almost created swing vote

Bjorn Freeman-Benson <bnfb@uw-june.arpa>
Sun, 7 Sep 86 10:44:01 PDT
Quoted without permission from the Seattle Times, Sunday Sep. 7, 1986:

AP, PHOENIX, Ariz. — Tuesday's primary elections in Maricopa County would
have been a mess if officials hadn't figured out that a compuer was set up
to give all Republican votes to the Democrats and vice versa.
    "If it had gone undetected, there would have a major, major problem with
the election," County Recorder Keith Poletis said Friday.  Poletis said that,
if the compuer hadn't been fixed, a race with three Republicans and one
Democrat would given the Democrat's votes to one of the Rebulicans.
    Votes cast for the remaining Republicans would have been zapped into the
void by the computer, because the software would find no Democratic opponents.
"The computer sorts the cards into two piles, and it was sorting the
Republicans into the Democrats' slots and the Democrats into the Republican
slots," Poletis said.
    A clerical error made the computer's cards were ordered was to blame for
the mix-up, said Joe Martina, director of the county's computer systems.  The
error was caught during the secretary of state's test of the country's cards
late Thursday.

End quote.

In my mind, this brings up an interesting question: should errors like this
be reported (1) to the general public and (2) to the software engineering
community?  I think the answer to (2) is yes — the more data we have on the
types of errors that occur involving computers, the better grasp we will have
on solving them.  However, for (1), I see this arguement:
    Con - The testing procedures before acceptance caught the error.
        - The public will just lose faith in computers.
    Pro - The public should know, because what if the testing hadn't?
    Con - The public in general is not knowledgeable about computers and
      software, and the general press is sensationalist.  Thus any
      case reported will not be studied in the necessary depth.
    - An analogy with civil engineering: should the public know that
      the first design for a bridge collapsed during testing?  Or is
      it just enough to know that the final bridge works?

                        Bjorn N Freeman-Benson


Computer Sabotage of Encyclopedia Brittania

Rosanna Lee <rosanna@CSL.SRI.COM>
Sat 6 Sep 86 18:09:51-PDT
Chicago Tribune [From San Jose Mercury News, Friday, Sept 5, 1986]

LAID-OFF WORKER SABOTAGES ENCYCLOPEDIA

CHICAGO - An employee of the Encyclopedia Britannica, disgruntled at having
been laid off, apparently committed computer sabotage by altering portions of
the text being prepared for updated editions of the renowned encyclopedia.

The discovery has prompted an exhaustive check to ensure that painstaking work,
in the words of the editor, "is not turned into garbage."

"We have uncovered evidence of deliberate sabotage in the EB computer files,"
editor-in-chief Tom Goetz disclosed in an Aug. 28 memo to editorial personnel
at the chicago headquarters of the oldest continually published reference work
in the English language.

The unidentified former employee has confessed and is helping undo the damage,
a spokesman said, although the company may press criminal charges.  He said the
44-million word 1987 edition is safe, but employees are believed to be laboring
overtime to catch alterations that could find their way into the 1988 edition.

Among the former employee's more vivid changes, sources said, was changing
references to Jesus Christ to Allah, the Moslem name for God.

Goetz declined to comment Thursday other than to say, "Everything is under
control."  Another industry executive said, "In the computer age, this is
exactly what we have nightmares about."

In the first of three memos to editorial staffers, Goetz wrote, "What is 
perhaps most distressing for each of us is the knowledge that some of our hard
work has been turned into garbage by either a very sick or a very vicious 
person."

At the time, he said that the actions constituted a crime under Illinois law,
that the company planned to pursue legal actions "vigorously" and that it was
issuing new computer passwords to employees.

In a staff memo dated Wednesday, Goetz informed employees that "we have
successfully concluded the matter of the sabotage of the encyclopedia's
data base.

"The 1987 printing is secure," Goetz stated.

The publication first was alerted to a problem, sources said, when a worker
scanned the computer data base and discovered the clearly odd insertion of
the names of a company executive and a private consulting firm apparently

   [There are several problems in believing that this audit-trail approach
    is fool-proof.  First of all, it relies on a password.  Masquerading is
    therefore a concern.  The second is probably more important — any
    self-respecting system programmer or cracker is probably able to alter
    the audit trail.  It is dangerous to assume that the only disgruntled
    employess are those who are NOT computer sophisticates... PGN]


F-16 software

<rti-sel!dg_rtp!throopw%mcnc.csnet@CSNET-RELAY.ARPA>
Fri, 5 Sep 86 13:19:25 edt
> It sounds very funny that the software would let you drop a bomb on the wing
> while in inverted flight but is it really important to prevent this? Is it
> worth the chance of introducing a new bug to fix this very minor problem?

>      [The probability is clearly NONZERO.  It is very dangerous to start
>       making assumptions in programming about being able to leave out an
>       exception condition simply because you think it cannot arise.  Such
>       assumptions have a nasty habit of interacting with other assumptions
>       or propagating.  PGN]

It is also dangerous to start making assumptions about the ways in which
the system will be used.  Can you really not think of a reason why one
would want to "drop" a bomb while the dorsal surface of the plane points
towards the planet's center (a possible interpretation of "inverted")?
I can think of several.

I am trying to make the point that the gross simplification of
"preventing bomb release while inverted" doesn't map very well to what I
assume the actual goal is: "preventing weapons discharge from damaging
the aircraft".  This is yet another instance where the assumptions made
to simplify a real-world situation to manageable size can easily lead to
design "errors", and is an architypical "computer risk" in the use of
relatively simple computer models of reality.

In addition to all this, it may well be that one doesn't *want* to
prevent all possible modes weapons discharge that may damage the
aircraft...  some of them may be useful techniques for use in extreme
situations.

   The more control,
   The more that requires control.
   This is the road to chaos.
                                --- PanSpechi aphorism {Frank Herbert}

Wayne Throop      <the-known-world>!mcnc!rti-sel!dg_rtp!throopw


Arbiter failures and design failures

Martin Harriman <MARTIN%SRUCAD%sc.intel.com@CSNET-RELAY.ARPA>
Fri, 5 Sep 86 09:38 PDT
Bob Estell raises two quite different failure mechanisms in his message.
The first mechanism he mentions is the well known problem of making a
reliable arbiter; he then goes on to discuss the quite different problem
of design errors in hardware, microcode, or systems software.

The arbiter problem is well known; fundamentally, there is no absolutely
reliable way to sample asynchronous signals in a synchronous system,
though there are ways of greatly reducing the probability of failure.
In this sense, no computer which incorporates asynchronous interrupts
is deterministic, since you can not predict its behavior cycle by cycle.
It is important to take this effect into account in the design of
the system and any software which cares about the timing of these external
events.

There are other interesting failure modes, where the arbiter essentially
says "maybe," instead of giving a clear yes or no answer; careful
circuit design can reduce the probability of these failures to one
failure every few thousand years (at least according to our last set
of simulations).

The bulk of Bob's message is a discussion of the probability of design
bugs.  Anyone who has seen the errata sheets for a microprocessor
or the ECO history of a mainframe will know that computer hardware
is imperfect.  This may be news to some computer programmers, but
there is such a thing as a computer error; for instance, the first
stepping of Intel's 80186 was convinced that the product of two
negative numbers was negative.

  --Martin Harriman (martin%srucad@sc.intel.com)
    Intel Santa Cruz


Systems errors (hardware AND humans)

Bill Janssen <janssen@mcc.com>
Fri, 5 Sep 86 18:07:08 CDT
Bob Estell's note on machine errors made me think of an error that
I found some years ago.  I was writing a C program that, among other
things, provided a virtual connection between two serial ports.  The
code looked something like this:

    while (in_connection_mode)
        {
        if (input_available(port1))
            port2->output = port1->input;
        else if (input_available(port2))
            port1->output = port2->input;
        }

where `port1' and `port2' were pointers to register banks on the
serial port controllers.  When we tried it out, it didn't work.  To
make a long story short, it didn't work because assembly code for
"port2->output = port1->input" was produced very efficiently as
(something like) "MOVB 4(A4),8(A5)", which would still have been
OK, except that both serial ports were on the same chip and the chip
needed a recovery interval after doing a read before doing a write.
Working code used the line "port2->output = temp = port1->input", to
introduce a slight delay!!

Now, where's the source of the error here?  What bugs me is that you
can PROVE that the (non-functional) code functions properly... if you
ignore the hardware quirks, which aren't documented.  And if the
compiler produced less efficient code (load register; store register)
the HLL code would work.  And if the machine architecture didn't have
memory-to-memory move instructions the code would work.  And if the
computer clock was slower, the code would work.  I tend to think that
the error was in the characterization of the hardware, which described
the two serial ports as independent.  But perhaps the error is
actually in not VERIFYING the hardware characterization...

Bill
--
 Bill Janssen, MCC Software Technology
 9430 Research Blvd, Austin, Texas  78759
 ARPA:  janssen@mcc.com            PHONE:  (512) 339-3682
 UUCP:  {ihnp4,seismo,harvard,gatech,pyramid}!ut-sally!im4u!milano!janssen


Re: Terminal (!) lockup

Roy Smith <cmcl2!phri!roy@seismo.CSS.GOV>
Fri, 5 Sep 86 18:23:34 edt
    I wonder: How common is this property [being able to break it
    by pushing the wrong combination of buttons] of terminals (or
    other equipment)?

    We have some CTS-2400 auto-dial modems that let you set all sorts
of parameters that get stored in eeprom.  It's not too hard to set it up so
it doesn't echo and doesn't produce any output at all.  This condition
persists even after power-cycling.  It's not really dead, but unless you
realize what you did and know the magic sequence to turn back on echoing
and command processing, it sure looks that way (if it looks like a duck and
quacks like a duck ...)

    Take a typical time-sharing system, erase the boot block from disk
and turn it off.  You've sure done a nice imitation of breaking it (I
consider having to toggle in a binary boot program as very much akin to
opening up a terminal to fix it).  If you've got a writeable control store,
you could mess yourself up even more.

    The (clever) people who designed the Apple LaserWriter must have
been thinking along these lines.  There are 2 serial interfaces on the LW.
You can run a little PostScript to change the baud rate (stored in eeprom)
on either or both.  If you want to disable one interface, you just set its
baud rate to 0.  According to the documention (I've never tried it :-)) it
won't let you set both channels to 0 baud.  If you could, there would be no
way to talk to it short of yanking the eeprom.

Please report problems with the web pages to the maintainer

x
Top